Right now there is a focus on large models where AI gets smarter as the parameter count and training time increases. There is a plateau, but the gains, while incremental, represent gains.
What exists in the datacenter cannot exist on my laptop, TV, phone, watch, and glasses. The RAM and GPU processing is not there, and thus we must go back to creating smaller models.
Thoughts, questions, and useful disagreement go here.