The obvious path is the Grok path. That is, anybody with a big pile of money can read some papers and hire some people and make a model which is at or near the frontier. Beating current models by a hair at current benchmarks is not as hard as it looks because you will be building the system to beat those benchmarks from the beginning. [1]

Six months or a year later people will start to realize that you're not really improving or making progress though because that's something entirely different.

Now real advances in the long term are going to come out of smaller companies working on things like

https://en.wikipedia.org/wiki/Mamba_(deep_learning_architect... [2]

Like the frontier models are just too expensive to do the experimental work which will lead to advances in the science, the science is going to advance through work on little models whereas companies like OpenAI and Anthropic are very committed to maximize the performance of their existing systems in the short term and it is the intense competition that will keep them in an "Innovator's Dilemma" situation where their customers are going to reject anything really new which doesn't perform the same. [3]

[1] ... and even if you don't cheat the model will cheat for you

[2] ... not necessarily that one in particular, but something like that

[3] companies like Microsoft that "disrupt themselves" ignoring their customers are afraid of an Innovator's Dilemma situation but are paradoxically not stuck in it because they are monopolies who can force their customers to do something they don't like.