No.

1. "Grokking" was shown on 4-digit modular arithmetic with a 1-layer transformer; this article extrapolates it to AGI and a $10B training run with exactly zero intermediate evidence.

2. The "small dataset" is 25 trillion tokens - literally the size of current frontier training sets - but calling it small sounds revolutionary.

3. BabyLM has spent 4 years failing to produce grokking on constrained data; the paper gets a footnote saying "those models were too small," which is unfalsifiable until someone burns $10B.

4. Chain-of-thought is already empirically required for frontier performance - it's expensive, bizarre, and nobody predicted it - yet somehow we're supposed to bet the farm on a phenomenon that has never scaled past arithmetic. We need that data, even if it is just "Actually, ..."

5. If you want to chase "recurrent depth", loop transformers rumored in Mythos/Fable are at least grounded in actual engineering; grokking-at-scale is just vibes ai bro science.

More data is and will always be the answer. Why are all labs distilling from each other?

My guess is human learning requires some major differences in training.

For instance, we learn to process a firehose of visual, sound, tactile and motor information, before we start thinking in well-defined concepts.

That is a major foundation, grounding, and highly organized state, on which we learn language. And is entirely missing for LLMs.

We learn a lot metaphorically, so it may be that the physical effects we learn to recognize act as highly efficient sub-vocabulary, for quickly learning new concepts both verbal and nonverbal.

So a series of different learning stages, each making the next more efficient, is likely to be a big part of any solution.

No amount of data/exposure will help a baby trade stocks. General fast learning requires other things to be learned first.