Modern neuroscience suggests that everything the human brain might be doing is basically a kind of predictive processing, i.e. hallucination based on inductive biases. I do not think this is the main bottleneck.

RAG is a prompting technique, how could they possibly incorporate it into the pre training?

Why would be a proper documents-at-hand based inquiry be «simple».

Information is at paragraph #1234 of book B456; that paragraph acquires special meaning in light of its neighbours, its chapter, the book. Further information is in other paragraphs of other books. You can possibly encode with some "strong" compression information (data), but not insight. The information that a query may point to can be a big cloud of fuzzy concepts. What do you input, how? How big should that input be? "How much" of the past reflection does the Doctor use to build a judgement?

RAG seems simple because it has simpler cases ("What is the main export of Bolivia").

Well, even if we assume for a moment that we aren’t talking about non-public data…

Then RAG which serves up knowledge already in the model’s pretraining data is still useful, because it primes the model for the specific context with which you want to engage it. I maybe can see what you are saying, like why can’t the model just do a good job without being re-reminded? But even in that sense, any intelligence, artificial or otherwise, will do better given more context.

And that ignores the reality of data outside the model’s pretraining corpus, like every single business’ internal data.

One problem is in datasets which use RAG. Training foundational model requires a lot of samples, and there aren't many. The only option is to use other models to generate, so called distillation.

BTW, RAG is similar to web search. Models can do it. Web server for RAG can be implemented.

This made me think – the fact that the underlying rule that nouns e.g. representing activities such as cooking can be formed from the corresponding verb via the suffix -er breaks in this case is just a historical / cultural artifact of languages, a type of completely unnecessary complication from the machines' standpoint. Maybe LLM hallucination might also partially be caused by this exception-based social modelling forced via our training data into all model architectures?