> These results are relatively low-hanging fruit for AI; none of them required developing an extensive new theory. [...] What we have not yet seen is an AI developing a substantial new conceptual framework in order to solve a mathematical problem. Much of mathematics proceeds by identifying the objects that are truly central to a question and then developing a theory that helps us understand them. Current AIs are very strong at searching and recombining existing ideas, but they are weak at building any deep and sustained new theory. [...] This speaks to a more general limitation of current AI systems [... W]hile they have larger working memories than humans do, know more about more different things than any particular human does, and can process information faster than humans, can, true novelty is still largely beyond their reach. // Of course, that distinction may not survive for very long. [...] None of these mathematical capabilities were explicitly designed for, or planned[: t]hey’re all emergent properties.

We are not seeing enough interest on what makes those capabilities emerge. It looks like the exploitation of a natural resource - technology oriented towards extraction - instead of exploiting the occasion towards scientific progress. Which is what would drive surpassing the current limitations.

For example: in a proper Intelligence, one does not think for n-thousand virtual hours than forgets everything like a stateless ROM based 8bit personal computer. The thinking experience has changed the Intelligence. That is not part of LLMs...