It is the scary thing actually. Cause once AI makes arguments that require a working memory of hundred items, then we as humans will have no way of understanding the arguments…
We can decompose and write things but only up to a point. when Ai can have a working memory that spans hundreds of books, we are necessarily going to have to trust the system.
That doesn't follow. We could still understand it just by studying it and committing it all to long-term memory, it just takes longer. And there's a hard cap on the working memory of LLMs, due to the quadratic scaling cost of the full attention layers that have proved unescapable for all SOTA LLMs.
But was it?
The age of humans comprehending things is coming to an end: our brains just won't have the capacity to make meaningful contributions to science, math, or technology.
That's something AI companies would really want you to believe.
Why would I care what they want me to believe?
Intuitively it would make sense that you can put math ability on a chart with a value for “general public” “smart high schooler” “smart undergrad” “smart PhD/ professional”. And you could place frontier AI somewhere on that chart over time from GPT 2 to now and see the trend.
Then you’d have to consider that either you believe there is a fundamental limit that is below peak human mathematician level or there’s not.
Your timelines are a bit unambitious. There's nobody expecting to make significant progress with a week of work.
Theres going to be this field day of low-hanging fruit that ML can round up, but after that I suspect it will be in fits and starts as a “connection maker” rather than some proof producer.
Yeah, as expected, an article about AI that's at the very least been polished using AI. For fucks sake we need an LLM flag to filter out slop.
Trying to convince us that mathematics and software engineering are "solved" is getting very tiring.
The pushback would probably be too much for the soon-to-be IPO-ed companies.
This has major implications that haven't been fully realized yet. On the math side, there are long machine generated proofs. On the code side, there are high volumes of code with similar code not being folded into functions.
Computers are simply better at math now, like in chess or go!
Has anyone tried feeding all of human knowledge to an LLM prior to Einstein's work and tried to have it reinvent physics?
How would you not care? Are you a robot?
They can say random stuff with the goal of increasing their shareholder value. Things they spit out do not have to be true. It is not easy to verify things they say, therefore, everything they say should be taken with a huge grain of salt.
I agree.
> The age of humans comprehending things is coming to an end: our brains just won't have the capacity to make meaningful contributions to science, math, or technology.
I don't know if I see this being true for quite a while, if ever.
There's an infinite space of possible statements and proofs. The only thing that makes certain proofs significant is that human mathematicians consider them significant; if AI came up with a proof of some statement that no humans could understand then no humans would bother investing further resources in building upon it, for the same reason we don't waste computational resources iterating over the infinite space of true statements in first-order logic.
This is not to say that a human couldn't understand a streamlined version or that the AI would not be better if it made more streamlined statements to begin with.
(I am not saying that everything mathematical that an AI produces is in any sense trivial.)
Math is not magic, a proof is just a series of applications of a set of rules on some axioms. A mathematician could understand any proof given enough time to study it; the only way for AI to make proofs that a human couldn't understand is by making really, really long proofs.
As to your second point, Terry Tao already has an answer [1]: the proof isn’t the contribution, shared understanding is. This issue was already raised back when the four-colour theorem was proved. Machine proving and machine proof checking are useful tools but they don’t mean anything without the interpretative work and the communication necessary to build shared understanding.
Why is it so hard to imagine we can build tools to think thoughts we can't comprehend?
If there's commercial value, I think it's inevitable. We don't fund mathematicians because it's cute when they understand a problem, but because their work tends to have applications with commercial value. The value can be captured without understanding the details.
Humans can’t compete with AIs on vastness of material they are familiar with, or the depth of effort they are willing and able to throw at a problem.
But scale isn’t the only aspect of difficult scientific endeavours. There’s also theory. And advancements sometimes come through hard graft of knotting together many things. And sometimes they come through the revelation of a deeper truth, or a new framework, a fundamental insight.
AI might help us reach the next level. But that doesn’t mean we won’t understand anything. It could be we have periods of vast intricacy we cannot follow, punctuated by profound elegance we (or at least experts) relatively easily can. And then the scaffolding we needed to get there falls away.
The tools we built to replace muscles have mostly obsoleted raw strength for tasks like excavating earth.
There's no reason to think we can't do the same for brains. And then we'll never need to think for a living again. Some people may want to do it as a commercially insignificant hobby, of course, the way people lift and compete in strongman competitions today.
We'll have AI taking care of our needs, the way a good mother takes care of their children.
For this to actually work in a way that benefits our species, humans will need to become something else/next through their interaction with the technology.
Anyways, sipping wine on the beach and doing puzzles when I feel like sounds nice.
I think perfect rationality doesn't exist, because it is rational to reject something that you don't understand. So rationality of a given physical system will always be bounded.
People go whole lives without being able to make it pan out.
https://en.wikipedia.org/wiki/J._Robert_Oppenheimer#:~:text=...
Law and medicine are fundamentally harder fields to obtain decent training data for, and LLM results are therefore expected to be less powerful. Also, making mistakes in these fields is costly, but perhaps you were alluding to that already.
user982•45m ago
LoganDark•44m ago
dgellow•33m ago
grebc•28m ago