Even though Postgres is a pretty good database, for any given hardware there is some number of rows that will break it. I don't expect anything less out of LLMs.
There's a much deeper issue with CoT and such that many of the domains that we are interested in reasoning over (engineering, science, finance, ...) involve at the very least first order logic + arithmetic which runs into problems that Kurt Godel warned us about. People might say "this is a problem for symbolic AI" but really it is a problem with the problems you're trying to solve, not a problem with the way you go out about solving them -- getting a PhD in theoretical physics taught me that a paper with 50 pages of complex calculations written by a human has a mistake in it somewhere.
(People I know who didn't make it in the dog-eat-dog world of hep-th would have been skeptical about that whole magnetic moment of the muon thing because between "perturbation theory doesn't always work" [1] and "human error" the theoretical results that were not matching experiment were wrong all along...)
[1] see lunar theory
zdw•7mo ago
> there is some number of rows that will break it. I don't expect anything less out of LLMs.
I'd expect better than 8 disk towers of Hanoi, which seems to be beyond current LLMs
PaulHoule•7mo ago
That's what, 255 moves? A reasonable way to do that via CoT would be for it to determine the algorithm for solving it (which it might "know" because it was in the training data, or perhaps it can look up with a search engine, or perhaps it can derive it) and then work all the steps.
If it has a 1% chance of making a mistake per step, which is likely, because the vector space data structure isn't the right structure to represent the problem, from the viewpoint of ordinary software, it has about an 8% chance of getting the whole thing right. I don't like those odds.
On the other hand, most LLMs can write a decent Python program to solve Hanoi, such as
def tower_of_hanoi(n, source, target, auxiliary):
if n == 1:
print(f"Move disk 1 from {source} to {target}")
return
tower_of_hanoi(n - 1, source, auxiliary, target)
print(f"Move disk {n} from {source} to {target}")
tower_of_hanoi(n - 1, auxiliary, target, source)
(thanks Copilot!) and if you (or it) can feed that to a Python interpreter there is your answer, unless N is so big it blows out the stack. (One of my unpopular opinion is that recursive algorithms are a lower teaching)
I wouldn't expect most humans to get Hanoi right at N=8 unless they were super-careful and multiple-checked their work. Something I learned getting a PhD in theoretical physics is that even the best minds won't get a 50-page calculation right unless they back it up with unit and integration tests.
zdw•7mo ago
I would posit that solution is just regurgitation, not actual thinking.
Then again, is teaching an actual person how to use the quadratic formula equivalent to reinventing it from nothing?
I wonder if that's what we're doing with AI - giving it a corpus of strategies, when it has no way of being lead along a though process as a human would, if it's even capable of following along.
PaulHoule•7mo ago
There's a much deeper issue with CoT and such that many of the domains that we are interested in reasoning over (engineering, science, finance, ...) involve at the very least first order logic + arithmetic which runs into problems that Kurt Godel warned us about. People might say "this is a problem for symbolic AI" but really it is a problem with the problems you're trying to solve, not a problem with the way you go out about solving them -- getting a PhD in theoretical physics taught me that a paper with 50 pages of complex calculations written by a human has a mistake in it somewhere.
(People I know who didn't make it in the dog-eat-dog world of hep-th would have been skeptical about that whole magnetic moment of the muon thing because between "perturbation theory doesn't always work" [1] and "human error" the theoretical results that were not matching experiment were wrong all along...)
[1] see lunar theory
zdw•7mo ago
I'd expect better than 8 disk towers of Hanoi, which seems to be beyond current LLMs
PaulHoule•7mo ago
If it has a 1% chance of making a mistake per step, which is likely, because the vector space data structure isn't the right structure to represent the problem, from the viewpoint of ordinary software, it has about an 8% chance of getting the whole thing right. I don't like those odds.
On the other hand, most LLMs can write a decent Python program to solve Hanoi, such as
(thanks Copilot!) and if you (or it) can feed that to a Python interpreter there is your answer, unless N is so big it blows out the stack. (One of my unpopular opinion is that recursive algorithms are a lower teaching)I wouldn't expect most humans to get Hanoi right at N=8 unless they were super-careful and multiple-checked their work. Something I learned getting a PhD in theoretical physics is that even the best minds won't get a 50-page calculation right unless they back it up with unit and integration tests.
zdw•7mo ago
Then again, is teaching an actual person how to use the quadratic formula equivalent to reinventing it from nothing?
I wonder if that's what we're doing with AI - giving it a corpus of strategies, when it has no way of being lead along a though process as a human would, if it's even capable of following along.