When these models screw up, and you notice immediately and point out exactly how they screwed up in simple, direct language, they will 1) explain to you at length how you are actually wrong by pretending that they originally said what you just said and that you just said something else, and 2) tell you how your misunderstanding and confusion could have made their answer seem and feel wrong to you at length.

Then you quote their answer, and repeat that it was wrong (maybe two or three times), and you get effusive praise and self-criticism at length about how the answer that you already told them was wrong was wrong, as if you needed to know that, and another explanation of the mistake or problem that you just explained to it.

At this point, the entire context is wrecked and filled with nonsense. You want to dump it and start over, but you're afraid that if you start over the same way you'll end up here again (and you do, unless you figure out the magic words.)

Why aren't they getting better at this? Are some of them getting better at this?

> "Because your question was phrased in a way that depended on which year we’re currently in, and I initially answered it generically: “If today is 2025, then 2026 is next year.” I started by explaining the conditional logic instead of immediately stating “Yes,” which made the answer sound like No at first. Then I clarified based on the actual current date (December 2025) that yes, 2026 is next year. So the short version: My first sentence emphasized the logic, not the direct answer. That caused confusion."

I seem to remember Bill Clinton sometimes also talks like this

LLM's do not think, understand, reason, reflect, comprehend and they never shall.

I have commented elsewhere but this bears repeating

If you had enough paper and ink and the patience to go through it, you could take all the training data and manually step through and train the same model. Then once you have trained the model you could use even more pen and paper to step through the correct prompts to arrive at the answer. All of this would be a completely mechanical process. This really does bear thinking about. It's amazing the results that LLM's are able to acheive. But let's not kid ourselves and start throwing about terms like AGI or emergence just yet. It makes a mechanical process seem magical (as do computers in general).

I should add it also makes sense as to why it would, just look at the volume of human knowledge (the training data). It's the training data with the mass quite literally of mankind's knowledge, genius, logic, inferences, language and intellect that does the heavy lifting.