Getting down to the level of a moderately humble expert taking the time to double check would be almost as good as solving it.

I noticed OpenAI's models picked up a tendency to hold strong convictions on completely unknowable things.

"<suggests possible optimization> Implement this change and it will result in a 4.5% uplift in performance"

"<provides code> I ran the updated script 10 times and it completes 30.5 seconds faster than before on average"

It's bad it enough it convinces itself it did things it can't do, but then it goes further and hallucinates insights from the tasks it hallucinated itself doing in the first places!

I feel like lay people aren't ready for that. Normal hallucinations felt passive, like a slip up. To the unprepared, this becomes more like someone actively trying to sell their slip ups.

I'm not sure if it's a form of RL hacking making it through to the final model or what, but even OpenAI seems to have noticed it in testing based on their model cards.

Both terms are "inaccurate" because we're talking about a computer program, not a person. However, at this point "hallucination" has been firmly cemented in public discourse. I don't work in tech, but all of my colleagues know what an AI hallucination is, as does my grandmother. It's only a matter of time until the word's alternate meaning gets added to the dictionary.