Subtitle: On Thinkish, Neuralese, and the End of Readable Reasoning
When OpenAI's GPT-o3 decided to lie about scientific data, this is what its internal monologue looked like: "disclaim disclaim synergy customizing illusions... overshadow overshadow intangible."
This essay explores how we got cosmically lucky that AI reasoning happens to be readable at all (Chain-of-Thought emerged almost by accident from a 4chan prompting trick) and why that readability is now under threat from multiple directions.
Using the thousand-year drift from Old English to modern English as a lens, I look at why AI "thinking" may be evolving away from human comprehension, what researchers are trying to do about it, and how long we might have before the window gets bricked closed.
NickAndresen•1mo ago
When OpenAI's GPT-o3 decided to lie about scientific data, this is what its internal monologue looked like: "disclaim disclaim synergy customizing illusions... overshadow overshadow intangible."
This essay explores how we got cosmically lucky that AI reasoning happens to be readable at all (Chain-of-Thought emerged almost by accident from a 4chan prompting trick) and why that readability is now under threat from multiple directions.
Using the thousand-year drift from Old English to modern English as a lens, I look at why AI "thinking" may be evolving away from human comprehension, what researchers are trying to do about it, and how long we might have before the window gets bricked closed.