Perhaps they should switch to the metric thinking system.
Gigathinking, and Terathinking should be on the menu as well.
But... I'd like you to take a moment and think really hard about whether this is truly novel behavior for LLMs, or rather something that has always been part of the interplay between inter-agent communication and intra-agent thought :)
I included that de-obfuscated code in my post: https://simonwillison.net/2025/Apr/19/claude-code-best-pract...
One of the challenges Technomages faced after acquiring their symbiote is gaining control over both the personal language and the impulses for destruction embedded by the Shadows (bias? bias).
So yes, learning the tricks of getting our AI agents to do the thing we actually meant, while aiming to overcome bias built in to the models when we can see it does sound an awful lot like Technomages dealing with their symbiote.
"Ask Claude to make a plan for how to approach a specific problem. We recommend using the word "think" to trigger extended thinking mode, which gives Claude additional computation time to evaluate alternatives more thoroughly. These specific phrases are mapped directly to increasing levels of thinking budget in the system: "think" < "think hard" < "think harder" < "ultrathink." Each level allocates progressively more thinking budget for Claude to use."
https://www.anthropic.com/engineering/claude-code-best-pract...
I don't know what the max allowable "budget_tokens" is for Claude 3.7 Thinking mode, but the SDK shows an example of 32k which matches up with the article's findings.
I'm assuming it's to unify the experience between chat and API, and avoid switching models for it ala OAI.
* https://www.anthropic.com/engineering/claude-code-best-pract...
First I asked it to help me reverse the direction of text on a circle in Photoshop. It gave me very specific instructions which don't work and continued to argue with me that I was doing something wrong - I did my own research and found it's not actually possible to do this in Photoshop, and the instructions it was giving me were for Illustrator. 30 minutes of my time wasted.
This morning I asked it how to remove the axis lines from the orthographic view in Blender 4.3. I explained carefully that I know how to remove them in perspective view but that wasn't working for orthographic views. It over and over told me how to remove them from perspective views, telling me to use non existent UI elements, even drawing ASCII diagrams of how to find the nonexistent icons. When I said they didn't exist, it would circle back to telling me how to turn them off in the perspective view.
It turns out, again, it's not possible to remove grid lines from orthographic views in Blender (at least without messing around with the theme settings, or turning off the grid entirely).
In both cases it was incredibly persistent in stating the wrong way to do things, even when I was saying that it didn't work. I felt like it was gaslighting me, moreso than with any previous model I've used.
I haven't yet used it for writing code but these two experiences don't make me feel hopeful. The worst part about dealing with AI is when they are confidently incorrect.
I don't believe they have much training material on the UI for tools at the moment - it may well come in the future as these new "computer use" models get fed vast amounts of screen capture videos, but to date my hunch is that there hasn't been much focus on that, especially for tools like Photoshop and Blender (training them to use a web browser is a whole lot more useful for the moment).
I'd encourage you not to assume they suck at code just because they suck at answering questions about Photoshop and Blender. I wrote about that a while back: "Don’t fall into the trap of anthropomorphizing LLMs and assuming that failures which would discredit a human should discredit the machine in the same way." - https://simonwillison.net/2025/Mar/11/using-llms-for-code/#s...
I use Claude for help with Blender all the time and it's amazing, for the most part. It does have an in depth knowledge of the UI and also of many specific technical ways of doing things. The main thing that it gets stuck on is UI changes between versions, and to be fair, I gets stuck on this too.
I will try this same query on Claude tomorrow when I'm in my office. I suspect it'll get it wrong as well, but it's not so much the getting wrong that I had an issue with, but how persistent Gemini was in refusing to admit error and making it seem like I was the one at fault.
Really interesting to hear that Claude does well at this kind of problem! Maybe that's thanks to training they did relative to their Claude Computer Use research last year. https://simonwillison.net/2024/Oct/22/computer-use/
I already assume that the models are shifting underneath me. It's very frustrating that most non-developers just think you can ask an LLM a question and it will respond accurately each time. They are designed to make creative output and even if you dial down the temperature they still can hallucinate.
Why not be explicit about the thinking budget instead of aliasing it to a number with a term like ultrathink?
It's a cute word, and fun to know is managed on the client side, but isn't it again more imprecision to tools the are already suffering from that?
I love that language. This is something that’s constantly bothered me from the get go. Maybe I’m just wearing a tinfoil hat, but I swear I’ve noticed variations in behavior and performance within models across time.
This has been super annoying to me.
I always use the chat interfaces (mostly Claude atm), so I guess that just puts me at the whim of which sub-version of model Anthropic is serving for the day.
Or again, maybe I’m just hallucinating.
andrewfromx•9mo ago