This is incredible!

> Here's something most developers overlook: if an LLM has a 2% JSON defect rate, and Response Healing drops that to 1%, you haven't just made a 1% improvement. You've cut your defects, bugs, and support tickets in half.

If part of my system can't even manage to output JSON reliably, it needs way more "healing" than syntax munging. This comes across as naive.

Dear Openrouter blog authors, could you please stop writing your blogposts with LLMs?

The content of your posts is really insightful and interesting, but it's feel like junk quality because of the way LLMs write blogposts.

What was your prompt?

This is good, is there a python library to do this ?

I thought structured output was done by only allowing tokens that would produce valid output. For their example of a missing closing bracket, the end token wouldn't be allowed, and it would only accept tokens that contain a digit, comma, or closing bracket. I guess that must not be the case, though. Doing that seems like a better way to address this.

This really gets at the heart of my instinctive dislike of how LLMs are being deployed. A core feature of computers, and tools in general, is reliability. I like software because you can set something up, run it, and (ideally) know that it will do the same job the same way each subsequent time you run it. I want a button that is clearly labeled, and when pressed, does a specific thing, acting like a limb, an extension of my will. I do not, in almost all cases, want my computer to be another distinct entity that I conduct social interactions with.

Maybe people got used to computers being unreliable and unpredictable as the UIs we shipped became more distracting, less learnable, always shifting and hiding information, popping up suggestions and displaying non-deterministic-seeming behavior. We trained users to treat their devices like unruly animals that they can never quite trust. So now the idea of a machine that embodies a more clever (but still unreliable) animal to wrangle sounds like a clear upgrade.

But as someone who's spent an inordinate amount of time tweaking and tuning his computing environment to prune out flakey components and fine-tune bindings and navigation, the idea of integrating a tool into my workflow that does amazing things but fails utterly even 1% of the time sounds like a nightmare, a sort of perpetual torture of low-grade anxiety.

How do they know the output needs to be in json format?

One of the best shitposts I have ever seen, by far. Absurdism taken to its finest form.

I have various workflows for which I use Claude (sonnet 4.5 mostly) to emit json and it’s literally more common to have an API error than to get defective json out - I can’t recall it happening at all in the last six months, probably much longer. I remember a customer over a year ago that was using gpt4o and had some logic to fix json errors, so evidently it happened for them.

I recognize this is more of an issue with smaller cheaper models now, though if the model can’t make proper json, what else is it breaking.