Now less is more.

This only works either if you are living in the future where AGI already happened, or if you're trying to work with extremely mundane things. My experience is the complete opposite, current models produce hot garbage when trying to familiarize themselves with the codebase. That's precisely the reason I'm trying to correct them by adding manual constraints. Not vice versa. Predicting what's going to be important and what's not seems to be a major bottleneck for them. They're easily overwhelmed.

It feels to me like the discretion of the humans is the main problem in your solution space example.

Some tasks require more structure than others. Sometimes, the constraint a human faces is “you must do the work in this module or on this function”.

If the prompt says “only change this function” then the human has a good reason for that. If the LLM can’t figure it out then letting them cook on it feels like a waste of my time and tokens.

Too often I’ve sent Opus or Fable now on what I thought was a simple task to find it chasing its tail.

Fortunately I find more success than tail-chasing with the LLMs but I’m dubious of any suggestion to just let them cook.

However, a solution to the tail-chasing for certain could be to intentionally expand the solution space and to consider how my prompts might be restricting the LLM’s options.

Enjoyed the blog thanks for sharing!

* Don't do ad-hoc fixes. Don't mutate the production data just to fix an issue. Fix the bug, ingest the bad data in code if necessary, or write a migration/repair script that an ops agent can run. * Write infrastructure as code, don't rely on remembering complex manual workflows. * Make sure there's a single model in the app with a single source of truth. Don't let truth get spread out across representations, it will get out of sync and bugs will result. Derived truths can be projected from the single truth, but they must update when the truth updates. * Write all logic in the application server, and send only a minimal visible payload to the web client. No calculations, except those needed for rendering the view, should happen in the client.

It goes on. These are all driven from the model making mistakes and having to have it go back and refactor to correct them.