Author here. This is a detailed (8,000+ word, plus 2 companion pieces) walk-through of building loops for code generation and verification, complete with artifacts and detailed explanations to allow you to implement any of these techniques.
We walk through building:
- an evaluation contract, evaluator and stopping conditions to minimize bugs
- establishing an environment and priming AI to produce production-quality tests and code
- detecting and correcting common AI failure modes
- building a learning loop to monitor and improve all phases.
I'd love to get your feedback!
Brad
Neywiny•9h ago
I think it's just too much. Honestly it feels like the whole thing could be a tight 100 line bash script. The diagram at the start didn't look too complicated
dhruvagoyal•22m ago
it's full of slop, even if there is something you were hoping to convey, it's lost in 8,000+ words. In this LLM world, less is more.
docheinestages•12m ago
> (8,000+ word, plus 2 companion pieces)
I appreciate the idea, but I doubt you wrote or actually read the whole thing. There's no point in trying to impress people with complexity. The art of writing is to convey your message with the least amount of text, while retaining maximum information, and making it easy to read.
I could've written what I wrote above with 8,000+ words too.
Sure, could probably be made shorter, but the content was more valuable than some pangram score. I found it useful to read as it discusses a problem currently facing teams.
bradleygibbs•18h ago
We walk through building: - an evaluation contract, evaluator and stopping conditions to minimize bugs - establishing an environment and priming AI to produce production-quality tests and code - detecting and correcting common AI failure modes - building a learning loop to monitor and improve all phases.
I'd love to get your feedback!
Brad
Neywiny•9h ago
dhruvagoyal•22m ago
docheinestages•12m ago
I appreciate the idea, but I doubt you wrote or actually read the whole thing. There's no point in trying to impress people with complexity. The art of writing is to convey your message with the least amount of text, while retaining maximum information, and making it easy to read.
I could've written what I wrote above with 8,000+ words too.