A few days ago I shared the V1 of PromptKelp [0], a tool I built to solve continuous improvement for my AI agent prompts. I wanted to share a quick update on how the "dogfooding" experiment is going, because it’s reached a compelling place for me.

Here's the loop I run multiple times per day now: - Pull up a prompt in promptkelp and evaluate it. - Make one of the improvements it suggests. - Upload the latest samples of iteractions from testers and dogfooders. - PromptKelp finds user frustrations and their root cause. PromptKelp identifies a fix. - Update the AI prompt with the fix. - Click "Save" to instantly ship with confidence.

I’ve reached the point where I genuinely can't get by without it. I can't imagine writing a prompt for any production AI system without this. It would be like shipping code without a compiler.

I’m finding so much value in this workflow that I’m hoping it resonates with others who are tired of blindly iterating on their prompts.

If you’re building LLM-heavy apps, I’d love to know: how are you handling the bridge between "prompt design" and "production code"?

Fun Meta-Development: I’ve officially moved PromptKelp’s own system prompts and LLM-based logic (like the Evaluator agent) into a standard "Pro" PromptKelp account. Initially, I was just using the UI to version-control things. But now, I’m using the PromptKelp API to fetch the latest production prompts directly into the app's codebase.

Link: https://promptkelp.com

[0] Previous post: https://news.ycombinator.com/item?id=46495499