I've realized I spend a lot of time hardcoding LLM prompt tweaks and trying to determine their effectiveness from a bit of personal testing. I spend a lot of time fiddling and making not informed decisions.

I've been working on an API that runs one of X number of prompts, and can be sent back a good or bad score based on some user interaction. I can see which prompt wins and make them the dedicated prompt, or add new test ones.

This has been big for me, but does anyone else want this? I'll probably continue to scratch my own itch, but is it worth making it accessible with user accounts, billing, etc?

Lmk: https://form.typeform.com/to/WXibpEkA