Backstory: At work I had to build an AI pipeline to run millions of prompts. First I just put the prompts into string consts and integrated directly with api, chaining one run onto the output of another – but it quickly became a maintenance nightmare. Iterating on prompts, testing them over datasets, experimenting with different chaining did not fit into the regular sdlc and running them at our scale was quite difficult as most of the time is spent on waiting for the api response while holding on to dear server resources. So we started using PromptLayer in order for the prompt engineer to build evals there, edit-improve prompts outside the code and chaining them together however he likes, and we were also running them on their infra. But the UI was cumbersome and the latency was disappointing and given other issues and having found no better alternative it triggered me into starting to build a better alternative.

One year and countless nights and weekends later I have what I like to call an IDE for AI, or more like a full DevOps platform as it goes from development of prompts and workflows through testing them all the way through the running infra. I tried to squeeze every nanosecond out of the stack, hyper-optimising every part of it to have as little overhead above the AI calls themselves as possible.

Docs: https://docs.promptjuggler.com App: https://promptjuggler.com

I built this with love and it's my first post here, so please be gentle (: