After several attempts to build directly on Fly Machines and reimplement this layer in each app, I threw my hands up. There was ANOTHER missing abstraction. Thankfully Fly.io shipped Sprites.dev, which gave me the compute primitive I needed: a vm that could idle between turns without losing its disk.
Agent Client Protocol solved the communication side. Fountain uses ACP to talk to different agent harnesses and exposes ACP on the client side so users can drive Fountain conversations from ACP-enabled apps.
Fountain is the api I wanted all along. Setup an environment, spec out an agent, give it some access and then just prompt it over http. It handles the machinery of managing the sandbox, running the agent harness, and streaming the response.