But for planning and drafting I do need capable models, which I grab from Zenmux or similar providers, and those are heavily rate-limited. I’d get most of the way through a task and hit a limit, which meant stopping, opening the config, swapping the provider, restarting, and picking the task back up from wherever it died. Three or four times per complex task. Losing the quota was annoying. Losing my place in the task was worse.
What I settled on was putting a gateway in front of everything and pointing OpenCode at that instead of at any single provider. One base URL in the OpenCode config, and the routing happens behind it. When one provider 429s, the request goes to the next one with headroom.
I tried LiteLLM first. It works, and if your stack is already Python it’s probably the more obvious pick.
But, I recently found out about Bifrost, and switched from LiteLLM. What made it fit my setup is that it’s a single Go binary with a local SQLite config store, so there’s no separate database service to run alongside it. I’ve got 9 providers configured with fallbacks between them, which is the part that actually fixed the interruptions.
Sharing in case it helps anyone with the same problem. Took me a while to land on this.