Aw thanks! We don't currently, but from a cost perspective as a user it shouldn't matter much since it's all bundled into the same subscription (we rate-limit by requests, not by tokens — our request rate limits are set to "higher than the amount of messages per hour that Claude Code promises", haha). We might at some point just to save GPUs though!

Interesting! Very open to the idea. What open-source fill-in-the-middle models are good right now? I've stayed on top of the open source primary coding LLMs, but haven't been following along for the open-source FIM ones.

Yes! We have a standard OpenAI-compatible API, and we don't restrict subscriptions from using it (unlike Anthropic, where API keys are billed differently unless you're using Claude Code directly, or in a tool that wraps Claude Code).

I'm not deeply familiar with Mastra, but reading their docs, it looks like they use the Vercel AI SDK — which is great, since Vercel's AI SDK can work with any OpenAI-compatible API, including ours. All you need to do is set a custom API base URL; in our case, that's https://api.synthetic.new/v1

Then just plug in your Synthetic API key, and you should be able to use any supported model. For example, to use GLM-4.5, you'd pass the following model string: "hf:zai-org/GLM-4.5"

The AI SDK docs are here for using custom base URLs: https://ai-sdk.dev/docs/ai-sdk-core/provider-management

You can also join our Discord if you need help! https://synthetic.new/discord should redirect you to our Discord server :)

Oh to be clear, the API prompts/completions can't be stored longer than 14 days or used for anything other than debugging — the data retention section takes priority over everything else. I believe the other requests mentioned refer to general web traffic requests and web UI data. Thank you for the great question!

For requests, it depends on the agent framework to a certain extent. We just count API requests. For frameworks that support parallel tool calls, assuming they're using the standard OpenAI parallel tool call API, the entire parallel batch only counts as one request — since it only generated a single API request, and we just count API requests. I don't know exactly how Cursor structures it but I'd be surprised if they were making 100 API requests per message — I assume they're using the normal parallel tool call API to send all tools in a single batch, which equates to only taking 1 request of your quota in the rate limit.

I'm on plane wifi right now but I'll benchmark later today — when I benchmarked GLM-4.5, I could get 150-200tps in the Bay Area, California. Qwen3 is probably somewhat lower TBH. We have an open-source coding agent that includes a TPS benchmarker that works with any OpenAI compatible API, including ours: https://github.com/synthetic-lab/octofriend

To run the TPS benchmark, just run:

    octo bench tps

All it does is ask the model to write a long story without making tool calls (although we do send the tool definitions over, to accurately benchmark differences in tool call serialization/parsing). It usually consumes a little over 1k tokens so it's fairly cheap to run against different usage-based APIs (and only consumes a single request for subscription APIs that rate limit by request).

Edit: forgot to add — for Qwen3 everything should be running in FP8.

This is good feedback thank you. We use Google only to track ad conversions, and use a cookie page to prevent them from even running until people give consent on the cookie form, but I agree it's not ideal and I've kind of hated having it. I'll see what I can do about the privacy policy — thank you for the reference to Kagi!