What is a typical LLM SDK?
Check the "Who It's For"
> Solo founders who use Claude Code daily and want to use it in toolchains
I can do that just fine without a REST API
> Backend developers adding AI capabilities to services and APIs
You can interact with Claude Code just fine. In fact I wouldn't use CC here since I want control over how it's ran, and CLAUDE.md just doesn't have that power.
> AI tinkerers building agentic workflows, automation, and experiments
Ditto.
> Data engineers who need structured, typed LLM output in pipelines
This has nothing to do with the REST part. I don't see how this is better than just an SDK with structured output.
> Platform teams exposing Claude Code to internal services
I guess I see the appeal a bit here, but anthropic has a platform that can handle agents for you, and this seems like a better fit than a clunky REST API between a CLI tool.
This isn't the right tool for every job. Most agentic implementations should use something like BAML.
For people who already have workflows established in Claude Code, they can use Koine to integrate those workflows without having to decouple from Claude Code. I'll think about some updates to the README to make this more clear.
Using an SDK with direct inference from a provider (the typical pattern, right?) means writing orchestrations and prompts from scratch. Claude code has a lot of that already. Relying on Claude Code for agentic orchestration is a bit of a black box solution, but it's a fairly stable pattern for many use cases.
In the end, all Koine does is be an SDK with structured output from Claude Code. The gateway pattern makes a convenient way to call an isolated Claude Code instance.
FWIW, the official Anthropic Agent SDK doesn't provide a way to stream json responses. Most SDKs, including Koine, do.
This isn't going to solve every use case. In most places a proper SDK like BAML will be better suited.
Cheers.
PS - The Anthropic Agent SDK doesn't have native support for streaming JSON. Other SDKs like Vercel and BAML (and Koine) solve this with prompt hacking and schema validation in the client.
isoprophlex•1mo ago
matthewpetty•1mo ago
Like any good SDK, Koine provides a method for managing multi-turn sessions. Data models can use the session uuid as a primary key for storing state.