I understand the hype but I wonder: what are the use cases for this kind of model? Could it be used in the context of coding agents, or is it more relevant in totally different situations?
For coding agents it'd only be useful in a subset of situations. E.g. you could imagine using one to classify bash tool calls into safe and unsafe for example.
Yeah, it could. The most obvious usage would be to have local fast cheap "feedback" / "control" over a slower more expensive agent (i.e. cc / codex / opencode). Things like "goals" could now be split from a long prompt into "actions" and "verifiers". Where for each action you also produce a verifier. Then after each action you run the verifier w/ this kind of "universal classifier" and decide if the step was done correctly, if it needs follow-up and so on.
Example: implement auth in this repo -> llm_plan() -> for item in plan generate_verifier() -> for item in plan implement() ; verify() ; accept() / followup().
Verifiers could be something like this. take a plan item as input, generate classification questions that might verify the task "is this following project conventions?" | "is this touching files from other tasks?", etc.
You can do that with LLMs, but some things might become cheaper / faster. And you can pretty much use it to check against an ever growing list of conventions. Yours or project specific.
prompt_eval=244 ms wall=245 ms schema_cache=hit generated=0
Move limit reached after 200 moves: score=16, length=19.
So, if a 12B dense model can offer this latency on a local old PC, then definitely you can scale it up with more powerful machines and get even lower latency.
My understanding is: it takes text input and it does one shot classification (no training data)
dunlin•52m ago