In Keen Code, a coding agent solo-developed by me, I have implemented a task-classifier using Jev and a simple auto-router based on the classification result. The article shares details about how it's implemented.
In short, it has a very simple idea:
- For every user message, ask Jev to classify the task in one of the three categories (simple, standard, complex) given current state of the conversation.
- Keen also asks Jev to put a probability on this question: "is this a new task?"
- Based on the result, Keen uses a heuristic to switch the current model to one of the three pre-configured models by user for the three categories
- To make the router work, users have to configure one model per task category inside the coding agent itself
The heuristic is also very simple:
When routing is enabled, Keen switches only if the selected-category probability is greater than 0.5 and either:
- new-task probability is at least 0.6; or - category probability is greater than 0.8 and the target category is a higher tier than the active routed model.
Thoughts and feedback are welcome. And if you are interested to know more about Keen Code, the repo can be found here: https://github.com/mochow13/keen-code