Hi all, I came across an interesting project called StateM. It is an open-source control layer for long-horizon agents, and I felt it was worth sharing here.
The basic idea is to move parts of agent control from the prompt or agent context to an externalized layer, making them explicit and enforceable. The interesting thing is what they call an “agent owning the control”: StateM uses YAML-defined runbooks and a CLI runtime to provide things like stateful checks, phase-specific context, blocking transitions, and persistent workflow state. The developing speed is great -- my agent is already using it and adapting its own runbook when it finds anything that should be updated. It even switched between runbooks when I threw it a curveball.
They achieved 95.3% on Terminal-Bench 2.1 using GPT-5.6 Sol xhigh + Adapted StateM (+ 10.4 points). It is even higher than GPT-5.6 Sol Ultra’s 91.9%. And they can use DeepSeek-V4-Flash to achieve the same performance as GPT-5.6 Sol Max on it, using 1/38 of the cost. These numbers are crazy. Is scaling the harness also an orthogonal axis to model scaling?
The benchmark is probably the less interesting part to me, but I am thinking: how important should agent harness be in agent reliability? In the long horizon tasks it seems more important than I thought.
johntrob14•35m ago
The basic idea is to move parts of agent control from the prompt or agent context to an externalized layer, making them explicit and enforceable. The interesting thing is what they call an “agent owning the control”: StateM uses YAML-defined runbooks and a CLI runtime to provide things like stateful checks, phase-specific context, blocking transitions, and persistent workflow state. The developing speed is great -- my agent is already using it and adapting its own runbook when it finds anything that should be updated. It even switched between runbooks when I threw it a curveball.
They achieved 95.3% on Terminal-Bench 2.1 using GPT-5.6 Sol xhigh + Adapted StateM (+ 10.4 points). It is even higher than GPT-5.6 Sol Ultra’s 91.9%. And they can use DeepSeek-V4-Flash to achieve the same performance as GPT-5.6 Sol Max on it, using 1/38 of the cost. These numbers are crazy. Is scaling the harness also an orthogonal axis to model scaling?
The benchmark is probably the less interesting part to me, but I am thinking: how important should agent harness be in agent reliability? In the long horizon tasks it seems more important than I thought.
Github: https://github.com/henryqin1997/statem Paper: https://arxiv.org/abs/2608.15089