Over the last 2 years, we observed computer use models improving at a rapid pace and saturating benchmarks. This new benchmark replaces Online-Mind2Web with our own Browserbase Benchmark v2 that better represents the complex tasks that browser agents face in the real world. It runs against 23 models (frontier and open-weight) and 9 harnesses (Claude Code to LangChain Deep Agents) on accuracy, speed, and cost.
This new benchmark confirmed our belief that the choice of an harness is becoming as important as the choice of a model. For example: claude-opus-5 runs 74% at $1.50/task on LangChain deep agents but 71% at ~$10/task on fx.
Why use Stagehand when agents can write CDP / Playwright on the fly for browser use
MiguelG719•30m ago
Token efficiency, performance, observability, and most importantly: permissions/security policies
starlightttt•40m ago
Love the design
smpandya•33m ago
Are there results comparing agents running different tools (agent-browser, playwright MCP, browse CLI), or is this mostly Stagehand focused?
MiguelG719•26m ago
You can swap the driver/tool yourself with the evals CLI! Just use `—tool-surface <one-of-the-supported-tools>`. Or define your own using the interface
pranaygup12•28m ago
What model family do you find is the best for browser use overall? or does it change pretty regularly
MiguelG719•22m ago
It changes so frequently, and the world wild web is vast so it’s dependent on your use case. In general the leaderboard reflects what we see working across a broad range of domains, but the best way to tell is to define your tasks and run the evals yourself; that’s what this is for
dericdinudaniel•23m ago
Cost difference across harnesses is interesting to me.
Would love to see more info about more optimizations in the harnesses to trim down costs.
devk03•21m ago
Hyper personalized harnesses are the edge that the labs cannot beat startups on. There will be a whole entire era of new harnesses coming out soon.
dericdinudaniel•19m ago
a whole new era of custom harnesses with unnecessary stuff trimmed out sounds like the future
MiguelG719•17m ago
It still feels like the harness and the model need to co-evolve together
MiguelG719•1h ago
Over the last 2 years, we observed computer use models improving at a rapid pace and saturating benchmarks. This new benchmark replaces Online-Mind2Web with our own Browserbase Benchmark v2 that better represents the complex tasks that browser agents face in the real world. It runs against 23 models (frontier and open-weight) and 9 harnesses (Claude Code to LangChain Deep Agents) on accuracy, speed, and cost.
This new benchmark confirmed our belief that the choice of an harness is becoming as important as the choice of a model. For example: claude-opus-5 runs 74% at $1.50/task on LangChain deep agents but 71% at ~$10/task on fx.
The eval harness is a CLI you can run yourself (pick harness + tools/mcps + model, pass high-level tasks, grades with LLM verifiers, has trials/concurrency/OTEL tracing): https://github.com/browserbase/stagehand/tree/main/packages/...
Happy to get into methodology, and if you want your model or harness added, just let me know.
alyssamaru•45m ago