It’s surprisingly hard to get a complete picture of what AI can and can’t do. Most of the attention goes to a handful of benchmarks focused on coding, math, and reasoning abilities. Meanwhile, researchers, companies, and domain experts are building incredible benchmarks and evals across healthcare, law, trades, and more.
But this work is scattered across research papers, GitHub repos, model cards, websites, and tweets.
We built BenchmarkList to bring it >2.4k benchmarks in one place and map the jagged frontier of AI.
With BenchmarkList you can: - Track new benchmarks and models. Follow a continuously updated feed of new evals, models, and results. - Compare model capabilities. See benchmarks tested on a specific model and compare it to other SOTA or open weight models. - Go deeper with our research. Explore dashboards tracking AI progress on human work. Read through deep dives and visualizations created by our team.
This is only the beginning. Our goal is to help researchers better understand and measure AI abilities and we are excited to serve the community with this public resource.