We are a CS research team from UIUC. We just open-sourced LLMRouter, a unified library for LLM routing across different settings.
The motivation came from our own research and engineering work. Over the past year, we built and benchmarked multiple LLM routing systems, including GraphRouter (ICLR’25), Router-R1 (NeurIPS’25), and PersonalizedRouter (TMLR’25). What we consistently ran into was not an algorithmic issue, but an infrastructure one.
Today, most LLM routers come with custom input/output formats, training pipelines, and evaluation setups. This makes routers hard to reuse, hard to compare, and costly to integrate into real systems.
LLMRouter aims to standardize this layer. It provides:
Unified support for single-round, multi-round, agentic, and personalized routing
Implementations of 16+ state-of-the-art LLM routing algorithms
One-line commands to swap routers without rebuilding pipelines
Built-in benchmarking with extensible routers, tasks, and metrics
In practice, routing across a mix of large and small models using LLMRouter can reduce LLM API costs by roughly 30–50% while maintaining overall quality.
We hope LLMRouter can play a role similar to PyG for GNNs — a shared foundation that makes LLM routing research and deployment easier and more comparable.
tao2024•1mo ago
We are a CS research team from UIUC. We just open-sourced LLMRouter, a unified library for LLM routing across different settings.
The motivation came from our own research and engineering work. Over the past year, we built and benchmarked multiple LLM routing systems, including GraphRouter (ICLR’25), Router-R1 (NeurIPS’25), and PersonalizedRouter (TMLR’25). What we consistently ran into was not an algorithmic issue, but an infrastructure one.
Today, most LLM routers come with custom input/output formats, training pipelines, and evaluation setups. This makes routers hard to reuse, hard to compare, and costly to integrate into real systems.
LLMRouter aims to standardize this layer. It provides:
Unified support for single-round, multi-round, agentic, and personalized routing
Implementations of 16+ state-of-the-art LLM routing algorithms
One-line commands to swap routers without rebuilding pipelines
Built-in benchmarking with extensible routers, tasks, and metrics
In practice, routing across a mix of large and small models using LLMRouter can reduce LLM API costs by roughly 30–50% while maintaining overall quality.
We hope LLMRouter can play a role similar to PyG for GNNs — a shared foundation that makes LLM routing research and deployment easier and more comparable.
GitHub: https://github.com/ulab-uiuc/LLMRouter
Project page: https://ulab-uiuc.github.io/LLMRouter/
Happy to answer questions or discuss design decisions.