Hi all, wanted to share a package we've open sourced called rag-select.
It’s a framework for comparing RAG configurations end-to-end (ingestion → chunking → retrieval → response) on a shared task set, rather than tuning components in isolation. The idea is to make the RAG architecture decision data/eval-driven, especially when balancing quality, latency, and cost across several possible design choices.
We’ve been using this internally to experiment with different setups and to separate RAG evaluation from application code, and it felt general enough to share.
agnim25•1h ago
It’s a framework for comparing RAG configurations end-to-end (ingestion → chunking → retrieval → response) on a shared task set, rather than tuning components in isolation. The idea is to make the RAG architecture decision data/eval-driven, especially when balancing quality, latency, and cost across several possible design choices.
We’ve been using this internally to experiment with different setups and to separate RAG evaluation from application code, and it felt general enough to share.
Repo: https://github.com/conclude-ai/rag-select
Some more details on our company page as well: https://useconclude.com/engineering/rag-select
Feel free to reach out if you're interested in using the package and have any questions or if you'd like to contribute as a developer. Thanks!