Hey HN,

We built RapidFire AI, an open-source framework that lets you compare dozens (or hundreds) of RAG and context engineering configurations in parallel, without needing a GPU cluster.

The problem: Tuning a RAG pipeline means experimenting with chunk sizes, embedding models, retrieval strategies, reranking thresholds, prompt schemes, generator models, and more. With traditional tools, you run these sequentially, wait for each to finish on the full dataset, and then compare. That's painfully slow and wastes tokens/compute on configs you'd have killed after seeing the first 10% of results.

Our approach: RapidFire AI shards your eval dataset and schedules all configs one shard at a time, cycling through them with efficient swapping. You get running metric estimates with confidence intervals in real time, based on online aggregation from the database systems literature. Spot a bad config early? Stop it. See a promising one? Clone it and tweak knobs on the fly, no restart needed.

On a beefy machine you can comfortably run 100+ configs in a single experiment. Want to see it in action without installing anything? We have a Google Colab tutorial that runs 4 RAG retrieval configs in parallel on a free Colab GPU, zero local setup, under 2 minutes to get started. It builds a financial Q&A pipeline on the FiQA dataset, grid-searches over chunk sizes and reranker settings, and shows live metrics with confidence intervals as the configs run. If you're only calling OpenAI or other closed APIs, you don't even need a GPU at all.

Colab: https://colab.research.google.com/github/RapidFireAI/rapidfi...

Key features:

- pip install rapidfireai, pure Python, works on CPU-only, single-GPU, or multi-GPU

- Wraps LangChain for retrieval/reranking, supports vLLM + OpenAI for generation

- Interactive Control (IC) Ops: stop, resume, clone-modify configs mid-run from a dashboard or notebook

- Online aggregation with confidence intervals for statistically informed early-stopping decisions

- Grid search and random search over any knob (chunk size, top-k, reranker model, prompt template, generator params, etc.)

- MLflow-based metrics dashboard with real-time plots

Example speedup: 16 RAG configs x 400 queries at ~10s/query takes ~18 hours sequentially. With RapidFire AI + IC Ops (stop poor performers early, clone winners), we got it down to ~4 hours on the same machine, a 4.7x improvement while exploring more configs and reaching better final metrics.

Docs: https://oss-docs.rapidfire.ai

Tutorial notebooks (FiQA financial QA, GSM8K math reasoning, SciFact claim verification): https://github.com/RapidFireAI/rapidfireai/tree/main/tutoria...

We'd love feedback on what knobs/integrations matter most to you. Happy to answer questions here.

The path to room-temperature superconductivity: A programmatic approach

A look inside Dialector, filmmaker Chris Marker's chatbot from 1988

U+237C ⍼ Right Angle with Downwards Zigzag Arrow Is a Symbol for Azimuth

GitLab Active Incident

Strait of Hormuz closure can become tipping point for global economy

UK Lords Back Facial Recognition Overreach, Protest Crackdown Powers

RCE in Your Test Suite: How AI Agent Skills Bypass Every Skill Security Scanner

Some Simple Economics of AGI

Show HN: Ink – Deploy full-stack apps from AI agents via MCP or Skills

Do animals have a future on Hollywood sets?

Education research: How students use AI-powered hints in programming courses

Temporal: The 9-Year Journey to Fix Time in JavaScript

I quit Rails core 4 years ago, here's what I've been up to

The T1 Trust – Building a new PRR T1 locomotive from the original plans

Turn Any Excel, CSV or Data File into an Interactive Dashboard in 5 Seconds

Show HN: Run 100 RAG experiments in parallel, even on a single GPU

I believe in SOTA models over custom ones

Modern wealth is a parlour game played by the well fed

Show HN: GladeKit – AI agent for Unity game development

Show HN: Another SQLite editor in browser powered by WASM and AI

Why are languages spoken at different speeds?

NASA's next X-ray mission, AXIS, has been killed

"If it sounds literary, it isn't": deceptively simple rules behind good writing

OpenAI's Race to Catch Up to Claude Code

QORA-LLM-2B – Pure Rust ternary inference, no multiplication needed

Ayar Labs, Wiwynn to cram 1,024 GPUs into photonic system

It's weird to have a skull full of poison

Let's Talk Knowledge Engineering

Show HN: Canonry – Open-source AEO monitor (track how AI engines cite you)

Elevated errors on login with Claude Code

Show HN: Trackless Links – clean URLs, no tracking (Safari)