1. Intent Detection Per Message
Every message is classified to decide the retrieval strategy:
- conversational (greetings, small talk) → skip RAG entirely
- document_search (explicit knowledge queries) → always retrieve
- complex/exploratory → adaptive retrieval
- tool_required → route to MCP tools instead
2. Adaptive Retrieval
When retrieval is needed, the system adjusts based on query characteristics:
- Similarity thresholds: 0.45 for general queries → 0.75 for exact data requests
- Result limits: 15 for specific queries → 30 for broad exploration
- Strategy: semantic-only vs. hybrid vs. temporal-boosted
"What was Q3 2024 revenue?" triggers exact_match mode at 0.75 threshold, while "Tell me about the company" uses standard semantic search at 0.45.
Full conversation history is passed to the model without truncation—modern LLMs have 100K-200K token windows, and RAG context is freshly curated each turn rather than accumulated.
4. Not Implemented (Yet)
- Conversation summarization/compression
- Explicit context window management
- "Forget previous context" mechanisms
For example, ”RAG context is freshly curated each turn rather than accumulated” is nonsensical: it implies that context is curated instead of replaced, but also that the previous context is forgotten and replaced, and neither means that full conversation history (including context) is passed without truncation.
Also, no mention of tool-based retrieval. I have a hunch this is not a real product, just AI slop. Prove me wrong.
That said, let me clarify the points you raised. On "freshly curated each turn rather than accumulated": this is actually the correct behavior, though I can see how the phrasing was confusing. The key is that conversation history and RAG context are two different things. The full conversation history... all previous user and assistant message... is always passed to the model without truncation. But the retrieved document chunkz are a separate layer. Those get requeried based on the current message and injected into the system prompt fresh each turn. Previous RAG results dont accumulate, they are replaced with whatever is most relevant to the current query.
Why this design? Say you ask about 'Q3 revenue' and then follow up with 'What about employe benefits?' If the system accumulated chunks, you'd have Q3 revenue data polluting the context when you're now asking about something completely different. Fresh retrieval ensures the context stays relevant to what you're actually asking about right now.
On tool-based retrieval: the system does have this through MCP integration. Query classification detects when tools are needed (Gmail, Calendar, Drive, Zapier, etc.) and routes accordingly. The model receives both the RAG context and access to tools, so it can use document knowledge and execute external actions within the same response when needed
Let me know if you have any other questions. I’ll be more than happy to answer, with the help of my ChatRAG instance, any other doubts you may have !
carlos_marcial•2mo ago
After a lot of trial and error, I settled on this tech stack for ChatRAG:
Frontend
- Next.js 16 (App Router) Latest React framework with server components and streaming
- React 19 + React Compiler: Automatic memoization, no more useMemo/useCallback hell
- Zustand: Lightweight state management (3kb vs Redux bloat)
- Tailwind CSS + Framer Motion: Styling + buttery animations
- Embed a chat widget version of your RAG chatbot on any web page, apart from creating a ChatGPT or Claude looking web UI
AI / LLM Layer
- Vercel AI SDK 5 – Unified streaming interface for all providers
- OpenRouter – Single API for Claude, GPT-4, DeepSeek, Gemini, etc.
- MCP (Model Context Protocol) – Tool use and function calling across models
RAG Pipeline
- Text chunking → documents split for optimal retrieval
- OpenAI embeddings (1536 dim vectors) – Semantic search representation
- pgvector with HNSW indexes – Fast approximate nearest neighbor search directly in Postgres
Database & Auth
- Supabase (PostgreSQL) – Database, auth, realtime, storage in one
- GitHub & Google OAuth via Supabase – Third party sign in providers managed by Supabase
- Row Level Security – Multi-tenant data isolation at the DB level
Multi-Modal Generation
- Use Fal.ai or Replicate.ai API keys for generating image, video and 3D assets inside of your RAG chatbot
Integrations
- WhatsApp via Baileys – Chat with your RAG from WhatsApp
- Stripe / Polar – Payments and subscriptions
Infra
- Fly.io / Koyeb – Edge deployment for WhatsApp workers
- Vercel – Frontend hosting with edge functions
My special sauce: pgvector HNSW indexes (m=64, ef_construction=200) give you sub-100ms semantic search without leaving Postgres. No Pinecone/Weaviate vendor lock-in.
Single-tenant vs Multi-tenant RAG setups: Why not both?
ChatRAG supports both deployment modes depending on your use case:
Single-tenant
- One knowledge base → many users
- Ideal for celebrity/expert AI clones or brand-specific agents
- e.g., "Tony Robbins AI chatbot" or "Deepak Chopra AI"
- All users interact with the same dataset and the same personality layer
Multi-tenant
- Users have workspace/project isolation — each with its own knowledge base, project-based system prompt and settings
- Perfect for SaaS products or platform builders that want to offer AI chatbots to their customers
- Every customer gets private data and their own RAG
My long term vision is to keep evolving ChatRAG so I can eventually release a fully open-source version for everyone to build with.