I'm building a SaaS platform in Sri Lanka that handles documents and other sensitive data.

Each user can upload their own documents and information, and the platform uses RAG to answer questions based on that user's data. That part makes sense to me.

My main concern is what happens when the user hasn't uploaded enough information. I still want the LLM to provide accurate answers using reliable information from the internet (or from a curated knowledge base), with proper citations.

These are the two architectures I'm considering:

Option 1:

Base LLM (OpenAI/Anthropic via Azure AI Foundry or Amazon Bedrock) ↓ Platform RAG (global knowledge base managed by us) ↓ User-specific RAG In this approach, we maintain a global knowledge base that we (the platform admins) curate and update. Every user can access this shared knowledge, while their own uploaded documents are searched through their personal RAG.

Option 2:

Open-source LLM ↓ Fine-tuned on Sri Lankan/domain-specific data ↓ User-specific RAG Here, we fine-tune an open-source model using Sri Lankan or domain-specific data, and each user still has their own RAG for their private documents.

My concerns are:

Is fine-tuning actually the right solution here, or is it unnecessary?

Is a global/shared RAG a better approach than fine-tuning?

How would you design this architecture if you wanted:

Accurate answers from domain knowledge

User-private document search

Citations/sources

Good scalability for thousands of users

I'm leaning toward Option 1 because fine-tuning seems expensive, time-consuming, and I have no experience with it yet. However, I'm not sure if I'm thinking about this correctly.

I'd really appreciate hearing how others would approach this problem

Backpacking Sleeping Pads, Tested on the Trail (2026)

Show HN: Tiny React charts that fit in a line of text, zero deps

Does the UK's liver transplant matching algorithm exclude younger patients?

Karen Hao: AI Doesn't Have to Be Built This Way

Using ThinkPad T480 as a mobile phone

Show HN: WaitToUnlock, a website that makes your Screen Time limits unbypassable

Prompt: The Next AI Challenge Isn't the Model. It's the Organization

Ask HN: Should I Combine Global Knowledge, Internet Search, and User RAG

Lessons from Crawling the Web at Scale

Show HN: Worklog, structured memory for Agent in one SQLite table

Prototyping Can Help You to Get Buy-In

Ask HN: How to Start with LLMs/Vibecoding?

Show HN: Thel, a terminal app for long running sessions

Open Knowledge format v0.2 tackles agentic trust

Making x.com links open xcancel automatically

Zero-defects code: the prescient Microsoft memo from 1989

ECharts rendering engine with Evilcharts: No more ugly charts

Show HN: Scanner for cellular automata rule space

Local-Thirst: But Linear

Setting up containerized dev environments you can spin up in seconds

Not sure about having kids? There's a coach for that

Show HN: Democrasite – Propose and vote on changes to this website

Show HN: Digital Invitation

AI use policy for my essay writing

Robot physics that only passes in the friendly world is a lie – Dual gate

When Middle Age Starts

What if LLMs escape through inferences itself? This is fiction. For now

The Problem with Capitalism

Gatwick Airport left without running water at both terminals

This July I Was Fired from Simple AI (A Deeply YC Company)