Hi! I wrote Doctor because I keep struggling with grounding on docs when working with agentic code editing (ex Roo, Claude Code).
Doctor uses crawl4ai to crawl websites, and then chunks and embeds them with langchain + litellm + openai, and finally stores all the vectors in duckdb. This allows your LLM to query the docs using semantic search over MCP, giving it grounded and up-to-date information for the things you're working on.
It requires an OpenAI key for the embedding process, but I'm working on giving users options in the future (different providers, local embedding using something like DPR or other transformer libs, etc.)
downrightmike•3h ago
And now we have a new bot for cloudflare to send to the infinite AI crawler labyrinth
kixpanganiban•3h ago
Doctor uses crawl4ai to crawl websites, and then chunks and embeds them with langchain + litellm + openai, and finally stores all the vectors in duckdb. This allows your LLM to query the docs using semantic search over MCP, giving it grounded and up-to-date information for the things you're working on.
It requires an OpenAI key for the embedding process, but I'm working on giving users options in the future (different providers, local embedding using something like DPR or other transformer libs, etc.)