I just built a small web app that helps you generate /llms.txt files for your website: https://llmstxt-cyuh.vercel.app GitHub: https://github.com/mandarwagh9/llmstxt

What is /llms.txt? Think of it like robots.txt, but for large language models. It’s a new emerging standard that gives LLMs a clean, Markdown-based map of your most important content (docs, guides, APIs). Instead of crawling bloated HTML, they get straight to the point.

The idea was popularized by Jeremy Howard and is gaining adoption across devtool companies like LangChain and Mintlify.

Why I built this I wanted a dead-simple way to auto-generate /llms.txt from any docs site or URL. This tool scrapes your site, extracts key Markdown links, and gives you a copy-paste-ready llms.txt.

No setup, no install, just paste your URL and download.

Stack

Frontend: Vercel + vanilla JS

Backend: Uses Gemini API (but can swap in any LLM)

Built with a focus on minimal UI + performance

Would love feedback, feature ideas, or suggestions for edge cases to cover (like private docs, multi-language support, etc.)

Thanks!