llms.txt is a Markdown file you serve at /llms.txt that gives AI tools (Claude, ChatGPT, Perplexity, Gemini) a clean, structured overview of your site’s most important pages. Think of it as a curated table of contents the AI can read in seconds, with links to the pages that contain the actual answers.
Why it matters
AI tools that visit your site at request time (Perplexity, ChatGPT’s browsing, Claude’s tool use) often have only a few seconds to figure out what your site is about. A well-written llms.txt cuts that decision time and dramatically increases the chance your most important pages get read and cited.
Common mistakes
- Treating it like a sitemap. Sitemaps are for crawlers. llms.txt is for human-language understanding, write it like a friendly index, not a URL dump.
- Linking to everything. llms.txt is a highlights reel, not a complete index. 10–20 well-chosen links beat 200 bloated ones.
- Forgetting llms-full.txt. The optional companion file (
/llms-full.txt) is your concatenated body content, useful when you want the AI to have the actual text, not just links.