llms.txt
llms.txt is a proposed Markdown file at a site's root that gives AI models a curated summary and link list, so they can find the right pages at inference time.
llms.txt is a proposed convention for a Markdown file served at a site's root (`/llms.txt`) that gives large language models a curated summary of the site and a list of its most useful pages, so an AI agent can find the right content at inference time without parsing HTML built for humans.
llms.txt is a proposed convention for a Markdown file served at a site's root (/llms.txt) that gives large language models a curated summary of the site and a list of its most useful pages, so an AI agent can find the right content at inference time without parsing HTML built for humans.
The proposal was published by Jeremy Howard of Answer.AI on September 3, 2024, with a v2 revision dated August 10, 2026. The format is deliberately plain: an H1 with the site name, a blockquote summary, optional detail paragraphs, then H2 sections of Markdown links with short notes, plus an "Optional" section for secondary material a model can skip when context is tight. It sits alongside robots.txt and sitemap.xml in spirit but serves a different reader: not a crawler deciding what to index, but a model deciding what to read right now. The related AGENTS.md convention does a similar job for coding agents inside a repository.
Two things are worth knowing before you add one. First, Google says you do not need "new machine readable files, AI text files, or markup" to appear in AI Overviews or AI Mode, so llms.txt is not a generative engine optimization lever for Google Search. Second, it does help agents and assistants that fetch your site directly, which is why documentation publishers adopted it early; Anthropic's Claude docs expose an index at claude.com/docs/llms.txt, and agentscamp.com publishes one at https://agentscamp.com/llms.txt. The AI content and search in 2026 guide places it in a content plan, and Claude Code for marketers shows how to generate one from a sitemap with an agent.
Frequently asked questions
- What goes in an llms.txt file?
- Markdown in a fixed shape: an H1 with the site or project name, a blockquote with a short summary, optional paragraphs of detail, then H2 sections that list Markdown links to the pages that matter, each with an optional note. A section titled Optional holds secondary links a model can skip. The spec at llmstxt.org shows the template.
- Does llms.txt help me rank in Google AI Overviews?
- Not according to Google. Its documentation says you do not need to create new machine-readable files, AI text files, or markup to appear in AI Overviews or AI Mode. llms.txt is aimed at AI agents and assistants that fetch a site directly, and at documentation sites where a model needs a map. Treat it as cheap and harmless, not as a ranking lever.
- Who uses llms.txt?
- The proposal page says thousands of sites publish one and lists OpenAI, Anthropic, and Google among the companies that do. Anthropic's Claude documentation, for instance, points at a docs index under /docs/llms.txt. This site publishes one at agentscamp.com/llms.txt.
Related
- Generative Engine Optimization (GEO)Generative engine optimization (GEO) is making content more likely to be cited or summarized in AI answers from ChatGPT, Gemini, or Google AI Overviews.
- AI OverviewsAI Overviews are Google's AI-generated summaries at the top of Search results, with source links; AI Mode is the fuller conversational version.
- Answer Engine Optimization (AEO)Answer engine optimization (AEO) is structuring content so search engines and AI assistants can lift a direct answer from it for AI Overviews and chatbots.
- Does AI-Written Content Rank in 2026? What Google Actually SaysWhat Google's own documentation says about AI-generated content in 2026: what it rewards, what scaled content abuse is, how AI features cite, and a checklist.
- Claude Code for Marketers: The Complete 2026 GuideWhy a marketer would open a terminal agent, when claude.ai or Cowork is the better tool, a CLAUDE.md for a content repo, five workflows, and what goes wrong.
- RAG (Retrieval-Augmented Generation)RAG retrieves relevant documents from your own data and injects them into an LLM's prompt at query time, grounding answers in facts the model wasn't trained on.
- AGENTS.mdAGENTS.md is a versioned instruction file that tells coding agents how to work in a repository — including commands, conventions, boundaries, and checks.