What is llms.txt?
llms.txt is a proposed web standard, stored at yourdomain.com/llms.txt, that provides AI language models with a structured, Markdown-formatted index of a site's most authoritative pages, their purpose, and how the site's content should be interpreted. It functions similarly to sitemap.xml but is designed for AI consumption rather than search engine crawling.
The standard was proposed by Jeremy Howard (co-founder of Answer.AI) in September 2024 to solve a practical problem: AI systems processing documentation sites would hit context window limits before understanding site structure. llms.txt gives AI systems a prioritized, pre-structured map.
A valid llms.txt includes a site name header, a one-sentence description block, and one or more sections listing page URLs with brief descriptions. An optional llms-full.txt includes full page content. As of 2026, ChatGPT, Claude, and Perplexity all parse llms.txt when present.
Sites without llms.txt are not automatically excluded from AI citation, but sites with a valid, well-structured llms.txt receive a prioritization advantage in AI retrieval systems that support the standard.