What is llms.txt, and does your site need one?
llms.txt is a markdown file at your domain root that tells AI assistants what your site contains and which pages matter, in a form they can read cheaply without crawling every page.
The problem it solves
An AI assistant asked about your product has a budget problem. It can fetch a handful of pages, not your whole site, and most of what it fetches is navigation, scripts and styling rather than substance. Whatever it happens to land on becomes its understanding of you.
llms.txt is a short, curated answer to "what is this site, and where is the good stuff?" — written in markdown because that is what models parse most cheaply.
What goes in it
An H1 with the site or company name, a blockquote summary, then H2 sections grouping links with short descriptions. That is the whole format. Keep it under a few hundred lines.
The discipline is editorial, not technical: link the pages you want cited, not every page you have.
# Acme Analytics > Acme Analytics is a product analytics platform for B2B SaaS teams, > with session replay, funnels and a REST API. ## Product - [Funnels](https://acme.com/product/funnels): Build and compare conversion funnels. - [Session replay](https://acme.com/product/replay): Watch real user sessions. ## Developers - [API reference](https://acme.com/docs/api): REST API with an OpenAPI spec. - [MCP server](https://acme.com/docs/mcp): Query your analytics from an AI client. ## Company - [Pricing](https://acme.com/pricing): Plans and limits.
How it differs from robots.txt and sitemap.xml
robots.txt says what crawlers may not touch. sitemap.xml lists every URL, for completeness. llms.txt does neither — it is a short editorial summary aimed at a reader with a token budget.
You want all three. They answer different questions: permission, inventory, and meaning.
Is it actually used?
Honestly: adoption is uneven. It is a convention, not a ratified standard, and no major AI vendor has committed to reading it on every request. Some crawlers fetch it, some ignore it.
It costs one static file to publish. The asymmetry is what makes it worth doing — near-zero cost, real upside if the convention holds.
Does this apply to your site?
The free checker runs this check and eighteen others, then generates the files to fix whatever fails.
Run the checkerRelated
- Which AI crawlers is your robots.txt blocking?GPTBot, ClaudeBot, PerplexityBot and Google-Extended each read robots.txt. A single Disallow can make your site invisible to AI answers — often unintentionally.
- JSON-LD is how AI answer engines know what you areStructured data turns a page into facts an engine can quote with confidence. Which schema types matter, and the @graph mistake that makes valid markup look invalid.