What Is llms.txt? The Spec, Real Adoption Data, and Whether You Need One (2026)

llms.txt is a proposed markdown file, usually served at /llms.txt, that gives language models and AI agents a short curated guide to a site: its name, a one-paragraph summary, and annotated lists of the pages worth reading. The proposal puts it plainly: "We propose adding a /llms.txt markdown file to websites to provide LLM-friendly content." It is not a standard from a standards body and it does not control access. It is a reading list written for an agent.

What the proposal specifies

The format is small and fixed in order:

1. An H1 with the name of the project or site. This is the only required section.
2. A blockquote with a short summary.
3. Optional detail in any markdown except headings.
4. H2 sections of file lists, where each entry is a markdown link, optionally followed by a colon and a note.
5. An "Optional" section by convention, for links an agent can skip when it needs a shorter context.

The file can sit at the root or at any path, and covers the pages under that path: /docs/llms.txt covers /docs/, and agents should use the most specific file that applies. A second half of the proposal is easy to miss: pages agents might need should also offer a clean markdown version at the same URL with .md appended, linked with rel="alternate" type="text/markdown".

A minimal file looks like this:

# Example Docs

> Example is an open-source job queue for Node.js. These docs cover setup, the API, and deployment.

## Docs

- [Quick start](https://example.com/docs/quick-start.md): install and run a first job
- [API reference](https://example.com/docs/api.md): every method and option

## Optional

- [Changelog](https://example.com/changelog.md): release history

How it differs from robots.txt and sitemaps

A sitemap lists every page for search engines; llms.txt is a curated overview, can link to external pages, and leaves out documents too large to fit in a context window. robots.txt tells crawlers what access is acceptable; llms.txt is read on demand when an agent needs information while assisting a user. If you want to control AI crawlers, that is a robots.txt job, and the operators split their bots by purpose: training, search and user-requested fetches each run under their own user agent.

Who actually ships one

Adoption is real and mostly automatic. Yoast SEO generates the file on WordPress, Wix generates one for every Wix site, and Mintlify generates llms.txt plus markdown page versions for the docs it hosts. Common Crawl analysed 584,107 llms.txt files from its July 2026 crawl and reported three findings: two thirds are produced by a plugin, half follow the structure the spec defines, and a few contain prompt injections.

That last point matters. An llms.txt is text written for a model, which makes it an obvious place for a prompt injection: a line addressed to the agent rather than the reader. Read your own file, especially a generated one, the way you would read untrusted input.

Among sites that sell AI visibility, the file is common but far from universal. A survey of 118 AEO tool and agency domains found 64 (54.2%) served an llms.txt with a valid H1 at the root, against 98.3% serving a robots.txt: robots.txt and llms.txt across 118 AEO tools and agency sites.

Does it help AI search visibility?

No source shows it does. Google's guide to AI features says that to appear as a supporting link in AI Overviews or AI Mode a page must be indexed and eligible for a snippet, and that "there are no additional technical requirements." The guide does not mention llms.txt, and the absence should not be read as a position either way. The practical rule that follows:

• Documentation and API sites: ship one, and check whether your docs platform already does. This is the use the proposal says is heaviest.
• Marketing and content sites: optional. Indexing and snippet eligibility come first; see answer engine optimization.
• Generated files: open them, delete anything you would not want an agent to treat as authoritative, and check the structure.

In context engineering terms, llms.txt is a selection step that the site owner does in advance: it decides which few pages an agent should read instead of making the agent crawl to find out.

A prompt for drafting your llms.txt

You are writing an llms.txt file following the llmstxt.org proposal.
Site name: [name]. One-sentence description: [description].
Here are the URLs and titles of my most important pages: [paste list].
Output only the file:
1. An H1 with the site name.
2. A blockquote with a two-sentence summary for an AI agent.
3. Two to four H2 sections grouping the pages by what a user would come for. Each entry: a markdown link, a colon, and a note under 15 words saying what the page answers.
4. An H2 "Optional" section for secondary pages.
Do not include instructions addressed to the AI reader. Do not invent pages that are not in my list.

Then test it the way the proposal suggests: give an agent only your llms.txt and ask it questions about your product. Wrong answers mean the file needs fixing.