TL;DR
- An llms.txt file is a plain-text Markdown file at the root of a website that gives AI tools a short summary and a curated list of the site’s most important pages. It was proposed by Jeremy Howard in 2024.
- Most businesses don’t need one to rank or get cited. Google says these files “will neither harm nor help your site’s visibility or rankings in Google Search, as Google Search ignores them.”
- AI crawlers rarely read it. Ahrefs found 97% of about 38,000 valid llms.txt files received zero requests in May 2026.
- It still has a use: Chrome’s Lighthouse now checks for llms.txt as part of readiness for AI agents, and documentation sites and coding tools benefit most.
- My advice: add one if it takes a few minutes (on WordPress, Rank Math can generate it), but spend your real effort on content, structure, and brand mentions.
What Is an llms.txt File?
An llms.txt file is a Markdown text file placed at yoursite.com/llms.txt that tells AI tools what your site is about and where its most useful pages are. It works like a short, curated table of contents written for language models rather than people.
The format was proposed by Jeremy Howard in 2024 and is maintained at llmstxt.org, where version 2 of the proposal was published in August 2026. According to the spec, a file follows this order:
- An H1 heading with the site or project name. This is the only required part.
- A blockquote summary with the main information someone needs to understand the rest of the file.
- Optional detail sections in plain Markdown.
- H2 sections with lists of links, each link pointing to a page with a short note on what it contains.
- An “Optional” section for secondary pages an AI can skip when it has limited space.
The spec also suggests offering clean Markdown versions of important pages at the same address with .md added, so AI tools can read them without menus, ads, and scripts.
How Is llms.txt Different From robots.txt and sitemap.xml?
llms.txt is a curated guide for AI tools, while robots.txt controls which crawlers may access your site and sitemap.xml lists every page you want search engines to index. The three files sit in the same place but do different jobs.
| robots.txt | sitemap.xml | llms.txt | |
|---|---|---|---|
| Main job | Allow or block crawlers | List all indexable pages | Summarize the site and highlight important pages |
| Audience | All crawlers | Search engines | AI tools and agents |
| Format | Plain-text rules | XML | Markdown |
| Official standard | Yes, widely supported | Yes, widely supported | Community proposal |
| Affects Google rankings | Indirectly, by controlling crawling | Helps discovery | No, Google Search ignores it |
If you want to allow or block AI crawlers like GPTBot or Google-Extended, robots.txt is still the tool for that. llms.txt doesn’t grant or deny access to anything.
Does Google Use llms.txt?
Google Search doesn’t use llms.txt, but Google’s Chrome team has started checking for it in a developer tool. Both are true, and they measure different things.
On the Search side, Google’s AI optimization guide (updated July 2026) says you don’t need “new machine readable files, AI text files, markup, or Markdown to appear in Google Search.” It adds that creating llms.txt files “will neither harm nor help your site’s visibility or rankings in Google Search, as Google Search ignores them.” Search Engine Land reported the clarification in June 2026.
On the Chrome side, Lighthouse, the free site-auditing tool built into Chrome, added an Agentic Browsing category that includes an llms.txt check. It “checks for the presence of a machine-readable summary at the domain root,” as covered by Search Engine Land in May 2026. That category measures how easily AI agents, tools that browse and act on websites for a user, can understand a site, and Google doesn’t present it as a ranking signal.
So llms.txt won’t help you rank in Google or appear in AI Overviews, but it may become part of how sites prepare for AI agents.
Do ChatGPT, Claude, and Perplexity Read llms.txt?
There’s no public evidence that major AI chat tools rely on llms.txt, and usage data suggests they rarely request it. According to Ahrefs, no major AI provider officially supports the standard.
The best available data comes from Ahrefs, which studied 137,000 domains in its web analytics product:
- 28% of those domains publish an llms.txt file.
- Of about 38,000 domains with a valid file, 97% received zero requests for it in May 2026.
So adoption is far ahead of actual use. The clearest fit today is software documentation, where an AI coding tool may need a clean list of docs pages.
Who Should Add an llms.txt File?
You should add an llms.txt file if your site has documentation or reference content that AI tools and developers use directly, and it’s optional for most other sites. The table below shows how I’d prioritize it.
| Site type | Should you add llms.txt? | Why |
|---|---|---|
| Software, API, or product documentation | Yes, worth doing | Coding assistants and AI agents use structured docs most |
| SaaS or tech product site | Worth doing | Low effort, and helps agents find product and pricing pages |
| Local business (law firm, clinic, cafe) | Optional, low priority | Google Business Profile, reviews, and service pages matter far more |
| Blog or content site | Optional | Only if your SEO plugin generates it automatically |
| Online store | Optional | Product data, structured data, and Merchant Center details matter more |
There’s no known downside beyond the time it takes. Google says it won’t hurt your rankings, and no AI platform has said it penalizes sites for having one.
For my own clients, llms.txt sits near the bottom of the list. With Immigration Lawyers USA, the higher-impact work was consolidating three competing Google Business Profile listings into one and building a five-hub site structure around the firm’s practice areas.
How to Create an llms.txt File
You create an llms.txt file by writing a short Markdown file with your site name, a summary, and links to your most important pages, then uploading it to your site’s root folder. It takes about 15 minutes by hand, or a couple of minutes with a plugin.
- Pick your most important pages. Choose 10 to 30 pages that best explain what you do: services, pricing, about, contact, and your most useful guides.
- Write the file. Start with an H1 for your business name, add a one- or two-sentence summary in a blockquote, then group links under H2 headings with a short note for each.
- Upload it to your root folder. Save it as llms.txt and place it so it loads at yoursite.com/llms.txt.
- On WordPress, use your SEO plugin. Rank Math can generate the file from your posts and pages: enable the LLMS Txt module in the Rank Math dashboard, then edit it under General Settings. This site’s own llms.txt is generated this way.
- Check it loads. Open the URL in a browser and confirm it shows plain text, not an error page.
Keep the file short and current. Its value is in being a clean summary, not a copy of your sitemap.
When I set up the llms.txt for famamun.com, the useful decisions were the summary at the top and which pages to include. I left out plugin template pages and other supporting pages, so the file points to the services and guides a reader would actually want, plus a short contact section.
What to Prioritize Instead for AI Visibility
If your goal is to be cited by ChatGPT, Gemini, Perplexity, or Google AI Overviews, put your effort into what those systems actually read: indexed pages with clear answers, credible sources, and mentions of your brand elsewhere. llms.txt is a small extra on top of that work.
- Make sure crawlers can reach your pages. Check robots.txt isn’t blocking the AI crawlers you want.
- Write answer-first, well-sourced content. I explain why in what GEO is and how it differs from AEO, including research showing statistics and quotations raise AI visibility.
- Build mentions off your site. Reviews, digital PR, and directory listings help AI tools recognize your business.
- Measure what’s happening. Use the steps in how to track whether your content is being cited by AI platforms.
For the bigger picture, see my guide to what AEO is and why it matters in 2026. If you’d like to know whether AI tools already mention your business, my free AI visibility audit checks it for you, or you can book a strategy call.
Frequently Asked Questions
Is llms.txt mandatory?
No. llms.txt is a community proposal, not an official standard, and Google says Search ignores it, so your site can rank and be cited without one.
Are llms.txt files worth it?
For documentation and software sites, yes, because AI coding tools use them. For most other businesses they’re optional: Ahrefs found 97% of about 38,000 valid llms.txt files got zero requests in May 2026.
Does llms.txt help with Google AI Overviews?
No. Google’s AI optimization guide says llms.txt files neither help nor harm visibility or rankings in Google Search, which includes its AI features.
Where do I put an llms.txt file?
Put it in your website’s root folder so it loads at yoursite.com/llms.txt, the same place as robots.txt.
What’s the difference between llms.txt and robots.txt?
robots.txt tells crawlers what they may access. llms.txt summarizes your site and points AI tools to your most useful pages, but it doesn’t allow or block anything.
Can llms.txt hurt my SEO?
No. Google states these files won’t harm your visibility or rankings in Google Search. The only cost is the time to create and update it.
