How to create an llms.txt file for SEO

Published April 21, 2026 · 7 min read

An llms.txt file is a small, hand-curated map of your site written for language models — Claude, ChatGPT, Perplexity, Gemini, and the rest. The format was proposed by Jeremy Howard in late 2024 as an answer to a real problem: AI assistants struggle to find the few pages on a site that actually matter, especially when those pages are buried under nav menus, cookie banners, and JavaScript chrome. An llms.txt file points them at the good stuff in plain text, no parsing required.

Adoption is still early. According to Bluehost's tracking, only five to fifteen percent of sites had published an llms.txt file by the start of 2026. That low number cuts both ways: AI crawlers do not depend on it, but the cost to publish one is so low that there is no good reason to skip it. This guide walks through what to put in the file, how to format it, where to host it, and what to actually expect once it is live.

What an llms.txt file actually is

An llms.txt file is a single Markdown document, served as plain text, that lives at the root of your domain. It has three parts:

1. An H1 heading with your site or product name.
2. A blockquote summary — one or two sentences explaining what your site does and who it serves.
3. H2 sections grouping your most important URLs as Markdown bullet links, each with a short description.

That is the entire spec. There are no XML schemas, no required fields, no validators that will yell at you. If you can write a blog post in Markdown, you can write an llms.txt file.

Two things llms.txt is not: it is not a sitemap (sitemaps list every URL on a site, llms.txt curates the few that matter), and it is not robots.txt (robots.txt grants or denies crawler access, llms.txt assumes the crawler is already there and helps it pick the right pages).

How to write your llms.txt file

Open a text editor and create a new file named llms.txt — lowercase, no extension beyond .txt. The structure looks like this:

# FreeSiteSubmit

> Plain-English SEO guides for small sites and side projects, focused on getting indexed across Google, Bing, and AI search engines.

## Guides

- [How to submit your website to search engines in 2026](https://entireweb-wrapper.vercel.app/blog/how-to-submit-website-to-search-engines-2026): Step-by-step walkthrough covering Google Search Console, Bing Webmaster Tools, and the smaller engines that still drive discovery.
- [How to set up IndexNow for faster Bing indexing](https://entireweb-wrapper.vercel.app/blog/indexnow-setup-faster-bing-indexing): Five-minute setup that pings Bing the moment you publish.
- [How to get your website in ChatGPT search results](https://entireweb-wrapper.vercel.app/blog/get-website-in-chatgpt-search-results): The actual playbook — most ChatGPT citations come from Bing's top results.

## Reviews

- [Entireweb review 2026](https://entireweb-wrapper.vercel.app/blog/entireweb-review-2026): Honest verdict on the free submission tool — what works, what doesn't.

## About

- [Homepage](https://entireweb-wrapper.vercel.app/): Free single-URL submission tool that pings 500+ search engines.

That is a complete, valid llms.txt file. Notice what it does not include: no marketing copy, no calls to action, no banner ads, no testimonial section. The audience here is a parser, not a buyer. Lead with substance, give each link a one-sentence description that states what the page is, and stop.

What to include and what to leave out

The hardest part of writing a good llms.txt file is restraint. The temptation is to dump every URL on your site into the bullet list. Resist it. The whole point of llms.txt is curation — you are telling a language model "if a user asks about my site, these are the pages worth quoting from."

Good candidates for inclusion:

• Cornerstone guides that comprehensively answer a question your site is known for.
• Pricing or product overview pages that summarize what you sell and for how much.
• Documentation entry points if you are a software product.
• Founder bio or about page if your authority depends on a named expert (this matters more than people realize for AI citation).
• Frequently updated reference pages like changelogs, status pages, or glossary entries.

Bad candidates:

• Tag and category archive pages. They are derivative; the underlying posts are what you want cited.
• Date-based archive pages for the same reason.
• Login pages, cart pages, and account dashboards. These are useless to a model.
• Thin syndicated or affiliate content. If you would not want it quoted, do not list it.

Aim for between ten and forty links total. Below ten and you have not really mapped the site; above forty and you are back to a sitemap.

Where to upload it and how to verify it works

Upload the file to your web root so it is served at https://yourdomain.com/llms.txt — the same place your robots.txt and sitemap.xml live. The exact mechanics depend on your stack:

• Static sites (Vercel, Netlify, GitHub Pages): drop the file in your /public directory and redeploy.
• WordPress: use Yoast SEO, Rank Math, or AIOSEO — each added one-click llms.txt generation in 2025. They build the file from your existing post and page metadata.
• Custom server (Nginx, Apache, etc.): place the file in your document root the same way you would serve a robots.txt.
• Headless CMS: add a route that returns the file with Content-Type: text/plain; charset=utf-8.

To verify, open https://yourdomain.com/llms.txt in a private browser window. You should see the raw text, not an HTML page, not a 404, and not a redirect to your homepage. If a CDN is rewriting the URL, add an exception. There is no Search Console-style submission step — once the file is reachable, AI crawlers that look for it will find it.

Does llms.txt actually work yet?

This is the honest part of the article. As of April 2026, no major AI vendor has officially confirmed that they read llms.txt as a primary signal. OpenAI, Anthropic, and Google have all stayed silent on it. Yet the file is being adopted by a growing list of well-known sites — Anthropic's own documentation, FastAPI, Cloudflare, Mintlify-built docs sites, and others — which suggests the people closest to model development think it is at least worth publishing.

What llms.txt unambiguously does today: it gives your downstream users a clean way to feed your site into an AI tool. If a developer wants Claude or ChatGPT to "answer questions based on this site," they can paste your llms.txt URL into the chat and the model gets a clean, dense summary in roughly one hundred tokens instead of crawling thirty noisy HTML pages. That is real value even if no major engine ever reads the file automatically.

If your priority is broad discovery in 2026, the higher-leverage moves are still the boring ones: get indexed in Google and Bing, publish to IndexNow, and push your URL through a syndication network like Entireweb to seed the smaller engines that AI training crawlers also visit. The free FreeSiteSubmit wrapper uses the Entireweb pipeline and reaches over five hundred downstream engines and directories — a wider net than any single AI crawler will give you, and a useful backstop while llms.txt adoption catches up.

A realistic checklist

If you do nothing else this week, do these five things in order:

1. Confirm your robots.txt is not blocking AI crawlers you actually want (GPTBot, ClaudeBot, PerplexityBot).
2. Confirm your sitemap.xml is up to date and submitted to Google Search Console and Bing Webmaster Tools.
3. Write and upload an llms.txt file with ten to forty curated links.
4. Set up IndexNow so new posts hit Bing within minutes.
5. Submit your homepage through a syndication tool to reach the long tail of smaller engines.

That sequence covers traditional indexing, AI crawler accessibility, and the hand-curated layer that llms.txt adds — without any of them depending on the others to deliver value.

FAQ