What is llms.txt — and does your site actually need one yet?
An llms.txt file is a short, plain-text map you place at the root of your domain to tell AI answer engines which pages matter most and how to read them. It won't fix a thin site — but on a strong one, it's becoming the cheapest way to influence how ChatGPT, Perplexity, and Google AI Overviews summarize you.
Let me back up. A client asked me last month whether they needed "that new AI file everyone's posting about." Fair question — there's a lot of noise. So here's what I've actually learned after adding llms.txt to a few dozen sites, watching what changed, and — more usefully — watching what didn't.
The short version: it's worth doing, it takes about twenty minutes, and almost everything written about it online overstates the magic. Let's separate the signal from the hype.
- →llms.txt is a curation file, not a permission file — it points engines at your best content rather than blocking crawlers like robots.txt does.
- →Adoption by the major engines is partial and unconfirmed — treat it as a low-cost hedge, not a guaranteed ranking lever.
- →It pays off most when your underlying content is already clean, structured, and worth citing.
- →llms.txt is the entry point, not the whole system — pair it with a full-content file and a structured index so models get both a curated summary and something to parse deeper.
The short answer: what llms.txt actually is
At its core, llms.txt is a Markdown file you publish at yourdomain.com/llms.txt. Where a sitemap lists every URL for search crawlers, llms.txt does the opposite — it's a deliberately short, human-written guide that says "if a model is trying to understand or summarize us, start here."
Think of it as the difference between handing someone your entire filing cabinet and handing them a one-page brief. Large language models work better with the brief. They have finite context windows, and they reward content that's easy to parse and hard to misread. A tidy llms.txt is you doing that parsing work for them, on your terms.
"llms.txt is the front door. What's behind it is what actually gets read."
Why a plain text file suddenly matters
For two decades the goal was simple: rank on Google's first page. That game still exists, but a growing share of your audience never sees a results page at all. They ask ChatGPT. They read Perplexity's synthesized answer. They glance at a Google AI Overview and move on. In every one of those moments, a model is deciding whether to mention you — and, if it does, what to say.
That's the shift llms.txt is responding to. When a model has to compress your entire site into a sentence or two, the sites that made themselves easy to compress win. The ones that buried their best material under interstitials, duplicated pages, and vague headings get summarized badly — or skipped.
None of this replaces the fundamentals. Technical SEO and crawl readiness still matter, and clean schema still does the heavy lifting. llms.txt is the layer on top — a way to point and say "this, specifically, is the part worth reading."
What actually goes inside one
The format is refreshingly boring: a short H1 with your site or brand name, an optional blockquote summarizing what you do, then a few Markdown link lists grouped under H2s — “Docs,” “Key services,” “Background.” Each link gets a short description so the model knows why it’s there.
Where teams get this wrong isn’t the format — it’s treating llms.txt as the only file. A model that can “only hold a little” is exactly why the curated entry point needs somewhere to point deeper: an llms-full.txt with expanded content inlined, a structured llms-index.json a model or agent can parse programmatically, and concept-level metadata (authorship, dates, provenance) sitting underneath. llms.txt stays short on purpose. It just shouldn’t be the whole system.
The mistake isn’t restraint at the top layer — it’s mistaking the top layer for the entire deployment.
Our AI Visibility Report shows exactly which pages are indexed and your site's exact authority metrics.
Does anything actually read it yet?
Here's the honest part, because you deserve it. As of this writing, no major engine has publicly committed to honoring llms.txt as a ranking input. Some crawlers appear to fetch it; others ignore it. Anyone promising you a guaranteed lift is selling certainty that doesn't exist yet.
So why do it? Because the cost is near zero and the downside is none. The file is twenty minutes of work, it can't hurt your existing SEO, and the moment adoption does become standard — the way robots.txt quietly did — you'll already be in place. I'd rather have the umbrella in the closet before it rains.
There's also a quieter benefit. The act of writing a good llms.txt forces you to answer "what are the ten pages that actually represent us?" Most teams have never answered that out loud. The clarity you get is worth the twenty minutes even if no model ever reads the file.
How to write one without overthinking it
Start with your name and a single sentence describing what you do. Then list your highest-intent pages: core services, your best proof, and a couple of cornerstone articles. Each gets a short, factual description — models don’t reward “world-class,” they reward specificity.
Save it as llms.txt, drop it at your domain root, and confirm it loads in a browser. That’s the top layer — not the whole deployment. Underneath it, an llms-full.txt carries the expanded content models actually need to answer questions accurately, and a structured llms-index.json gives agents something they can parse programmatically instead of guessing from prose.
The part teams get wrong is treating this as a manual, quarterly chore. Content changes constantly — new pages, updated services, revised proof points — and a file that’s stale is worse than no file at all, because it’s confidently wrong. Set it up once to regenerate automatically when your content changes, and you never have to remember to “revisit” anything.
Where this goes next
My read is that llms.txt is an early signal of a larger change: the web is being re-architected for machines that read on our behalf. Whether this exact file becomes the standard or gets replaced by something tidier in eighteen months, the underlying skill — curating the version of yourself you want an AI to repeat — is only going to matter more.
So treat llms.txt as practice. Publish one, keep your content genuinely worth citing, and pay attention to how the engines describe you. The brands that learn to speak clearly to models now will be the ones those models trust later.
Frequently asked questions
Is llms.txt the same as robots.txt?
No. robots.txt tells crawlers what they may or may not access — it's about permission. llms.txt is about curation: it points language models toward the content you most want understood and summarized. They solve different problems and can coexist.
Will adding llms.txt improve my Google rankings?
There's no confirmed ranking benefit today, and you shouldn't expect one. Its value is positioning: it's a low-cost hedge for the growing share of discovery happening inside AI answers, and it costs nothing in traditional SEO terms.
How many links should it include?
Fewer than you think — roughly eight to fifteen high-intent URLs. The file's power comes from restraint. A short, curated list signals clarity; a long one reads like a sitemap and dilutes the message.
Where exactly does the file go?
At the root of your domain, reachable at yourdomain.com/llms.txt, just like robots.txt. Save it as plain Markdown, upload it, and confirm it loads in a browser. That's the whole deployment.
Andrew Ruditser writes about technical SEO, AI crawl readiness, structured data, web architecture, and digital strategy for MAXPlaces Marketing.
