Is Publishing llms.txt Actually Changing How AI Assistants Cite Your Site?

When SE Ranking built a model to predict how frequently a website gets cited in major AI assistant answers, they included "has llms.txt" as a potential signal. Removing it improved the model's accuracy. That finding, from a study across roughly 300,000 domains, is one of the cleaner pieces of evidence we have on whether the file actually affects AI discoverability — and it points the wrong way for the GEO community that's been loudly promoting the spec.

This post digs into where adoption actually stands, what the adoption pattern tells us about whether people have measured results, and where the file does appear to be doing something useful.

Where the data is from

SE Ranking's study covered approximately 300,000 domains and looked at both llms.txt presence and frequency of AI citations. Rankability runs a monthly crawl of the top 1,000 websites by traffic and tracks the adoption rate. Originality.ai has been monitoring llms.txt presence across more than three million sites since mid-2025.

How widespread is llms.txt actually?

SE Ranking found llms.txt on 10.1% of the ~300,000 domains they analyzed — roughly one in ten sites eighteen months after the spec was first proposed. Rankability's tighter sample of the top 1,000 websites puts it at 8.7%; expand that to the top 10,000 and it drops to 5.6%, because larger, more technically resourced sites tend to adopt faster.

The technology sector is well ahead of the pack. Rankability's top-1,000 data puts tech-sector adoption at 36.4% — nearly four times the overall rate. The spec originated in developer circles and spread through those communities first. Finance, healthcare, and legal domains haven't moved as quickly, sitting well below 10% adoption among their respective top-100 domains — compliance review cycles slow down any new file type that touches public-facing content.

Here's the part that should raise your eyebrows: adoption is flat across traffic tiers. Low-traffic sites (0–100 visits) showed 9.88% adoption. Mid-traffic sites (1,001–5,000 visits) were at 10.54%. High-traffic sites (100,001+ monthly visits) were at 8.27%. That distribution is almost perfectly even. If having an llms.txt file were driving meaningful improvements in AI citation frequency, the high-traffic sites — which have both more resources to optimise and more incentive to do so — would show substantially higher adoption. They don't. That pattern looks a lot more like cargo-culting than results-driven adoption.

Is the number of sites growing?

Sharply. Originality.ai has tracked llms.txt instances across more than three million monitored sites since mid-2025. In June 2025 they counted 4,088 valid files. By May 2026 that had reached 36,120 — an 8.8x increase over twelve months. Including the companion formats (llms-full.txt and ai.txt), the total number of adopting sites reached approximately 38,980.

That's impressive growth from a near-zero base. The question is whether it reflects observed results or just the spread of a checklist item. The flat traffic-tier distribution from SE Ranking's data, combined with the citation model result, both point toward the latter — people are adding the file because other people are adding the file.

Does it actually change how often AI assistants cite you?

This is the uncomfortable answer.

SE Ranking's researchers ran a predictive model to identify which factors correlate with AI citation frequency across their 300,000-domain dataset. They included llms.txt presence as a variable. When they removed it, the model improved. The file wasn't just uncorrelated with better citation rates — it was introducing noise that made predictions worse. That's a specific kind of null result that's harder to explain away than a simple absence of correlation.

Bot log data reinforces this. Across multiple large-scale studies, the major AI indexing crawlers — GPTBot, ClaudeBot, PerplexityBot, OAI-SearchBot — skip /llms.txt the vast majority of the time. They don't use the file as an input to what pages they crawl or how they weight content. The bots generating citation outputs are doing what they've always done: fetching pages, parsing content, building indices from raw HTML.

Who does actually read it?

Agentic tooling does.

IDE coding assistants, API-oriented agents, and MCP-connected tools do fetch /llms.txt when they're orienting on a new codebase, product, or API surface. That use case is genuinely different from AI search indexing. A coding assistant that's about to help a developer work with your library benefits from a structured document that points to your API reference, changelog, and quickstart — it gives the agent a map of your content rather than requiring it to crawl and infer structure from scratch.

For that narrower audience, the file is genuinely useful. The mismatch is that most llms.txt advice is framed around SEO and AI citation outcomes, not around serving agents that are actively working with your product. The GEO community picked up the spec and positioned it as an AI-search ranking signal, but that conflates two different access patterns: agents actively oriented on your product versus crawlers building a general-purpose index.

What should you actually take away from this?

First, separate the two questions: "will this improve my AI search citations?" versus "will this help agents that are actively working with my content?" The evidence strongly says no to the first and yes to the second, in the right context.

If you run a developer tool, a public API, or technical documentation, publish an llms.txt. Point it at your most useful machine-readable content. Think of it as a sitemap for agents rather than an SEO signal.

If you're a content publisher, retailer, or consumer brand hoping to improve your presence in AI assistant answers, the citation data says the file probably won't move anything measurable. The more direct question is what AI crawlers actually receive when they request your pages — not what a browser renders, but the static HTML that arrives before any JavaScript executes. If your content lives behind client-side rendering, many of the crawlers that feed citation pipelines are arriving to an empty document shell. Closing that gap has clearer evidence behind it than adding /llms.txt does.

The broader point: the adoption curve for llms.txt has moved faster than the evidence base. That's not unusual for emerging standards, but it's worth being clear-eyed about. Eighteen months in, the most honest reading of the available data is that it's a genuinely useful file for specific audiences and a cargo-cult item for most of the sites that have it.