Does any AI crawler actually read your llms.txt? The data says almost certainly not.

Here's a number that should make you pause before touching your AI SEO checklist this quarter: of roughly 38,000 domains that published an llms.txt file in May 2026, 97% received zero requests for it from any bot that month. That's the finding from Ahrefs' analysis of 137,000 websites using actual server-log data — not survey responses, not self-reported figures. Just raw access logs showing what crawlers actually asked for. If you spent time building out a structured llms.txt to guide AI assistants to your best content, the evidence says there's an overwhelming chance nobody looked at it.

Where does this data come from?

Ahrefs ran their study across all 137,210 domains in their Web Analytics product that received traffic in May 2026. They combined file-presence checks — does /llms.txt return 200? — with server-log analysis showing whether any bot actually requested it during the month. That second step is what makes the study interesting: you're measuring real behaviour, not just what someone published and forgot about. One important caveat worth keeping in mind: Ahrefs customers skew heavily toward technically-minded, AI-aware site owners. The 28% adoption rate in their sample is almost certainly an upper bound for the web at large. Studies covering 300,000+ general-purpose domains find closer to 10% adoption overall.

Is adoption actually growing, or does the data flatter?

The top-line adoption numbers look strong. Across the top 10,000 websites, llms.txt files rose from 1.04% in July 2025 to 5.61% in June 2026 — roughly 5.4× in twelve months. Some individual platforms show even higher rates.

But can you trust that Shopify figure? Not quite. In late April and early May 2026, Shopify silently pushed an llms.txt file to every store on the platform by default — no opt-in, no merchant notification, no announcement. That's where the 78% comes from: a platform switch, not 78% of merchants actively deciding to publish one. Contentful (22.9%) and Adobe Experience Manager (19%) probably reflect a mix of default tooling and deliberate adoption. WordPress at 8.7% is the cleaner number for human intent: each of those sites means someone installed a plugin or opened a text editor. If you strip out platform-default behaviour, voluntary adoption of llms.txt is still a small minority of the web.

So who is actually reading these files?

Even among the 3% of files that received any request at all in May 2026, the picture isn't what you'd hope for.

GPTBot topped the list of named AI tools requesting llms.txt files. But named AI tools only accounted for 19.5% of total fetches in that already-tiny slice. Another 12% came from the industry studying itself: llms.txt checker tools, GEO research services, and academic crawlers. That means the majority of requests for llms.txt files — even the ones that got traffic at all — weren't from AI assistants trying to understand your site. They were from tooling confirming the file exists.

Put it another way: if you publish an llms.txt today, there's roughly a 97% chance no AI bot requests it this month. Of the 3% that do get requests, most traffic isn't from the AI assistants you're trying to reach.

What do the major platforms actually say?

Google's Gary Illyes confirmed in July 2025 that Google doesn't support llms.txt and has no plans to. In late May 2026, Google published a new guide on optimising for generative AI features that explicitly stated machine-readable files like llms.txt "aren't needed" to appear in AI-generated results. That's a clear statement from the company behind the world's most widely used AI search surface.

No major AI assistant product has announced that llms.txt factors into its ranking or source-inclusion decisions. The file format gained fast momentum in the SEO community — but momentum in the community and support from the platforms themselves are very different things. Smaller AI tools and research crawlers do read the files, as the Ahrefs data shows, but they're not the engines driving measurable referral traffic at scale.

Does having one actually improve your AI citation rate?

Statistical analysis published in mid-2026 found no measurable correlation between publishing an llms.txt and how often a domain gets cited by the dominant AI assistants in their answer surfaces. That's a null result from a reasonably large dataset — and it's exactly what you'd expect given how few AI tools are actually requesting the files.

That doesn't mean the file is worthless forever. The spec could get official support from a major platform at any time, and having one already in place would be a low-cost win. The question is whether it belongs near the top of your priority list right now.

What should you focus on instead?

The single biggest factor in whether AI crawlers can understand your content is whether they can read it at all on the first fetch. Most AI search bots request raw HTML without executing JavaScript — which means any content that lives inside a client-side rendered bundle is effectively invisible to them. Pre-rendered or server-side rendered HTML, clear heading structure, Schema.org markup, and canonical tags are the signals these crawlers are actually consuming, on every single request.

If key sections of your site — product descriptions, article bodies, pricing tables — only appear after JavaScript hydrates the page, that's the leak to fix. Compared to the effort of ensuring reliable server-rendered HTML for crawler user agents, llms.txt sits much further down the impact curve.

Publish an llms.txt if it takes an hour and you want to be ready for the day adoption actually lands. For documentation-heavy or content-focused sites it's reasonable to use it to highlight priority sections or exclude internal tooling. But treat it as light infrastructure for a future scenario, not a lever you can pull today to move AI visibility. The server logs are pretty clear on where the attention is actually going — and it isn't your llms.txt.