Is Your Website Invisible to AI Search? The JavaScript Gap Nobody's Talking About

17% of the world's top fintech companies deliver zero content to an AI crawler making a standard HTTP request. Not degraded content, not partial content — a blank page. That's the finding from a May 2026 analysis of 274 fintech homepages measuring what actually arrives when you strip out JavaScript. And it's not a fintech-specific problem: a peer-reviewed study of 405 small-business websites published in June 2026 found that one in four had at least one piece of answer-critical information — pricing, contact details, descriptions, service specifics — present in the rendered version but simply absent from the raw HTML that AI crawlers receive.

This isn't a content quality question. It's a visibility question. The information exists — you can see it in a browser. It just isn't in the initial HTTP response that AI crawlers fetch, which means for practical purposes it doesn't exist to them.

Where does this data come from?

Three studies sit behind this post. The first is Joshua Gutierrez's June 2026 paper on SSRN, which tested 405 websites using a dual-capture method: once with a headless browser running full JavaScript execution (simulating how Googlebot sees it), and once with a plain HTTP GET request carrying a GPTBot user agent and no JavaScript execution (simulating how AI search crawlers see it). 368 sites returned usable data.

The second is a May 2026 analysis of 274 fintech homepages from the world's top fintech companies list — published by Web Performance Tools, Inc. via Search Engine Journal. The third is a December 2024 server-log study by Vercel and MERJ covering over 500 million GPTBot requests, examining exactly what those crawlers do when they encounter JavaScript files on a page.

What percentage of sites actually have this problem?

Gutierrez's dual-capture approach gives a direct answer: 25.0% of analysed sites (95% confidence interval: 20.8–29.7%) had at least one answer-critical element present in the browser-rendered DOM but completely absent from the raw HTTP fetch.

The fintech study gives harder categories. Of 274 fintech homepages:

• 17% (47 sites) delivered zero readable content to an AI crawler without JavaScript
• 36% delivered less than 80% of their homepage content in raw HTML
• 64% had their main content present in the initial static response

The same 274 sites were also tested with a browser running for five seconds: 273 of the 274 delivered 80% or more of their content. The content is all there. It just requires JavaScript to surface — and that's the step AI crawlers skip.

So which crawlers actually execute JavaScript?

A 2026 analysis of 23 major AI agents found that 69% cannot execute JavaScript. The Vercel and MERJ server-log study makes this concrete for the two highest-volume crawlers.

GPTBot fetches JavaScript files in about 11.5% of requests — then does nothing with them. ClaudeBot downloads JavaScript files in approximately 23.84% of requests — same result. PerplexityBot and the other major AI crawlers follow the same pattern. They're downloading JS source code, likely to examine URLs and extract static strings, but they're not running it. The page content that JavaScript would render never materialises.

The meaningful exception is Google's crawler, which has access to a full web rendering service infrastructure built over many years. This creates an asymmetry that shows up directly in practice: a React single-page application can rank position one on Google for a competitive keyword while being completely absent from AI search results on every other platform. Performing well on traditional search, invisible everywhere else.

Why haven't the other AI crawlers built the same capability? Running a full headless browser at GPTBot's scale — 569 million requests in a single month on Vercel's network alone — is expensive in a way that doesn't match the economics of training and retrieval crawls. Google's investment made sense for a product tied directly to search advertising revenue. The calculation looks different for crawlers primarily built around AI training data collection.

Does this affect Bing-backed AI search too?

Worth flagging: approximately 92% of AI assistant web search queries route through Bing's index. Bingbot does have some JavaScript rendering capability, but it's limited — considerably less than Googlebot's full rendering service. So AI search platforms that pull from Bing are also partially exposed to this problem, depending on which pages Bing chose to render when it indexed your site and whether your content made it into that indexed version.

What should you actually do?

Check your own raw HTML first. Run curl -A "GPTBot" https://yourdomain.com and look at what comes back. Is your headline there? Your product or service descriptions? Pricing? FAQs? If your main content requires JavaScript to appear in the browser but isn't in the raw HTML, AI crawlers are getting an empty shell.

If your framework supports server-side rendering, enable it. Next.js (React), Nuxt (Vue), and SvelteKit all support SSR or static site generation as standard options. The content delivered to browsers stays the same — you're just also making it available in the initial HTML response before JavaScript runs.

For sites that can't migrate immediately, a pre-rendering approach at the edge is a practical alternative: detect AI crawler user agents and serve a pre-rendered HTML snapshot rather than the SPA shell. This is a targeted fix that directly addresses what the crawlers are receiving, without requiring a full architecture change.

The figure that stays with me is the fintech one: 47 of 274 top fintech companies — some of the best-resourced sites on the internet — delivered a completely blank page to a standard AI crawler fetch. Their content is excellent. Their engineering is excellent. None of that matters if the crawler gets nothing back.