Anyone can say they're GPTBot.
A crawler announces itself with one line of text it writes itself. Scrapers copy that line — User-Agent: GPTBot — and get treated like the real thing.
What's actually happening
Every request carries a User-Agent: a short label saying what sent it. Real crawlers use theirs honestly. But the label is chosen by whoever sends the request, and nothing stops a scraper on any server from writing “GPTBot” or “Googlebot” in it.
When you serve crawlers something special — an AI-optimized version of each page, or a fully rendered copy of a JavaScript site — that label is how they are recognized. A fake one gets the same treatment. The scraper walks away with the clean, structured copy you prepared for AI, and every uncached page it asks for is processed and counted against your plan's monthly optimizations.
The fakes show up in your crawler numbers too. Traffic that looks like GPTBot is read as interest from ChatGPT, when some of it may be someone else entirely. Until you check, there is no way to tell how much.
1 line
of text is all it takes to claim to be a crawler
4
vendors publish IP ranges you can check a claim against
0
changes to serving until you choose to enforce
How Wrenda fixes it
Check where the request came from, not just what it says. Google, Bing, OpenAI and Perplexity publish the IP ranges their crawlers use, so a claim can be tested.
- 01A request claims to be Googlebot, Bingbot, GPTBot or PerplexityBot, and your rule says to optimize or pre-render it. Wrenda compares the address it came from against that vendor's published ranges.
- 02The verdict is recorded: verified, unverified, or no ranges for vendors that publish none, such as Anthropic for ClaudeBot. “No ranges” is never treated as a fake.
- 03Your domain's dashboard shows what share of checked claims failed and which user agents sent them. By default that is all that happens: nothing changes until you have seen the evidence.
- 04When you are ready, choose Demote, which serves an unverified claim your normal page instead of the optimized copy, or Block, which returns a 403. Verified crawlers are served exactly as before.
What changes after you install
Know how many claims are real
See what share of checked crawler claims failed, and the user agents behind them, before deciding anything.
Fakes lose the optimized copy
In Demote or Block, a claim that fails the check never receives the AI-optimized or pre-rendered version.
Failed claims stop spending your allowance
A demoted or blocked claim triggers no optimization, so it does not count against your plan.
Real crawlers are never caught out
Vendors without published ranges are never penalized, and a vendor outage keeps the last good ranges in use.
Setting up crawler rules? Read the bot handling docs or see what AI optimization serves to the real ones.
Check the claim
Serve the real crawlers, not the copies.
Included on every plan. Observe first, enforce when the evidence says so.
Start 14-day trial14-day free trial — No credit card required