ClaudeBot Impersonation: AI Crawler Credential-Theft Threat

Every program that visits a website announces itself in one line of the request. Chrome says it is Chrome. Googlebot says it is Googlebot. Anthropic's crawler says it is ClaudeBot. As GreyNoise put it, nothing in the request itself proves any of it is true.

On August 28, 2026, the security-intelligence firm disclosed that automated scanners are impersonating the web crawlers of OpenAI, Anthropic, Google, DeepSeek and others, using exact user-agent strings, to hunt for exposed credentials. For any business that whitelists AI crawlers so its content shows up in ChatGPT, Claude or Perplexity answers, this is a security boundary, not a content-policy debate.

What GreyNoise found: 824 addresses, 13 fake crawler names

Between July 28 and August 23, 2026, GreyNoise observed a cluster of scanners posing as 13 AI crawler names from eight companies, requesting .env files, cloud access keys, private keys and password stores from exposed web servers. Six names, belonging to Anthropic, OpenAI, Google and Perplexity, arrived from the same 824 IP addresses in almost identical volume; none ever requested /robots.txt.

The addresses are spread across 795 separate /24 networks, so there is no single network to block. GreyNoise checked all 824 against the published IP ranges for Anthropic, OpenAI, Google, Perplexity and Amazon; none matched. Two of Amazon's crawler names were also forged, under user-agent strings Amazon does not document. And Google-Extended, a robots.txt opt-out token with no user-agent string of its own, was sent as a user agent in 263,849 sessions, every one forged by definition.

Two caveats matter. GreyNoise confirmed requests, not exfiltration: nothing in its data shows a file was returned or that any organization was affected, and it is not naming the operator. The finding also does not mean all crawler traffic is fake; thousands of real ClaudeBot sessions arrived from Anthropic's published range (216.73.216.0/22) over the same window. But the impostor's ClaudeBot string matches Anthropic's character for character, so no user-agent rule can tell them apart.

Why an AI crawler allowlist can't protect you

The user agent is a client-supplied header. A control that checks the name but not the address can be bypassed by forging the name, as this cluster did. GreyNoise traced it with JA4/JA4H fingerprinting, which identifies the software making the requests, not the machine. The same fingerprint carried more than 1,500 user-agent strings over the previous 90 days, most identifying as ordinary browsers.

Behavior does not separate them either. A real crawler reads /robots.txt first; Anthropic's real crawler requested it more than any other path, 12% of its traffic, and never asked for a credential file. The impostors inverted both signals, and nearly every forged session carried the same "Web Crawler" label real traffic carries. GreyNoise's rule: allow only when the fingerprint and a published address agree.

What the scanners were after

The targeted paths read like a checklist of everything a developer leaves lying around. The most requested were /.env and its variants, /.aws/credentials and /.git/config. Independent telemetry from Known Agents adds the files AI coding assistants write when configured: /.claude/settings.json, /.codex/config.toml, /service-account.json, /terraform.tfstate and /rclone.conf.

An .env file is where applications keep their secrets: database passwords, API keys, cloud credentials. If a copy sits in a web-accessible directory, a request for /.env returns the whole thing. GreyNoise tagged the activity with labels including ENV Crawler, AWS Configuration Scanner and EC2 IAM Credential Access Attempt, and tied it to CVE-2025-30208, a Vite file-disclosure flaw patched earlier this year.

The AEO trade-off: visibility vs. exposure

This is where the finding lands on your business. If you allow GPTBot, ClaudeBot, OAI-SearchBot or PerplexityBot so your content surfaces in AI answers, you are exactly the profile these impostors target. As PPC Land put it, an allowlist granting access on the strength of a name grants that access to an impostor sending the same name.

The exposure is real, and so is the cost of overreacting. DataDome tested 698,214 live sites with a spoofed assistant user agent; 79.7% did not block or challenge the impersonation, and 79.2% returned an HTTP 200. Rutgers and Wharton research found news publishers that blocked model crawlers lost roughly 7% of weekly traffic within six weeks. Known Agents puts Googlebot as the most impersonated identity at 0.5%; ClaudeBot, GPTBot and PerplexityBot each run around 0.1%.

The answer is not "ban all AI crawlers" and it is not "trust the name." It is verify, then allow. That verification is part of a broader AI agent security risks review and belongs in the AI agent risk checklist; for the full walkthrough, see our step-by-step AI agent security audit.

How to verify an AI crawler (checklist)

Verification is a two-signal check: the address must come from the vendor's published range for the exact crawler name, and the request must behave like a crawler. Work through this list before granting any AI crawler access:

1. Never treat a user-agent string as identity. Any program can claim to be GPTBot or ClaudeBot; the header is not a credential.
2. Check the source IP against the vendor's published range for the exact crawler name you saw. Anthropic: claude.com/crawling/bots.json. OpenAI: openai.com/gptbot.json, /searchbot.json, /chatgpt-user.json. Google: developers.google.com/static/crawling/ipranges/common-crawlers.json. Perplexity: perplexitybot.json. Amazon: amazonbot IP list.
3. Require both signals to agree: IP in the published range AND plausible crawler behavior.
4. Flag any "crawler" that never requests /robots.txt. Real crawlers fetch it first and cache it; judge across days, not single visits.
5. Alert on credential paths regardless of user agent. /.env, /.aws/credentials, /.git/config, service-account.json, terraform.tfstate.
6. Put an IP-range check behind every place a crawler name grants access: allowlists, WAF rules, rate-limit bypasses.
7. Keep .env, .git and cloud credential files out of the web root, and rotate any key that was ever reachable from a web path.
8. Assign an owner and a refetch schedule for each vendor IP list. Google refreshes daily; a stale list admits impostors or flags real crawlers.
9. Don't block everything to be safe. Use robots.txt for opt-outs; Anthropic warns that IP-blocking can impede its crawler from reading robots.txt.
10. Treat red-flag strings as proof of forgery. Google-Extended as a user agent is always forged; Google documents that it has no HTTP user-agent string.

The DeepSeek problem: you can't verify what isn't documented

DeepSeek appears in GreyNoise's headline and in the wider 13-name set, but it is not one of the six detailed names. Independent trackers report DeepSeek has not published official crawler documentation, a verified user-agent string, or IP ranges; it crawls without a named identifier, indistinguishable from ordinary browser traffic. Any "DeepSeekBot" rule copied from a robots.txt template is unverified, and you cannot allowlist an identity with no published address list to verify it against. If DeepSeek visibility matters, use the documented channels that feed the engine.

Google-Extended isn't a crawler (and other red flags)

Google-Extended is a robots.txt control token publishers use to manage whether crawled content may be used for training Gemini models. It is not a crawler and has no HTTP user-agent string, so any request carrying it as a user agent is forged by definition. A clean red flag for your logs.

Other signals worth alerting on: credential-path requests, no /robots.txt fetch before deeper paths, unusual path volume from one address, and any attempt against /.git/config or /service-account.json.

What to do this week

You do not need to wait for a vendor announcement. GreyNoise published all 824 addresses as a CSV, so you can run the same checks against your own logs.

1. Search your access logs for those 824 addresses and for any credential-path request above; confirm those paths return 404. If any return content, treat every secret as exposed and rotate.
2. Move .env, .git and cloud credential files out of the web root, reject these paths at the edge, and rotate any cloud key that was ever reachable from a web path.
3. Assign an owner and refetch schedule for each vendor IP list; Google refreshes daily.
4. If you run Vite, confirm you are on a patched release (6.2.3, 6.1.2, 6.0.12, 5.4.15 or 4.5.10 or newer).

For the broader picture, our AI infrastructure security gateways guide covers where these controls sit in your stack, and our Amazon AI training data audit shows the same allowlist question from a vendor's data-collection posture.

The AI-era web runs on trust in a self-reported name, and that trust is now billed like any other identity control. Verify the address, keep secrets out of the web root, and stay visible to the crawlers that are who they say they are.

Run the free AI audit tool

Or work through the AI agent risk checklist to see where else your AI tooling is exposed.

Frequently asked questions