01
They arrive wearing a name
A request says it's GPTBot or ClaudeBot right there in the User-Agent. But that text is just a string — anyone can type it. A scraper hiding behind a trusted name looks identical to the real thing.
Web Bot Auth · 2026
A dozen AI agents visited your site today.
Can it tell which ones were real?
AI assistants now browse the web on behalf of the people using them. The honest ones are starting to prove who they are with a signature. The rest just borrow a famous name — and most sites wave every one of them straight through, real or not.
No signup. One read-only request per identity. Takes about ten seconds.
SUSPECT LINEUP
ChatGPT-User
OpenAI
UNVERIFIED
claimed identity,
no proof
ClaudeBot
Anthropic
UNVERIFIED
claimed identity,
no proof
PerplexityBot
Perplexity
UNVERIFIED
claimed identity,
no proof
Google-Agent
Google
UNVERIFIED
claimed identity,
no proof
Googlebot
Google
UNVERIFIED
claimed identity,
no proof
02
Now some carry proof
A new standard, Web Bot Auth, lets an agent cryptographically sign its requests — real proof of identity, not a claim. Google and a handful of others have started. Most haven't yet.
03
Your site probably ignores both
It treats the signed, the unsigned, and the outright forged exactly the same: it lets them in. The scanner shows you that blind spot in plain terms — then Crawlprint helps you watch it.
See who's really reading your work.
Run the scanner on a site you own. It takes ten seconds and needs nothing installed.