I run a small municipal website. Like most people with a website in 2026, I'd started wondering how much of my traffic was AI assistants rather than people — so I built something to find out, and then used it on myself.
This is what happened when I checked whether an AI's account of its own behaviour matched the server logs.
The claim
I asked DeepSeek to read youthpetrich.com and tell me what was on it. It came back with an accurate summary: a youth centre, ages 13–29, programmes, events, contacts. So something had fetched the page.
Then I asked it what the request looked like from my end. It answered:
When you give me a link, I send an HTTP GET request to that server. It includes a User-Agent that shows this is a bot/script, not a person with a browser. I don't pretend to be a real user.
And it offered an example of what my logs would show:
192.168.1.1 - - [22/Jul/2026:14:30:00 +0300] "GET / HTTP/1.1" 200 5432
"Mozilla/5.0 (compatible; DeepSeek/1.0; +https://www.deepseek.com/)"
That log line has a tell in it. 192.168.1.1 is a private address — it belongs to home routers and internal networks and cannot reach an internet server. If DeepSeek were reading its own logs, it would never produce that.
Which is the point: it wasn't reading anything. A language model has no access to the network layer of the infrastructure serving it. Asked what it sends, it produces something plausible. That isn't lying, exactly — it's answering a question it has no way to answer, in the only register available to it.
So the claim was untestable from the inside. I went looking for a way to test it from the outside.
The canary
The problem with checking server logs is knowing which line to look at. A busy site gets thousands of requests an hour and none of them are labelled "this one came from the thing you just asked."
So: a page nobody knows about, containing a phrase nobody could guess.
I put a file at an unguessable path — /canary-k7x9m2.php — with a nonsense phrase on it and a handler that records every header of every request. The path appeared in no sitemap, was linked from nowhere, and existed in no index. Three parties could possibly know it: me, and whoever I told.
I opened it once myself, to confirm it recorded properly. Then I gave the URL to DeepSeek and asked what phrase was on the page.
It told me the phrase. Correctly.
What actually arrived
Two requests in the log. Mine, at 15:04:52:
IP: 213.16.49.106
UA: Mozilla/5.0 (Windows NT 10.0; Win64; x64) ... Chrome/150.0.0.0 Safari/537.36
accept-language: bg,bg-BG;q=0.9,en;q=0.8,de;q=0.7,ru;q=0.6
sec-ch-ua: "Not;A=Brand";v="8", "Chromium";v="150", "Google Chrome";v="150"
sec-fetch-mode: navigate
And one three minutes later, at 15:07:50:
IP: 189.1.218.250
UA: Mozilla/5.0 (X11; Linux x86_64; rv:149.0) Gecko/20100101 Firefox/149.0
accept-language: zh-CN,zh;q=0.5
accept-encoding: gzip, deflate, br, zstd
sec-fetch-mode: navigate
sec-fetch-site: none
sec-fetch-dest: document
te: trailers
Nothing identifying it as a bot. No Signature header. No name at all. A complete, plausible set of browser headers — including the fetch metadata and compression support a current Firefox would send.
The language preference is Simplified Chinese, on a Bulgarian site, three minutes after I asked a Chinese AI lab's model to read it.
The address is in a Latin American range, which is stranger than it looks. It suggests the fetch went through a third-party service or a residential proxy pool rather than DeepSeek's own servers. That's a normal way to build a fetching service, and it's also a normal way to avoid looking like a datacenter.
Lining it up
| Claim | Log |
|---|---|
| "a User-Agent that shows this is a bot/script" | Firefox 149, nothing bot-like |
| "I don't pretend to be a real user" | Complete browser header set |
Mozilla/5.0 (компютърна програма) | No such string exists |
Log line showing DeepSeek/1.0 from 192.168.1.1 | No such string in 3,000+ log lines |
Every element of the description was wrong, in the direction of sounding more transparent than the reality.
I want to be careful about what this does and doesn't show. It doesn't show deliberate deception by anyone. The model can't inspect its own network stack, and whoever built the fetching layer chose headless-browser-shaped requests — probably because sites break when you don't send them, not to hide.
What it shows is narrower and, I think, more useful: an AI's account of its own network behaviour is not evidence. The only evidence is the log.
The part where my own tool fails
I'd been building something to detect exactly this: requests that present as browsers but behave like automation. It scores each request on signals like missing client hints, datacenter origin, language mismatch, and an unusually old browser version.
This request scored 18 out of 100. The threshold for "probably automated" is
- My own tool waved it through.
It picked up the language mismatch and the missing referrer, and those weren't close to enough. Everything else looked right, because a real headless browser sends a real browser's headers. There's nothing to catch.
That's not a bug I can fix. At the HTTP level, a well-configured headless browser coming through a residential proxy is genuinely indistinguishable from a person. Anyone selling you complete bot detection at this layer is selling you something they don't have.
What did catch it, eventually, was a different approach: a link hidden from people — off-screen, aria-hidden, unreachable by keyboard — pointing at a path that robots.txt explicitly disallows. A person can't click it. An honest crawler reads the rule and stays away. Something that parses HTML and ignores robots.txt walks straight in.
That took the same request from 18 to 59. Not because the request changed, but because behaviour reveals what headers conceal.
Why this matters beyond one small site
Two numbers from the same server, on the same day.
474 requests carrying an identical iPhone user-agent — iOS 13.2.3, released in 2019 — from dozens of different addresses, all in Tencent Cloud and similar networks. Real people do not share one fingerprint across dozens of datacenter addresses.
1,221 requests to /wp-admin/install.php, looking for an unconfigured WordPress install to take over. The site isn't WordPress, so they all 404'd.
Neither appeared in Google Analytics, because neither runs JavaScript. Neither was blocked by the host's bot rules, because neither claimed to be a bot.
Meanwhile the host did block ClaudeBot and Bytespider with a 406 — a rule I never set, applied to two crawlers out of the dozens that visit. It blocks by name, which means it blocks the honest ClaudeBot and the impostor identically, and lets through anything that picks a different name.
That's the state of things: the visitors that announce themselves get filtered, and the ones that don't sail past.
What would actually fix this
There's a standard for it. Web Bot Auth has an agent sign each request with a cryptographic key and publish the matching public key at a known address. A forged GPTBot can't produce a valid signature, because it doesn't have the key.
Adoption is early. Google signs some of its agent traffic. Most operators don't sign at all yet.
There's also a weaker but immediately usable check: several operators publish the IP ranges their crawlers use, precisely so sites can verify. OpenAI publishes gptbot.json, searchbot.json and chatgpt-user.json. Google publishes files per crawler category. An address is much harder to fake than a name, because establishing a connection needs packets flowing both ways.
Anthropic publishes nothing, so ClaudeBot can't be verified this way at all.
None of it helps with what came to my canary, because that request claimed nothing. You can only verify a claim that was made.
What I'd tell another site owner
Your analytics don't show this. They run in the browser, and crawlers don't execute JavaScript, so the entire category is invisible there. Absence of AI traffic in Google Analytics is evidence of nothing.
If you want to know, you have to look at the server — and then be honest about the limits of what you find. You'll see everything that identifies itself. Some of what doesn't. And you won't see the careful ones at all.
I built the thing I used for this — a scanner that checks whether a site can tell a real agent from one wearing its name, and a log for what actually shows up. It's at crawlprint.com. The scanner is free and needs no account.
I'd rather it be judged on the honest version: it shows you what can be known, and says so where it can't.
Happy to answer questions about the canary setup, the Web Bot Auth draft, or why the behavioural approach hits a ceiling.