Original data
How often do AI crawlers revisit your site? Our numbers, bot by bot
For 90 days we measured how often each AI bot actually comes back to re-read a page — across 17,850 page-bot pairs, with every bot's identity verified against published IP ranges. Perplexity returns every 5.5 days. ChatGPT every 14. Claude every 23.5. Here are the numbers, the method, and what they mean for every page of yours that changes.
Change a price today. How long does the old price keep living inside AI answers?
Nobody can answer that question exactly — an assistant's memory is not observable from the outside. But half of the question measures very well: how often each AI bot actually comes back to re-read your pages. Between two visits, whatever else happens, a bot cannot learn anything new about a page. That recrawl cycle is a physical floor: your change cannot propagate faster than it.
We measured that cycle for 90 days. Here is what it looks like.
The method, before the numbers
Snorklee measures traffic server-side, including bot visits. Every bot hit is checked against the IP ranges published by the operators: a fake GPTBot is discarded. On that basis:
- Window: 90 days, closing August 6, 2026.
- Material: 17,850 page-bot pairs re-read on at least 3 distinct days, from the sites we measure in production. This is real observational data — not a panel representative of the whole web.
- Calculation: per pair, cycle = (last visit − first) ÷ (number of visit days − 1); then the median per bot.
- Excluded: spoofed bots, and families seen on fewer than 100 pairs (DuckDuckGo AI, CommonCrawl) — too few for an honest median.
- Stated limit: a crawl is not an index update, nor a model-memory update. We measure the re-reading, not what the assistant serves. But no re-reading means no update is possible: these cycles are floors.
The numbers: from 5.5 days to over three weeks
Median recrawl cycle for the same page, per bot (90 days)
| Bot | Pairs observed | Median cycle | In practice |
|---|---|---|---|
| Perplexity | 464 | 5.5 d | Your change can be re-read within the week |
| Huawei AI | 3,230 | 6.5 d | Weekly re-reading |
| ByteDance AI | 2,447 | 8 d | Weekly re-reading |
| Meta AI | 647 | 11 d | One to two visits per fortnight |
| ChatGPT | 8,231 | 14 d | Two weeks between background reads |
| Amazon AI | 2,661 | 18 d | Almost three weeks |
| Claude | 158 | 23.5 d | Over three weeks: the slowest we measured |
Three things struck us in these numbers.
Perplexity is the freshest. Consistent with its answer-engine positioning: an index that wants to be current has to re-read often. If your content changes a lot, Perplexity is probably where the served version is closest to reality.
The crawlers nobody talks about are among the most diligent. PetalBot (Huawei) and Bytespider (ByteDance) re-read faster than ChatGPT — on our data they account for almost a third of all observed pairs between them. The public conversation about AI crawlers is mostly OpenAI and Anthropic; the actual traffic tells a different story.
Claude is the slowest, by far. Over three weeks between two background reads of the same page. Stale information therefore lives longer at Anthropic than anywhere else — with a live-reading nuance we get to below.
ChatGPT's two speeds
The 14-day median hides a two-tier reality, very visible on our own pages.
Our pricing page — the kind of page whose content matters financially — was re-read by ChatGPT 41 times in 90 days, roughly every 2 to 3 days. Meanwhile the median of its background crawl across all pages stays at 14 days. In other words: AI crawlers keep a hot list of pages they re-read very often, while the rest of the site lives at the slow rhythm.
There is a third mechanism on top: ChatGPT-User, the live read. When a user asks a question that involves your page, the assistant can go re-read it that very moment — short-circuiting every cycle above. But on-demand reading only covers the pages the assistant decides to fetch: for everything else, the background cycle rules.
The practical consequence matters: the real question is not "how fast do AIs crawl the web" but "are my critical pages on the hot list". That answer is measurable, page by page — not on average.
What this changes if your pages move
If your site is a stable brochure, these cycles are trivia. If your pages carry information that expires — prices, stock, promotions, versions, subscription tiers — they become a scheduling constraint:
- A 5-day promotion can fall between two reads. At ChatGPT (14 d), Amazon (18 d) and Claude (23.5 d), a short campaign will very likely start and end without any background crawl seeing it. For half the bots in our table, your promotion simply never existed.
- A price change lives twice for one to three weeks. The old price remains the last version some bots have read until their next visit. If an assistant quotes a stale price to someone comparing offers, you will never know — the sale is lost without a trace.
- Perishable information must be readable on the first pass. Price and availability in the HTML and in
schema.org/Offermarkup, not only in JavaScript that crawlers don't execute. A bot that returns every 14 days and cannot read the price leaves for another 14 days with nothing. - Don't block the bots that send you traffic. The "block all AI bots" reflex cuts exactly the re-reads that keep your information fresh — and the assistant traffic that comes with them. If you want a fine-grained policy, separate training from search: we detailed how in "AI crawlers: block or allow?".
- Schedule against the observed cycle of your pages, not a web-wide average. Our medians are reference points, not your reality: your homepage doesn't move at the rhythm of your deep pages. The only number that matters for a decision is the one measured on your pages.
Measure it on your own site
Everything in this article comes out of Snorklee's AI traffic tab: which bots read which pages and when (identity verified against IP ranges, impostors flagged), which of your pages get read but never visited by a human, and what your robots.txt actually allows. Measurement is passive and server-side, with no cookie and no banner — your visitors see nothing, and neither do the bots.
See your own cycles: free 14-day trial, no credit card — the first bot visit usually shows up within a day.
How often does ChatGPT crawl a site?
On our data (90 days, IP-verified identity), the median background crawl re-reads a page every 14 days. But pages judged important get re-read far more often — our pricing page every 2 to 3 days — and ChatGPT-User can read a page live the moment a user asks a question. Frequency depends on the page, not just the site.
Does a crawl mean the AI is up to date?
No. A crawl is a read, not a guarantee that the index or the model's memory was updated. The reverse logic does hold, though: without a re-read, no update is possible. The crawl cycle is a floor — your change cannot propagate faster.
Should you block these bots to protect yourself?
Blocking every AI bot also cuts the ones that power answer engines and send you visitors. The useful distinction is by purpose: blocking training is a defensible choice; blocking search and assistants means opting out of tomorrow's results. A robots.txt can do one without the other.