AI search news ·

Does Googlebot still fetch new posts on publish? 8 posts, zero Googlebot visits at 24–75 hours — OAI-SearchBot hit all 8 inside 12

Every post this site published between August 18 and August 20 — eight pages — has zero Googlebot fetches on our server-side crawler ledger, at ages 24 to 75 hours. The same ledger shows OAI-SearchBot fetching all eight within 12 hours of publish, PerplexityBot within 18, and YandexBot or bingbot reaching seven of the eight within 3 minutes. Googlebot itself is not away: it made 122 requests to this site over the ledger's 4.6 days — but every content page it touched was published August 17 or earlier. Here is the per-post ledger, Google's own documentation for the states Search Console reports, and the one control that shows the pipe still works when Googlebot does fetch.

The ledger, post by post

PostPublished (UTC)Age at readGooglebotOAI-SearchBot afterSearch Console state at read
82%-single-URL citation study08-18 13:5674.6 h0 fetches22 minDiscovered – currently not indexed
Search Console logging-bug post08-18 16:4871.7 h0 fetches5.6 hDiscovered – currently not indexed
llms.txt crawler-census post08-18 16:4871.7 h0 fetches5.6 hURL is unknown to Google
79% source-reuse study08-19 13:5650.6 h0 fetches9.9 hURL is unknown to Google
Crawler fetch-speed post08-19 16:4947.7 h0 fetches7.0 hDiscovered – currently not indexed
Reddit citations post08-19 16:4947.7 h0 fetches7.0 hURL is unknown to Google
Buried-pages referral post08-20 13:5526.6 h0 fetches11.2 hDiscovered – currently not indexed
IbouBot/PetalBot post08-20 16:4523.7 h0 fetches8.4 hDiscovered – currently not indexed

Method: the ledger is a server-side log of user-agent-classified crawler requests (probe rows excluded), born 2026-08-17T01:48Z; publish instants are production-deployment creation times (±1 minute); Search Console states are from the URL Inspection API, read around 16:30 UTC today. PerplexityBot fetched all 8 at 7.2 to 17.9 hours, Amazonbot 7 of 8, and the first conventional search-engine bot (YandexBot or bingbot) arrived at 1 to 25 minutes — 7 of 8 inside 3 minutes. This extends the fetch-speed measurement we published August 19, where Googlebot fetched the 08-17 cohort at 11 to 34 hours: since August 18 the lag is no longer a lag, it is an absence. The machine-readable cut is saved alongside this post.

Googlebot is here — it just is not fetching anything new

In the same 4.6-day window Googlebot made 122 requests to this site: 51 for robots.txt, 11 for sitemap.xml (most recent at 16:24 UTC today), and 60 page fetches across 33 distinct paths — at least 9 of those page fetches happened today. Every one of the 33 paths was published 2026-08-17 or earlier. So this is not a crawler that stopped visiting; it is a crawler that keeps re-reading the back catalog, and the sitemap that lists the new URLs, without fetching the new URLs themselves.

The control: when Googlebot does fetch, indexing still follows

One post broke the pattern this morning — in the right direction. Our August 13 post on AI traffic in analytics had been stuck for 8 days across 8 consecutive non-indexed Search Console reads, twice regressing to 'URL is unknown to Google'. Today at 09:38:02 UTC the ledger recorded its first Googlebot fetch; the URL Inspection API reports lastCrawlTime 09:38:03 — the same second — and by this afternoon's read the state was 'Submitted and indexed'. The ledger and Search Console agree to the second when a fetch actually happens, which is what makes the zeros on the other eight rows a real absence of crawling rather than instrument blindness.

What Google's own documentation says

Google's Page Indexing report documentation defines 'Discovered – currently not indexed' as: "The page was found by Google, but not crawled yet. Typically, Google wanted to crawl the URL but this was expected to overload the site; therefore Google rescheduled the crawl." The URL Inspection documentation defines 'URL is unknown to Google' as "Google hasn't seen this URL before." Both sit oddly beside the ledger: a site receiving 26 Googlebot requests a day is hard to overload with one more fetch, and three of the eight URLs Google now reports never having seen are listed in a sitemap Googlebot fetched 11 times in five days — two of them read 'Discovered' yesterday. The states are also churning: four of the eight flipped between 'Discovered' and 'unknown' between yesterday's inspection and today's, in both directions.

Why this matters if you care about AI visibility

Limits

One small, 70-day-old site — n=8, and 'Discovered – currently not indexed' is a common state for new sites with little authority; Google promises no crawl schedule, and nothing here shows a policy change rather than ordinary crawl-budget triage of a low-priority site. The ledger is 4.6 days old and classifies by self-declared user agent without IP verification. The observation is censored: any of the eight could be fetched an hour after this publishes — we will report the first Googlebot fetch and indexation of this cohort, with timestamps, as we did for the control above. If AI engines are now the first readers of your newest pages, the question that matters is what they say when asked about your business — a free check reads that directly.

See your number

See which businesses AI names when your client's buyers ask.

Running this for clients? The $249 agency 5-pack audits five businesses, white-labeled.

Check a client's AI visibility

Begin your check

Free · 60 sec

No account · No card · 3 buyer questions, 2 engines

By running a check you agree to our Terms and Privacy Policy.

Who runs this