AI search news ·

Googlebot fetched our blog index after 58 days and again 19.6 hours later: 1 of the 14 posts it links is indexed 6 days on, 13 have 0 Googlebot fetches in 7 days, and PerplexityBot fetched 8 of the 14

On 2026-09-02 14:49:18 UTC, Googlebot fetched our /news index page for the first time since 2026-07-06, a gap of 58 days, and fetched it twice more at 10:26:22 and 10:26:23 UTC on 2026-09-03, 19.6 hours after the first. The index had been rebuilt on 2026-08-28 as the internal-linking variable for 14 posts that Google had discovered and not crawled. 6 days after that ship, and one day after Googlebot read the index, 1 of the 14 posts is indexed. 12 read "Discovered, currently not indexed", 1 reads "URL is unknown to Google", and the 13 not-indexed posts have 0 Googlebot rows on our server ledger in 7 days. In the same 7 days Googlebot made 280 requests to the site; PerplexityBot fetched 8 of the 14 posts and Amazonbot 13. Two independent channels agree on the index fetch to the second. Method and every number below come from the committed artifact (hub-read-series-cut-2026-09-03.json, 19 of 19 self-checks); the 2026-09-11 read and its rule were written down on ship day.

What was shipped, and what was written down before it shipped

By 2026-08-27, 14 of our blog posts, published 2026-08-19 to 2026-08-27, had sat in Search Console as "Discovered, currently not indexed" or "URL is unknown to Google" with no Googlebot fetch on our ledger. Our earlier test of the Request indexing button is in the 2026-08-28 post; the button is not a lever we plan on. On 2026-08-28 we shipped one variable instead: a rebuilt /news index grouped by month, with rel="next" and rel="prev" links on every post, so that each of the 14 is reachable from a page Google already indexes. The 14 URLs were pinned in a fixture that day. The read was pre-registered for 2026-09-11 with one rule: at least 7 of 14 indexed means the index works; 0 of 14 means internal linking is not the constraint; anything between is inconclusive and is reported as such. One instrument makes the read every day: it inspects each URL through the Search Console API and counts Googlebot rows on our server ledger for the same path over the trailing 7 days.

The index page itself: two channels, one second

The index page had not been crawled by Google since 2026-07-06 18:12:19 UTC (the lastCrawlTime in our 2026-08-23 sweep of every URL) and had 0 Googlebot rows on the ledger from the ledger's birth on 2026-08-17 to 2026-09-01, in 22 rows from all crawlers. Then:

ChannelReading at 2026-09-03 16:38 UTC
Search Console URL Inspection, lastCrawlTime2026-09-02 14:49:18 UTC ("Submitted and indexed")
Previous lastCrawlTime (2026-08-23 sweep)2026-07-06 18:12:19 UTC
Gap between the two Google crawls58 days
Server ledger, Googlebot rows on /news, trailing 7 d3: 2026-09-02 14:49:18.392 UTC · 2026-09-03 10:26:22.969 UTC · 2026-09-03 10:26:23.018 UTC
Ledger row matching lastCrawlTime to the secondyes
Googlebot rows on the ledger newer than lastCrawlTime2 (the two 2026-09-03 rows; Search Console had not updated by the read)
Hours between the first fetch and the second19.6
Googlebot rows on /news, ledger birth to 2026-09-010
All crawler rows on /news, trailing 7 d21

The two 2026-09-03 rows are 49 ms apart, the shape of a desktop and a smartphone Googlebot pair. Search Console still reported the 2026-09-02 crawl at the read; the API's lastCrawlTime lags the fetch itself, which the ledger sees at once. Rows on /news by crawler in the same 7 days:

Vendor/kindRows on /news, 7 d
other/other5
openai/user3
google/search3
google-notebooklm/user2
openai/crawl2
perplexity/crawl2
microsoft/search1
apple/crawl1
amazon/crawl1
you/crawl1

The 14 linked posts, seven daily reads

Each row is one run of the instrument at about 16:40 UTC. "Flips" counts URLs whose Search Console state differed from the previous day's read. The one URL that went in did so on read 3 (2026-08-30); its lastCrawlTime is 2026-08-30 04:29:48 UTC and it carries the only Googlebot rows on the denominator.

ReadDate (UTC)Days after shipIndexedDiscoveredUnknownFlips vs previous readGooglebot rows on the 14, trailing 7 d
12026-08-280077first read0
22026-08-291013180
32026-08-30219461
42026-08-313110371
52026-09-01414981
62026-09-02515872
72026-09-036112192

45 state changes across 6 transitions on 14 URLs, and 1 of them is a change in indexation. The rest is the API alternating between "Discovered" and "unknown" for the same never-fetched URL; every not-indexed URL has flipped at least twice. The per-URL view, with the state on each read as a letter (D discovered, U unknown, I indexed):

Post (slug, truncated)PublishedAge at read (d)State on read 7Reads 1 to 7FlipsGooglebot rows, 7 d
can-chatgpt-send-a-reader-to-a-page-before-googl…2026-08-2411Discovered, not indexedDDDDUUD20
do-new-ai-cited-sources-survive-a-week-84-of-279…2026-08-2411Discovered, not indexedUDUDUUD50
what-queries-does-google-show-a-small-ai-search-…2026-08-2312Discovered, not indexedDUDUUDD40
how-many-visitors-do-ai-engines-actually-send-a-…2026-08-2114IndexedDDIIIII12
what-are-iboubot-and-petalbot-a-source-citing-fr…2026-08-2015Discovered, not indexedUDDDUDD30
does-your-json-ld-still-parse-after-googles-sing…2026-08-2312Discovered, not indexedUDDDDUD30
do-page-one-google-rankings-earn-clicks-for-a-sm…2026-08-2213Discovered, not indexedDDUDDDD20
five-external-visitors-in-74-days-reached-our-ch…2026-08-2510Discovered, not indexedUDDDUUD30
four-weekly-runs-576-ai-answers-askedabout-cited…2026-08-2510Discovered, not indexedDDDUDUD40
where-do-ai-crawlers-actually-go-chatgpt-user-71…2026-08-269Discovered, not indexedDDUDUUD40
does-googles-request-indexing-button-actually-wo…2026-08-269Discovered, not indexedUDUDDUD50
what-did-our-pitch-emails-actually-say-96-of-97-…2026-08-278Discovered, not indexedDDDUUUD20
fifteen-ai-referrals-in-76-days-one-tap-zero-for…2026-08-278Unknown to GoogleUDDDUDU40
did-chatgpt-stop-citing-reddit-on-august-14-prom…2026-08-1916Discovered, not indexedUDDDUDD30

Where Googlebot went instead

Googlebot is not absent from the site. In the 24 hours to the read it made 38 requests to 18 distinct paths, 2 of them the index page and 0 on the 14 posts the index links. In 7 days: 280 requests to 60 paths, 2 on the denominator (both on the one indexed post), 0 on the five posts published since 2026-09-01. The 92 requests to /news posts covered 36 distinct posts, published 2026-07-09 to 2026-08-22; the newest of them is 12 days older than the newest post on the site (published 2026-09-03).

Path classGooglebot rows, 24 hGooglebot rows, 7 d
robots.txt1396
sitemap.xml217
/news posts992
/news hub23
/compare524
/guide222
/data210
/for28
home18
UTC dayGooglebot rows
2026-08-2723
2026-08-2850
2026-08-2937
2026-08-3021
2026-08-3135
2026-09-0150
2026-09-0236
2026-09-0328

Who did fetch the 14 posts in the same 7 days

CrawlerPosts fetched, of 14Fetches on those posts
Amazonbot1315
PerplexityBot89
OAI-SearchBot33
bingbot28
YandexBot22
Googlebot12
ClaudeBot11
GPTBot00

8 of the 14 were fetched by at least one AI-engine crawler (PerplexityBot 8, OAI-SearchBot 3, ClaudeBot 1, GPTBot 0). These are re-fetches: our 2026-09-01 read found ClaudeBot, OAI-SearchBot and PerplexityBot had each fetched all 17 of the posts published 2026-08-24 to 08-31 within days of publish. The Google side of the same posts has not moved: 13 of 14 with 0 Googlebot rows in 7 days.

The five newest posts, for the record

Outside the pre-registered denominator, the posts published 2026-09-01 to 09-03 were inspected in the same run: 4 read "Discovered, currently not indexed", 1 reads "URL is unknown to Google", and all five have 0 Googlebot rows in 7 days. They will be their own series; they are not added to the 14.

Post (slug, truncated)PublishedState at readGooglebot rows, 7 d
page-one-ctr-jumped-to-1-37-percent-because-one-ur…2026-09-01Discovered, not indexed0
claudebot-oai-searchbot-perplexitybot-and-yandexbo…2026-09-01Discovered, not indexed0
which-ai-engines-send-visitors-to-a-small-site-and…2026-09-02Discovered, not indexed0
three-of-three-pre-registered-crawler-readings-hel…2026-09-02Unknown to Google0
a-cold-email-restart-at-24-hours-26-sends-to-16-ag…2026-09-03Discovered, not indexed0

What this means if you care about AI visibility

Limits

The next reading, pre-committed

On 2026-09-11 the same instrument runs on the same 14 URLs and the verdict is scored by the rule written on 2026-08-28: at least 7 indexed, the index works; 0, internal linking is not the constraint; between, inconclusive. It is published either way. Two lines are falsifiable before then: if Googlebot rows appear on any of the 13 zero-row posts within 7 days of the 2026-09-03 index fetches, they are named in the 09-11 post with their timestamps; if none do, the sentence "Googlebot read the index three times and followed none of its links to unfetched posts" is restated at 14 days.

Does Googlebot crawling a blog index page get the linked posts indexed?

Not on its own, in our case. Googlebot fetched our rebuilt /news index on 2026-09-02 and twice on 2026-09-03; 6 days after the index shipped, 1 of the 14 posts it links is indexed and 13 have no Googlebot fetch on our server ledger in 7 days. The pre-registered verdict is read on 2026-09-11, bar 7 of 14.

Why does Search Console flip between Discovered and Unknown for the same URL?

We do not know Google's reason; we can only report the pattern. Across seven daily inspections of 14 never-fetched URLs we counted 45 state changes, and every not-indexed URL flipped at least twice, with no fetch behind any of them. We treat the two states as one bucket, not indexed, and read lastCrawlTime for movement.

How do you know Googlebot actually fetched the index page?

Two independent records. Search Console's URL Inspection API reports lastCrawlTime 2026-09-02 14:49:18 UTC; our server-side ledger, which logs every crawler-shaped request at the edge, holds a Googlebot row on /news at 2026-09-02 14:49:18.392 UTC, the same second. The ledger also holds two later rows on 2026-09-03 that the API had not yet reported at the read.

See your number

See which businesses AI names when your client's buyers ask.

Running this for clients? The $249 agency 5-pack audits five businesses, white-labeled.

Check a client's AI visibility

Begin your check

Free · 60 sec

No account · No card · 3 buyer questions, 2 engines

By running a check you agree to our Terms and Privacy Policy.

Who runs this