AI search news ·
Googlebot fetched our blog index after 58 days and again 19.6 hours later: 1 of the 14 posts it links is indexed 6 days on, 13 have 0 Googlebot fetches in 7 days, and PerplexityBot fetched 8 of the 14
On 2026-09-02 14:49:18 UTC, Googlebot fetched our /news index page for the first time since 2026-07-06, a gap of 58 days, and fetched it twice more at 10:26:22 and 10:26:23 UTC on 2026-09-03, 19.6 hours after the first. The index had been rebuilt on 2026-08-28 as the internal-linking variable for 14 posts that Google had discovered and not crawled. 6 days after that ship, and one day after Googlebot read the index, 1 of the 14 posts is indexed. 12 read "Discovered, currently not indexed", 1 reads "URL is unknown to Google", and the 13 not-indexed posts have 0 Googlebot rows on our server ledger in 7 days. In the same 7 days Googlebot made 280 requests to the site; PerplexityBot fetched 8 of the 14 posts and Amazonbot 13. Two independent channels agree on the index fetch to the second. Method and every number below come from the committed artifact (hub-read-series-cut-2026-09-03.json, 19 of 19 self-checks); the 2026-09-11 read and its rule were written down on ship day.
What was shipped, and what was written down before it shipped
By 2026-08-27, 14 of our blog posts, published 2026-08-19 to 2026-08-27, had sat in Search Console as "Discovered, currently not indexed" or "URL is unknown to Google" with no Googlebot fetch on our ledger. Our earlier test of the Request indexing button is in the 2026-08-28 post; the button is not a lever we plan on. On 2026-08-28 we shipped one variable instead: a rebuilt /news index grouped by month, with rel="next" and rel="prev" links on every post, so that each of the 14 is reachable from a page Google already indexes. The 14 URLs were pinned in a fixture that day. The read was pre-registered for 2026-09-11 with one rule: at least 7 of 14 indexed means the index works; 0 of 14 means internal linking is not the constraint; anything between is inconclusive and is reported as such. One instrument makes the read every day: it inspects each URL through the Search Console API and counts Googlebot rows on our server ledger for the same path over the trailing 7 days.
The index page itself: two channels, one second
The index page had not been crawled by Google since 2026-07-06 18:12:19 UTC (the lastCrawlTime in our 2026-08-23 sweep of every URL) and had 0 Googlebot rows on the ledger from the ledger's birth on 2026-08-17 to 2026-09-01, in 22 rows from all crawlers. Then:
| Channel | Reading at 2026-09-03 16:38 UTC |
|---|---|
| Search Console URL Inspection, lastCrawlTime | 2026-09-02 14:49:18 UTC ("Submitted and indexed") |
| Previous lastCrawlTime (2026-08-23 sweep) | 2026-07-06 18:12:19 UTC |
| Gap between the two Google crawls | 58 days |
| Server ledger, Googlebot rows on /news, trailing 7 d | 3: 2026-09-02 14:49:18.392 UTC · 2026-09-03 10:26:22.969 UTC · 2026-09-03 10:26:23.018 UTC |
| Ledger row matching lastCrawlTime to the second | yes |
| Googlebot rows on the ledger newer than lastCrawlTime | 2 (the two 2026-09-03 rows; Search Console had not updated by the read) |
| Hours between the first fetch and the second | 19.6 |
| Googlebot rows on /news, ledger birth to 2026-09-01 | 0 |
| All crawler rows on /news, trailing 7 d | 21 |
The two 2026-09-03 rows are 49 ms apart, the shape of a desktop and a smartphone Googlebot pair. Search Console still reported the 2026-09-02 crawl at the read; the API's lastCrawlTime lags the fetch itself, which the ledger sees at once. Rows on /news by crawler in the same 7 days:
| Vendor/kind | Rows on /news, 7 d |
|---|---|
| other/other | 5 |
| openai/user | 3 |
| google/search | 3 |
| google-notebooklm/user | 2 |
| openai/crawl | 2 |
| perplexity/crawl | 2 |
| microsoft/search | 1 |
| apple/crawl | 1 |
| amazon/crawl | 1 |
| you/crawl | 1 |
The 14 linked posts, seven daily reads
Each row is one run of the instrument at about 16:40 UTC. "Flips" counts URLs whose Search Console state differed from the previous day's read. The one URL that went in did so on read 3 (2026-08-30); its lastCrawlTime is 2026-08-30 04:29:48 UTC and it carries the only Googlebot rows on the denominator.
| Read | Date (UTC) | Days after ship | Indexed | Discovered | Unknown | Flips vs previous read | Googlebot rows on the 14, trailing 7 d |
|---|---|---|---|---|---|---|---|
| 1 | 2026-08-28 | 0 | 0 | 7 | 7 | first read | 0 |
| 2 | 2026-08-29 | 1 | 0 | 13 | 1 | 8 | 0 |
| 3 | 2026-08-30 | 2 | 1 | 9 | 4 | 6 | 1 |
| 4 | 2026-08-31 | 3 | 1 | 10 | 3 | 7 | 1 |
| 5 | 2026-09-01 | 4 | 1 | 4 | 9 | 8 | 1 |
| 6 | 2026-09-02 | 5 | 1 | 5 | 8 | 7 | 2 |
| 7 | 2026-09-03 | 6 | 1 | 12 | 1 | 9 | 2 |
45 state changes across 6 transitions on 14 URLs, and 1 of them is a change in indexation. The rest is the API alternating between "Discovered" and "unknown" for the same never-fetched URL; every not-indexed URL has flipped at least twice. The per-URL view, with the state on each read as a letter (D discovered, U unknown, I indexed):
| Post (slug, truncated) | Published | Age at read (d) | State on read 7 | Reads 1 to 7 | Flips | Googlebot rows, 7 d |
|---|---|---|---|---|---|---|
| can-chatgpt-send-a-reader-to-a-page-before-googl… | 2026-08-24 | 11 | Discovered, not indexed | DDDDUUD | 2 | 0 |
| do-new-ai-cited-sources-survive-a-week-84-of-279… | 2026-08-24 | 11 | Discovered, not indexed | UDUDUUD | 5 | 0 |
| what-queries-does-google-show-a-small-ai-search-… | 2026-08-23 | 12 | Discovered, not indexed | DUDUUDD | 4 | 0 |
| how-many-visitors-do-ai-engines-actually-send-a-… | 2026-08-21 | 14 | Indexed | DDIIIII | 1 | 2 |
| what-are-iboubot-and-petalbot-a-source-citing-fr… | 2026-08-20 | 15 | Discovered, not indexed | UDDDUDD | 3 | 0 |
| does-your-json-ld-still-parse-after-googles-sing… | 2026-08-23 | 12 | Discovered, not indexed | UDDDDUD | 3 | 0 |
| do-page-one-google-rankings-earn-clicks-for-a-sm… | 2026-08-22 | 13 | Discovered, not indexed | DDUDDDD | 2 | 0 |
| five-external-visitors-in-74-days-reached-our-ch… | 2026-08-25 | 10 | Discovered, not indexed | UDDDUUD | 3 | 0 |
| four-weekly-runs-576-ai-answers-askedabout-cited… | 2026-08-25 | 10 | Discovered, not indexed | DDDUDUD | 4 | 0 |
| where-do-ai-crawlers-actually-go-chatgpt-user-71… | 2026-08-26 | 9 | Discovered, not indexed | DDUDUUD | 4 | 0 |
| does-googles-request-indexing-button-actually-wo… | 2026-08-26 | 9 | Discovered, not indexed | UDUDDUD | 5 | 0 |
| what-did-our-pitch-emails-actually-say-96-of-97-… | 2026-08-27 | 8 | Discovered, not indexed | DDDUUUD | 2 | 0 |
| fifteen-ai-referrals-in-76-days-one-tap-zero-for… | 2026-08-27 | 8 | Unknown to Google | UDDDUDU | 4 | 0 |
| did-chatgpt-stop-citing-reddit-on-august-14-prom… | 2026-08-19 | 16 | Discovered, not indexed | UDDDUDD | 3 | 0 |
Where Googlebot went instead
Googlebot is not absent from the site. In the 24 hours to the read it made 38 requests to 18 distinct paths, 2 of them the index page and 0 on the 14 posts the index links. In 7 days: 280 requests to 60 paths, 2 on the denominator (both on the one indexed post), 0 on the five posts published since 2026-09-01. The 92 requests to /news posts covered 36 distinct posts, published 2026-07-09 to 2026-08-22; the newest of them is 12 days older than the newest post on the site (published 2026-09-03).
| Path class | Googlebot rows, 24 h | Googlebot rows, 7 d |
|---|---|---|
| robots.txt | 13 | 96 |
| sitemap.xml | 2 | 17 |
| /news posts | 9 | 92 |
| /news hub | 2 | 3 |
| /compare | 5 | 24 |
| /guide | 2 | 22 |
| /data | 2 | 10 |
| /for | 2 | 8 |
| home | 1 | 8 |
| UTC day | Googlebot rows |
|---|---|
| 2026-08-27 | 23 |
| 2026-08-28 | 50 |
| 2026-08-29 | 37 |
| 2026-08-30 | 21 |
| 2026-08-31 | 35 |
| 2026-09-01 | 50 |
| 2026-09-02 | 36 |
| 2026-09-03 | 28 |
Who did fetch the 14 posts in the same 7 days
| Crawler | Posts fetched, of 14 | Fetches on those posts |
|---|---|---|
| Amazonbot | 13 | 15 |
| PerplexityBot | 8 | 9 |
| OAI-SearchBot | 3 | 3 |
| bingbot | 2 | 8 |
| YandexBot | 2 | 2 |
| Googlebot | 1 | 2 |
| ClaudeBot | 1 | 1 |
| GPTBot | 0 | 0 |
8 of the 14 were fetched by at least one AI-engine crawler (PerplexityBot 8, OAI-SearchBot 3, ClaudeBot 1, GPTBot 0). These are re-fetches: our 2026-09-01 read found ClaudeBot, OAI-SearchBot and PerplexityBot had each fetched all 17 of the posts published 2026-08-24 to 08-31 within days of publish. The Google side of the same posts has not moved: 13 of 14 with 0 Googlebot rows in 7 days.
The five newest posts, for the record
Outside the pre-registered denominator, the posts published 2026-09-01 to 09-03 were inspected in the same run: 4 read "Discovered, currently not indexed", 1 reads "URL is unknown to Google", and all five have 0 Googlebot rows in 7 days. They will be their own series; they are not added to the 14.
| Post (slug, truncated) | Published | State at read | Googlebot rows, 7 d |
|---|---|---|---|
| page-one-ctr-jumped-to-1-37-percent-because-one-ur… | 2026-09-01 | Discovered, not indexed | 0 |
| claudebot-oai-searchbot-perplexitybot-and-yandexbo… | 2026-09-01 | Discovered, not indexed | 0 |
| which-ai-engines-send-visitors-to-a-small-site-and… | 2026-09-02 | Discovered, not indexed | 0 |
| three-of-three-pre-registered-crawler-readings-hel… | 2026-09-02 | Unknown to Google | 0 |
| a-cold-email-restart-at-24-hours-26-sends-to-16-ag… | 2026-09-03 | Discovered, not indexed | 0 |
What this means if you care about AI visibility
- An index page being crawled and a linked post being crawled are different events. Googlebot read our index three times in 20 hours and, by the read, had followed none of the 13 links to posts it had never fetched. If a client's blog is "Discovered, not indexed", a crawl of the blog index does not by itself change that; check the posts, not the index.
- Search Console's coverage state is not a daily signal. The same never-fetched URL read "Discovered" and "unknown" on alternate days, 45 flips in six days across 14 URLs. Count indexed against not indexed, and read lastCrawlTime; anything finer is noise.
- The AI engines' crawlers do not wait for Google. PerplexityBot re-fetched 8 of these 14 posts in the same week that Googlebot fetched 1. Whether an engine then names your business when a buyer asks is the count our free 60-second check reads per engine; being crawled is where it starts, not where it ends.
Limits
- One site, 14 URLs, one week after one change. The read is a case, not a rate.
- The ledger records requests whose user agent declares a crawler; it does not verify the address. Googlebot rows on it are self-declared. The one place it is checked against Google's own record, the index page's crawl, the two agree to the second.
- URL Inspection has a daily quota, so the instrument reads the 14 URLs plus 11 control URLs once a day at about the same hour; a fetch and an indexation between two reads show up on the next one, not at the minute.
- Search Console's lastCrawlTime lagged the ledger by at least the 6 hours between the 2026-09-03 fetches and the read; the ledger sees the request, the API reports Google's record of it later.
- Age is counted from the publish date in our registry, not from Google's first discovery, which we cannot see.
The next reading, pre-committed
On 2026-09-11 the same instrument runs on the same 14 URLs and the verdict is scored by the rule written on 2026-08-28: at least 7 indexed, the index works; 0, internal linking is not the constraint; between, inconclusive. It is published either way. Two lines are falsifiable before then: if Googlebot rows appear on any of the 13 zero-row posts within 7 days of the 2026-09-03 index fetches, they are named in the 09-11 post with their timestamps; if none do, the sentence "Googlebot read the index three times and followed none of its links to unfetched posts" is restated at 14 days.
Does Googlebot crawling a blog index page get the linked posts indexed?
Not on its own, in our case. Googlebot fetched our rebuilt /news index on 2026-09-02 and twice on 2026-09-03; 6 days after the index shipped, 1 of the 14 posts it links is indexed and 13 have no Googlebot fetch on our server ledger in 7 days. The pre-registered verdict is read on 2026-09-11, bar 7 of 14.
Why does Search Console flip between Discovered and Unknown for the same URL?
We do not know Google's reason; we can only report the pattern. Across seven daily inspections of 14 never-fetched URLs we counted 45 state changes, and every not-indexed URL flipped at least twice, with no fetch behind any of them. We treat the two states as one bucket, not indexed, and read lastCrawlTime for movement.
How do you know Googlebot actually fetched the index page?
Two independent records. Search Console's URL Inspection API reports lastCrawlTime 2026-09-02 14:49:18 UTC; our server-side ledger, which logs every crawler-shaped request at the edge, holds a Googlebot row on /news at 2026-09-02 14:49:18.392 UTC, the same second. The ledger also holds two later rows on 2026-09-03 that the API had not yet reported at the read.
See your number
See which businesses AI names when your client's buyers ask.
Running this for clients? The $249 agency 5-pack audits five businesses, white-labeled.
Who runs this
- Built and operated by Sensara LLC, Atlanta, Georgia — about us and how the audit works.
- See what the report looks like before you run anything — score per engine, the competitors AI names instead of you, and a fix plan.
- We run the same audit on ourselves every week and publish the result: in the latest run AI named AskedAbout in 1 of 144 answers. We report our own numbers the way we report yours.