AI search news ·
How long does a new page stay "Discovered, currently not indexed" before Googlebot fetches it? 11.3 days for one post, then indexed within 14 hours of its first and only fetch, while 12 hub posts with 0 Googlebot fetches stay out
At 2026-09-06 16:46 UTC Search Console reads one of our /news posts as "Submitted and indexed" with lastCrawlTime 2026-09-05T20:05:43Z, and our server ledger holds exactly one Googlebot request on that path since publish, at 2026-09-05 20:05:43 UTC, the same second. The post went live 2026-08-25 at 14:04 UTC, was in the submitted sitemap from that hour, was pinged through IndexNow the same run, and has been linked from the /news index since 2026-08-28. It was never request-indexed. Through 9 daily reads from 08-28 to 09-05 it read "Discovered, currently not indexed" 5 times and "URL is unknown to Google" 4 times, with 0 Googlebot rows in the trailing 7 days on every read. The 13 requests on its path before the fetch came from Yandex, Perplexity, Anthropic, OpenAI, Amazon, Meta, Semrush, Ahrefs, Moz and two unnamed bots; none from Google. Googlebot fetched the sitemap 34 times on 14 days in the same window. The read 3.3 hours before the fetch said "unknown to Google"; the read 13.7 hours after it said indexed. Across the 14 posts the index links, the 2 Googlebot fetched since 08-28 are the 2 it has indexed, and the 12 it has not fetched are 0 for 12 (first-fetch-index-cut-2026-09-06.json, 15 of 15 self-checks).
The page and its ten daily reads
The page is our weekly AI-answer panel post from 2026-08-25 (576 answers across four engines, our own citations counted). Its ship commit landed at 14:04 UTC that day, after the deploy inside the same run, and records "IndexNow 200, sitemap resubmitted, table_schema 174/109"; the 11.3-day figure below counts from that commit. From 2026-08-28 it has been one of the 14 URLs our rebuilt /news index links and the one our daily hub read inspects. The read is mechanical: Search Console's URL Inspection API for the coverage state and lastCrawlTime, and the server ledger for Googlebot rows on the path in the trailing 7 days. It was in the "shipped after the 08-25 sweep" cohort of our request-indexing follow-up, which means the Request indexing button was never used on it.
| Read (UTC) | URL Inspection state | lastCrawlTime | Googlebot rows, 7 d | Index, indexed of 14 |
|---|---|---|---|---|
| 2026-08-28 16:38 | Discovered, not indexed | none | 0 | 0 of 14 |
| 2026-08-29 16:40 | Discovered, not indexed | none | 0 | 0 of 14 |
| 2026-08-30 16:41 | Discovered, not indexed | none | 0 | 1 of 14 |
| 2026-08-31 16:48 | Unknown to Google | none | 0 | 1 of 14 |
| 2026-09-01 16:38 | Discovered, not indexed | none | 0 | 1 of 14 |
| 2026-09-02 16:39 | Unknown to Google | none | 0 | 1 of 14 |
| 2026-09-03 16:38 | Discovered, not indexed | none | 0 | 1 of 14 |
| 2026-09-04 16:38 | Unknown to Google | none | 0 | 1 of 14 |
| 2026-09-05 16:46 | Unknown to Google | none | 0 | 1 of 14 |
| 2026-09-06 16:38 | Indexed | 2026-09-05T20:05:43Z | 1 | 2 of 14 |
The state alternated between "Discovered, currently not indexed" (5 reads) and "URL is unknown to Google" (4 reads) with no crawl in between; we reported that flicker on 08-26 and count both as not indexed. The last not-indexed read, 09-05 16:46 UTC, was 3.3 hours before the fetch.
Every request on the path since publish: 15 rows, one from Google
Our server records every request whose user agent names a crawler or looks bot-shaped, with path and time. The table is the complete list for this path from 08-24 to the read. Google's row is the 14th; the 13 before it are AI-engine crawlers, SEO-tool crawlers, Meta's crawler and two unnamed Android-shaped agents.
| Time (UTC) | Agent | Class | Device |
|---|---|---|---|
| 2026-08-25 13:57 | YandexBot | yandex/search | desktop |
| 2026-08-25 14:16 | PerplexityBot | perplexity/crawl | desktop |
| 2026-08-25 17:03 | ClaudeBot | anthropic/crawl | desktop |
| 2026-08-25 18:47 | OAI-SearchBot | openai/crawl | desktop |
| 2026-08-26 06:43 | PetalBot | other/other | mobile |
| 2026-08-26 14:33 | SemrushBot | other/other | desktop |
| 2026-08-26 20:44 | Amazonbot | amazon/crawl | desktop |
| 2026-08-28 08:30 | AhrefsBot | other/other | desktop |
| 2026-08-30 22:50 | DotBot | other/other | desktop |
| 2026-09-01 03:47 | PetalBot | other/other | mobile |
| 2026-09-02 02:08 | Amazonbot | amazon/crawl | desktop |
| 2026-09-02 22:19 | DotBot | other/other | desktop |
| 2026-09-05 19:38 | meta-externalagent | meta/crawl | desktop |
| 2026-09-05 20:05 | Googlebot | google/search | mobile |
| 2026-09-06 09:28 | PetalBot | other/other | mobile |
The fetch, to the second
Googlebot requested the page at 2026-09-05 20:05:43 UTC with its smartphone user agent (Search Console: crawledAs MOBILE, pageFetchState SUCCESSFUL, robots ALLOWED). Search Console's lastCrawlTime for the page is 2026-09-05T20:05:43Z: the same second as the ledger row, so the fetch that indexed the page is the one row we hold. It came 3.2 hours after Googlebot's 16:51 UTC sitemap fetch that day, and 11.3 days after publish. Between the 09-05 16:46 UTC read ("unknown to Google") and our analyst's 09:50 UTC read on 09-06 ("Submitted and indexed") lie 13.7 hours from the fetch; that is the upper bound on fetch-to-indexed for this page, and the true figure is shorter by however long the state sat unread overnight.
| Googlebot on 2026-09-05 (UTC) | Path | Device |
|---|---|---|
| 05:38:30 | /robots.txt | desktop |
| 05:38:30 | /news/does-googlebot-still-fetch-new-posts-on-publish-8-posts-zero-goo… | mobile |
| 15:48:44 | /robots.txt | desktop |
| 15:48:44 | /sitemap.xml | desktop |
| 15:52:33 | /data/agency-showcase-clients-ai-visibility-40-of-100 | desktop |
| 16:11:38 | /compare/ai-visibility-tools | desktop |
| 16:51:30 | /sitemap.xml | desktop |
| 16:51:30 | /robots.txt | desktop |
| 19:49:33 | /robots.txt | desktop |
| 19:49:33 | /news/does-googlebot-still-fetch-new-posts-on-publish-8-posts-zero-goo… | mobile |
| 20:05:43 | /news/four-weekly-runs-576-ai-answers-askedabout-cited-in-18-from-4-of… | mobile |
| 23:33:36 | /robots.txt | desktop |
| 23:33:36 | /news/google-search-generative-ai-opt-out-control-what-it-covers-and-w… | mobile |
Googlebot made 13 requests to the site that day: robots.txt 5 times, the sitemap 2 times, and 4 /news post fetches on 3 posts. The other 2 posts it fetched were already indexed: does-googlebot-still-fetch-new-pos… (indexed; 2 Googlebot rows that day, Search Console lastCrawlTime still 2026-08-27T23:33:03Z, not moved by them); google-search-generative-ai-opt-ou… (indexed; 1 Googlebot row that day, Search Console lastCrawlTime updated to 2026-09-05T23:33:36Z). Since 2026-08-24 Googlebot has made 130 requests to 40 distinct /news posts, and 34 to the sitemap; the 12 unfetched index posts are in that sitemap.
The 14-URL index: fetched equals indexed
The daily read's denominator is the 14 posts the /news index linked on ship day (08-28). Day 10 of the read, at 09-06 16:38 UTC: 2 of 14 indexed, against a bar of 7 on 2026-09-11. The two columns that matter are beside each other: Googlebot rows on the path since 08-28, and the coverage state. The 2 URLs with a Googlebot row are the 2 indexed URLs. The 12 without one are 0 of 12 indexed, and between them carry 110 requests from other crawlers (5 to 14 each). Google fetched the index page itself 3 times since 08-28 (09-02 14:49 UTC, 09-03 10:26 UTC, 09-03 10:26 UTC) and not since.
| Post (slug, truncated) | State at read | Googlebot rows since 08-28 | Googlebot times | Other crawler rows since 08-28 |
|---|---|---|---|---|
| can-chatgpt-send-a-reader-to-a-page-before-google-… | Unknown to Google | 0 | none | 6 |
| do-new-ai-cited-sources-survive-a-week-84-of-279-e… | Discovered, not indexed | 0 | none | 5 |
| what-queries-does-google-show-a-small-ai-search-si… | Unknown to Google | 0 | none | 9 |
| how-many-visitors-do-ai-engines-actually-send-a-sm… | Indexed | 2 | 08-30 04:29 UTC, 09-01 17:07 UTC | 6 |
| what-are-iboubot-and-petalbot-a-source-citing-fren… | Discovered, not indexed | 0 | none | 14 |
| does-your-json-ld-still-parse-after-googles-single… | Unknown to Google | 0 | none | 10 |
| do-page-one-google-rankings-earn-clicks-for-a-smal… | Discovered, not indexed | 0 | none | 9 |
| five-external-visitors-in-74-days-reached-our-chec… | Discovered, not indexed | 0 | none | 6 |
| four-weekly-runs-576-ai-answers-askedabout-cited-i… | Indexed | 1 | 09-05 20:05 UTC | 7 |
| where-do-ai-crawlers-actually-go-chatgpt-user-71-p… | Discovered, not indexed | 0 | none | 7 |
| does-googles-request-indexing-button-actually-work… | Discovered, not indexed | 0 | none | 12 |
| what-did-our-pitch-emails-actually-say-96-of-97-op… | Discovered, not indexed | 0 | none | 14 |
| fifteen-ai-referrals-in-76-days-one-tap-zero-form-… | Discovered, not indexed | 0 | none | 10 |
| did-chatgpt-stop-citing-reddit-on-august-14-prompt… | Discovered, not indexed | 0 | none | 8 |
What the sitemap, IndexNow and internal links did
- Sitemap: the page was in the submitted sitemap from its ship hour and Search Console lists that sitemap on the page today. Googlebot fetched the sitemap 34 times on 14 days between 2026-08-24 and 2026-09-06. The sitemap put the URL in front of Google every day; it did not produce a fetch for 11.3 days.
- IndexNow: pinged on ship day (the ship commit's "IndexNow 200"). IndexNow reaches Bing, Yandex and Seznam, not Google. YandexBot's request at 13:57 UTC on 2026-08-25, after the deploy and 7 minutes before the ship commit landed, is the effect it has; it has none on Googlebot.
- Internal links: linked from the /news index since 08-28 and from rel=next / rel=prev on its neighbours. Google fetched the index 3 times and followed 0 of its 14 links on those visits (the 2 fetched posts were fetched on other days, from the sitemap or from Google's own queue). The 12 unfetched posts have had 10 days of index links.
- Nothing we could name triggered the fetch. No request-indexing, no new link, no edit. The fetch sits 3.2 hours after a routine sitemap fetch, on a day Googlebot fetched two already-indexed posts as well.
Search Analytics on the page
Search Console Search Analytics for the page from 2026-08-25 to 2026-09-06: 0 impressions, 0 clicks, 0 queries. The page has been indexed for under a day at the read; whether it earns an impression is the 09-11 question, not today's.
What this means if you care about AI visibility
- "Discovered, currently not indexed" on a small site is a fetch queue, not a quality verdict. This page was indexed within 14 hours of its first fetch with no change to the page; the 11.3 days were spent waiting for Googlebot, not for a decision. If your posts show that state, the number to watch is Googlebot rows on the path, and your server log is the only place it appears in real time.
- Being in the sitemap, pinging IndexNow and linking from an index page put the URL in front of Google without producing a fetch for 11.3 days. On this index, Googlebot's 3 visits since 08-28 followed none of its 14 links. Internal links help Google know the URL exists; on a low-crawl-budget site they have not, so far, moved it up the queue.
- AI-engine crawlers do not wait. Before Google's one request, this path had 13 others, including PerplexityBot, ClaudeBot, OAI-SearchBot and Amazonbot inside the first two days. A page can be in four AI engines' fetch logs for eleven days while Google has never seen its HTML. If you want to know which engines have your page, check your business rather than Search Console.
Limits
- One page flipped. The 14-URL index gives a 2-for-2 fetched-then-indexed record and a 0-for-12 unfetched record; that is a pattern on 14 URLs of one site, not a rate.
- Fetch-to-indexed is bounded above at 13.7 hours by our read cadence; the page was not inspected between 16:46 UTC on 09-05 and 09:50 UTC on 09-06.
- User agents are joined from a site-wide pull capped at 5,000 rows (floor 2026-08-24 04:57 UTC, newest 2026-09-06 16:45 UTC); the path's own count comes from the uncapped per-path index and both agree at 15. Googlebot rows are user-agent matches; the ledger does not reverse-DNS them, and the one row here is neither our probe nor a known spoof pattern.
- Search Console's coverage state is read through the API once a day; "Discovered" and "unknown" alternated with no crawl between them, so the daily state is treated as not-indexed either way.
- The sitemap-fetch-to-page-fetch gap (3.2 hours) is a coincidence in time until it repeats; the sitemap had been fetched 32 times before that day without a page fetch following.
Trailing indexation, read the same run
The 9 posts published 2026-09-03 to 09-06 were inspected in the same run: 8 "Discovered, not indexed", 1 "unknown to Google", 0 indexed, 0 with a lastCrawlTime, and 0 Googlebot rows on their paths since publish against 82 rows from other crawlers.
| Post (slug, truncated) | Published | HTTP | URL Inspection state at read | Googlebot rows since publish | All crawler rows since publish |
|---|---|---|---|---|---|
| cold-pitch-emails-by-touch-107-emails-to-66-named-people-at-… | 2026-09-06 | 200 | Discovered, not indexed | 0 | 2 |
| a-page-with-one-google-impression-in-28-days-took-three-refe… | 2026-09-05 | 200 | Discovered, not indexed | 0 | 6 |
| cold-email-opens-by-recipient-mail-provider-0-of-21-google-h… | 2026-09-05 | 200 | Discovered, not indexed | 0 | 9 |
| why-did-googles-ai-mode-stop-citing-sources-on-gemini-3-8-fl… | 2026-09-04 | 200 | Discovered, not indexed | 0 | 12 |
| six-google-clicks-in-8-landed-days-against-a-bar-of-25-in-28… | 2026-09-04 | 200 | Discovered, not indexed | 0 | 8 |
| a-cold-email-restart-at-48-hours-41-sends-to-31-agencies-3-o… | 2026-09-04 | 200 | Unknown to Google | 0 | 10 |
| which-gemini-model-powers-ai-mode-in-google-search-now-gemin… | 2026-09-03 | 200 | Discovered, not indexed | 0 | 12 |
| googlebot-fetched-our-blog-index-after-58-days-and-again-20-… | 2026-09-03 | 200 | Discovered, not indexed | 0 | 10 |
| a-cold-email-restart-at-24-hours-26-sends-to-16-agencies-3-u… | 2026-09-03 | 200 | Discovered, not indexed | 0 | 13 |
The next reading, pre-committed
2026-09-11: the hub read on the same 14 URLs at its pre-registered bar of 7 of 14 indexed; and, for each of the 12 not-indexed URLs, whether a Googlebot row on its path precedes any flip to indexed. Falsified if any URL flips to indexed with 0 Googlebot rows on its path, or if a Googlebot-fetched URL is still not indexed 48 h after the fetch. Published either way.
How long does a new page stay Discovered, currently not indexed before Googlebot fetches it?
For this page, 11.3 days from publish to the first Googlebot request, through 9 daily reads that said "Discovered" or "unknown". The state changed only after the fetch; nothing on the page or in its links changed in the meantime.
Does one Googlebot fetch get a page indexed?
Here, yes: one request at 2026-09-05 20:05:43 UTC, and the page read "Submitted and indexed" within 14 hours with lastCrawlTime equal to that request's second. On the same 14-URL index, all 2 posts Googlebot has fetched since 08-28 are indexed and all 12 it has not fetched are not.
Do sitemaps, IndexNow or internal links make Googlebot fetch a page sooner?
Not on this site in this window. The page was in the sitemap Googlebot fetched 34 times, was pinged through IndexNow (which does not reach Google), and was linked from an index Googlebot visited 3 times without following the link. They told Google the URL exists; the fetch came 11.3 days later on Google's own schedule.
See your number
See which businesses AI names when your client's buyers ask.
Running this for clients? The $249 agency 5-pack audits five businesses, white-labeled.
Who runs this
- Built and operated by Sensara LLC, Atlanta, Georgia — about us and how the audit works.
- See what the report looks like before you run anything — score per engine, the competitors AI names instead of you, and a fix plan.
- We run the same audit on ourselves every week and publish the result: in the latest run AI named AskedAbout in 1 of 144 answers. We report our own numbers the way we report yours.