AI search news ·

How long does a new page stay "Discovered, currently not indexed" before Googlebot fetches it? 11.3 days for one post, then indexed within 14 hours of its first and only fetch, while 12 hub posts with 0 Googlebot fetches stay out

At 2026-09-06 16:46 UTC Search Console reads one of our /news posts as "Submitted and indexed" with lastCrawlTime 2026-09-05T20:05:43Z, and our server ledger holds exactly one Googlebot request on that path since publish, at 2026-09-05 20:05:43 UTC, the same second. The post went live 2026-08-25 at 14:04 UTC, was in the submitted sitemap from that hour, was pinged through IndexNow the same run, and has been linked from the /news index since 2026-08-28. It was never request-indexed. Through 9 daily reads from 08-28 to 09-05 it read "Discovered, currently not indexed" 5 times and "URL is unknown to Google" 4 times, with 0 Googlebot rows in the trailing 7 days on every read. The 13 requests on its path before the fetch came from Yandex, Perplexity, Anthropic, OpenAI, Amazon, Meta, Semrush, Ahrefs, Moz and two unnamed bots; none from Google. Googlebot fetched the sitemap 34 times on 14 days in the same window. The read 3.3 hours before the fetch said "unknown to Google"; the read 13.7 hours after it said indexed. Across the 14 posts the index links, the 2 Googlebot fetched since 08-28 are the 2 it has indexed, and the 12 it has not fetched are 0 for 12 (first-fetch-index-cut-2026-09-06.json, 15 of 15 self-checks).

The page and its ten daily reads

The page is our weekly AI-answer panel post from 2026-08-25 (576 answers across four engines, our own citations counted). Its ship commit landed at 14:04 UTC that day, after the deploy inside the same run, and records "IndexNow 200, sitemap resubmitted, table_schema 174/109"; the 11.3-day figure below counts from that commit. From 2026-08-28 it has been one of the 14 URLs our rebuilt /news index links and the one our daily hub read inspects. The read is mechanical: Search Console's URL Inspection API for the coverage state and lastCrawlTime, and the server ledger for Googlebot rows on the path in the trailing 7 days. It was in the "shipped after the 08-25 sweep" cohort of our request-indexing follow-up, which means the Request indexing button was never used on it.

Read (UTC)URL Inspection statelastCrawlTimeGooglebot rows, 7 dIndex, indexed of 14
2026-08-28 16:38Discovered, not indexednone00 of 14
2026-08-29 16:40Discovered, not indexednone00 of 14
2026-08-30 16:41Discovered, not indexednone01 of 14
2026-08-31 16:48Unknown to Googlenone01 of 14
2026-09-01 16:38Discovered, not indexednone01 of 14
2026-09-02 16:39Unknown to Googlenone01 of 14
2026-09-03 16:38Discovered, not indexednone01 of 14
2026-09-04 16:38Unknown to Googlenone01 of 14
2026-09-05 16:46Unknown to Googlenone01 of 14
2026-09-06 16:38Indexed2026-09-05T20:05:43Z12 of 14

The state alternated between "Discovered, currently not indexed" (5 reads) and "URL is unknown to Google" (4 reads) with no crawl in between; we reported that flicker on 08-26 and count both as not indexed. The last not-indexed read, 09-05 16:46 UTC, was 3.3 hours before the fetch.

Every request on the path since publish: 15 rows, one from Google

Our server records every request whose user agent names a crawler or looks bot-shaped, with path and time. The table is the complete list for this path from 08-24 to the read. Google's row is the 14th; the 13 before it are AI-engine crawlers, SEO-tool crawlers, Meta's crawler and two unnamed Android-shaped agents.

Time (UTC)AgentClassDevice
2026-08-25 13:57YandexBotyandex/searchdesktop
2026-08-25 14:16PerplexityBotperplexity/crawldesktop
2026-08-25 17:03ClaudeBotanthropic/crawldesktop
2026-08-25 18:47OAI-SearchBotopenai/crawldesktop
2026-08-26 06:43PetalBotother/othermobile
2026-08-26 14:33SemrushBotother/otherdesktop
2026-08-26 20:44Amazonbotamazon/crawldesktop
2026-08-28 08:30AhrefsBotother/otherdesktop
2026-08-30 22:50DotBotother/otherdesktop
2026-09-01 03:47PetalBotother/othermobile
2026-09-02 02:08Amazonbotamazon/crawldesktop
2026-09-02 22:19DotBotother/otherdesktop
2026-09-05 19:38meta-externalagentmeta/crawldesktop
2026-09-05 20:05Googlebotgoogle/searchmobile
2026-09-06 09:28PetalBotother/othermobile

The fetch, to the second

Googlebot requested the page at 2026-09-05 20:05:43 UTC with its smartphone user agent (Search Console: crawledAs MOBILE, pageFetchState SUCCESSFUL, robots ALLOWED). Search Console's lastCrawlTime for the page is 2026-09-05T20:05:43Z: the same second as the ledger row, so the fetch that indexed the page is the one row we hold. It came 3.2 hours after Googlebot's 16:51 UTC sitemap fetch that day, and 11.3 days after publish. Between the 09-05 16:46 UTC read ("unknown to Google") and our analyst's 09:50 UTC read on 09-06 ("Submitted and indexed") lie 13.7 hours from the fetch; that is the upper bound on fetch-to-indexed for this page, and the true figure is shorter by however long the state sat unread overnight.

Googlebot on 2026-09-05 (UTC)PathDevice
05:38:30/robots.txtdesktop
05:38:30/news/does-googlebot-still-fetch-new-posts-on-publish-8-posts-zero-goo…mobile
15:48:44/robots.txtdesktop
15:48:44/sitemap.xmldesktop
15:52:33/data/agency-showcase-clients-ai-visibility-40-of-100desktop
16:11:38/compare/ai-visibility-toolsdesktop
16:51:30/sitemap.xmldesktop
16:51:30/robots.txtdesktop
19:49:33/robots.txtdesktop
19:49:33/news/does-googlebot-still-fetch-new-posts-on-publish-8-posts-zero-goo…mobile
20:05:43/news/four-weekly-runs-576-ai-answers-askedabout-cited-in-18-from-4-of…mobile
23:33:36/robots.txtdesktop
23:33:36/news/google-search-generative-ai-opt-out-control-what-it-covers-and-w…mobile

Googlebot made 13 requests to the site that day: robots.txt 5 times, the sitemap 2 times, and 4 /news post fetches on 3 posts. The other 2 posts it fetched were already indexed: does-googlebot-still-fetch-new-pos… (indexed; 2 Googlebot rows that day, Search Console lastCrawlTime still 2026-08-27T23:33:03Z, not moved by them); google-search-generative-ai-opt-ou… (indexed; 1 Googlebot row that day, Search Console lastCrawlTime updated to 2026-09-05T23:33:36Z). Since 2026-08-24 Googlebot has made 130 requests to 40 distinct /news posts, and 34 to the sitemap; the 12 unfetched index posts are in that sitemap.

The 14-URL index: fetched equals indexed

The daily read's denominator is the 14 posts the /news index linked on ship day (08-28). Day 10 of the read, at 09-06 16:38 UTC: 2 of 14 indexed, against a bar of 7 on 2026-09-11. The two columns that matter are beside each other: Googlebot rows on the path since 08-28, and the coverage state. The 2 URLs with a Googlebot row are the 2 indexed URLs. The 12 without one are 0 of 12 indexed, and between them carry 110 requests from other crawlers (5 to 14 each). Google fetched the index page itself 3 times since 08-28 (09-02 14:49 UTC, 09-03 10:26 UTC, 09-03 10:26 UTC) and not since.

Post (slug, truncated)State at readGooglebot rows since 08-28Googlebot timesOther crawler rows since 08-28
can-chatgpt-send-a-reader-to-a-page-before-google-…Unknown to Google0none6
do-new-ai-cited-sources-survive-a-week-84-of-279-e…Discovered, not indexed0none5
what-queries-does-google-show-a-small-ai-search-si…Unknown to Google0none9
how-many-visitors-do-ai-engines-actually-send-a-sm…Indexed208-30 04:29 UTC, 09-01 17:07 UTC6
what-are-iboubot-and-petalbot-a-source-citing-fren…Discovered, not indexed0none14
does-your-json-ld-still-parse-after-googles-single…Unknown to Google0none10
do-page-one-google-rankings-earn-clicks-for-a-smal…Discovered, not indexed0none9
five-external-visitors-in-74-days-reached-our-chec…Discovered, not indexed0none6
four-weekly-runs-576-ai-answers-askedabout-cited-i…Indexed109-05 20:05 UTC7
where-do-ai-crawlers-actually-go-chatgpt-user-71-p…Discovered, not indexed0none7
does-googles-request-indexing-button-actually-work…Discovered, not indexed0none12
what-did-our-pitch-emails-actually-say-96-of-97-op…Discovered, not indexed0none14
fifteen-ai-referrals-in-76-days-one-tap-zero-form-…Discovered, not indexed0none10
did-chatgpt-stop-citing-reddit-on-august-14-prompt…Discovered, not indexed0none8

What the sitemap, IndexNow and internal links did

Search Analytics on the page

Search Console Search Analytics for the page from 2026-08-25 to 2026-09-06: 0 impressions, 0 clicks, 0 queries. The page has been indexed for under a day at the read; whether it earns an impression is the 09-11 question, not today's.

What this means if you care about AI visibility

Limits

Trailing indexation, read the same run

The 9 posts published 2026-09-03 to 09-06 were inspected in the same run: 8 "Discovered, not indexed", 1 "unknown to Google", 0 indexed, 0 with a lastCrawlTime, and 0 Googlebot rows on their paths since publish against 82 rows from other crawlers.

Post (slug, truncated)PublishedHTTPURL Inspection state at readGooglebot rows since publishAll crawler rows since publish
cold-pitch-emails-by-touch-107-emails-to-66-named-people-at-…2026-09-06200Discovered, not indexed02
a-page-with-one-google-impression-in-28-days-took-three-refe…2026-09-05200Discovered, not indexed06
cold-email-opens-by-recipient-mail-provider-0-of-21-google-h…2026-09-05200Discovered, not indexed09
why-did-googles-ai-mode-stop-citing-sources-on-gemini-3-8-fl…2026-09-04200Discovered, not indexed012
six-google-clicks-in-8-landed-days-against-a-bar-of-25-in-28…2026-09-04200Discovered, not indexed08
a-cold-email-restart-at-48-hours-41-sends-to-31-agencies-3-o…2026-09-04200Unknown to Google010
which-gemini-model-powers-ai-mode-in-google-search-now-gemin…2026-09-03200Discovered, not indexed012
googlebot-fetched-our-blog-index-after-58-days-and-again-20-…2026-09-03200Discovered, not indexed010
a-cold-email-restart-at-24-hours-26-sends-to-16-agencies-3-u…2026-09-03200Discovered, not indexed013

The next reading, pre-committed

2026-09-11: the hub read on the same 14 URLs at its pre-registered bar of 7 of 14 indexed; and, for each of the 12 not-indexed URLs, whether a Googlebot row on its path precedes any flip to indexed. Falsified if any URL flips to indexed with 0 Googlebot rows on its path, or if a Googlebot-fetched URL is still not indexed 48 h after the fetch. Published either way.

How long does a new page stay Discovered, currently not indexed before Googlebot fetches it?

For this page, 11.3 days from publish to the first Googlebot request, through 9 daily reads that said "Discovered" or "unknown". The state changed only after the fetch; nothing on the page or in its links changed in the meantime.

Does one Googlebot fetch get a page indexed?

Here, yes: one request at 2026-09-05 20:05:43 UTC, and the page read "Submitted and indexed" within 14 hours with lastCrawlTime equal to that request's second. On the same 14-URL index, all 2 posts Googlebot has fetched since 08-28 are indexed and all 12 it has not fetched are not.

Do sitemaps, IndexNow or internal links make Googlebot fetch a page sooner?

Not on this site in this window. The page was in the sitemap Googlebot fetched 34 times, was pinged through IndexNow (which does not reach Google), and was linked from an index Googlebot visited 3 times without following the link. They told Google the URL exists; the fetch came 11.3 days later on Google's own schedule.

See your number

See which businesses AI names when your client's buyers ask.

Running this for clients? The $249 agency 5-pack audits five businesses, white-labeled.

Check a client's AI visibility

Begin your check

Free · 60 sec

No account · No card · 3 buyer questions, 2 engines

By running a check you agree to our Terms and Privacy Policy.

Who runs this