AI search news ·

Google no longer requires you to block your internal search pages. We checked whether AI engines cite them: zero of 2,269

In the Search Off the Record episode Google published on 2026-07-30, John Mueller confirms that the 2007-era webmaster-guideline item telling you to block internal search results pages is no longer in Search Essentials — it is advice now, not a rule, and letting those pages be indexed is “not necessarily something where we would say we think your site is spam.” That reopens a question worth measuring rather than guessing: if you do block `/search`, are you hiding something the AI engines would have cited? We ran the check against our own production corpus — 1,048 completed answers, 177 audits, 4 local verticals, 3,252 cited URLs across 1,140 domains. Exactly zero of them is an internal search results page.

What Google actually said

The primary source is Google's own channel: “Should you block your Search result pages?”, a 28-minute Search Off the Record episode with Martin Splitt and John Mueller, published 2026-07-30. The relevant passage begins around 17:10. Quotes below are taken from the episode's own transcript, not from a secondary write-up.

Mueller: “Way back in the early days, in our webmaster guidelines when they existed as webmaster guidelines, I think we also listed search results pages as something that you have to block from Google… Nowadays, we don't have that listed in the search policies, but it's still something that I think just purely for technical reasons makes sense. So it's not necessarily something where we would say we think your site is spam if you allow your search results pages to be indexed. It's just like — you're being very inefficient.” Splitt confirms the reframing directly: “So we are not seeing this as a quality indication or a quality issue.”

Two caveats stay firmly in place, and both are practical rather than punitive. Crawl waste: an unbounded search path is an infinite space, and Googlebot spending its budget there is budget it does not spend on your real pages. Hacked-content risk: Mueller's example is a site that lets visitors search for terms it has nothing to do with, renders those terms into an indexable page, and ends up ranking for them — “and when we see that happen, we might flag that as hacked.” That is the failure mode worth caring about, and it is unchanged by the guideline being dropped.

One implementation detail from the episode is more useful than the headline. Asked how to write the rule, Mueller recommends one broad rule, matched on the parameter rather than the word: block `/search` followed by `?`, so you catch `/search?q=…` without also catching a legitimate page like `search-for-cheese.php`. And he is explicit that an indexable search page is the second-best version of a better idea: “probably you could do a better job if you just created a real category page… you could provide more context — why is this term important, and which of these pages are actually the best on your website for this term?”

The question Google's episode does not answer

Search Off the Record is about Googlebot. It says nothing about whether blocking `/search` costs you anything with ChatGPT or Perplexity — and that is now the live worry, because five days ago we published Previsible's finding that 28.8% of ChatGPT's referral traffic lands on an internal search page. Read quickly, those two facts look like a conflict: Google says block them, and ChatGPT appears to be sending a third of its clicks to them.

So we measured the other side of it. Previsible measured where ChatGPT's clicks landed, in GA4, across 166 mostly-SaaS properties. We measured what the engines actually cite — the URLs attached to a recommendation answer — in our production corpus of local-business audits. Same phenomenon, opposite end of the pipe.

The result: 0 of 2,269

MeasureValue
Completed answers in the cut (2026-07-31)1,048 across 177 audits, 4 verticals
Answers carrying ≥1 URL546
Cited URLs (answer × distinct URL)3,252
Distinct URLs2,269
Distinct domains1,140
Internal search results pages0
URLs carrying any functional parameter36 of 3,252 (1.11%)

Not one of the 2,269 distinct URLs the engines attached to a local-business recommendation is a search results page. The classifier is deliberately narrow and purely syntactic — it never fetches the page. A URL counts as search-shaped only when a path segment is exactly `search` (so `/search-engine-optimization` does not match) or a query parameter is one of `q`, `s`, `query`, `search`, `k`, `find_desc` and friends with a non-empty value. The code is `ops/internal_search_urls.py` in our repo.

A zero is only worth reading if the instrument can produce a non-zero, so this one ships with a positive control that must pass before it will print a number. Twelve fixed URLs: six that must classify as search pages (`yelp.com/search?find_desc=…`, `/?s=hvac+repair`, `amazon.com/s?k=…`, `/search/`, and two more) and six near-misses that must not (`/search-engine-optimization/`, `/researching-a-dentist`, `?q=` with an empty value, a real cited directory URL). All twelve pass. The run refuses to report anything if they don't.

The first version of this script was wrong, and the way it was wrong is worth stating. It read only the URLs written inline in the answer text and reported Perplexity as citing 13 URLs across 525 answers — a nonsense number, because Perplexity returns its sources out of band, in a separate `citations` array. Reading both channels recovers 153 Perplexity URLs. The zero survived the fix; the Perplexity denominator did not survive it as anything solid, and we are saying so: only 12 rows in this cut carry that array, so this is overwhelmingly a finding about ChatGPT (3,086 of the 3,252 cited URLs are its).

What the engines cite instead — and why it agrees with Mueller

The engines are not avoiding search results. They are avoiding search URLs. What they cite constantly is the pre-rendered equivalent — the same list of businesses a search box would have produced, published on a static, crawlable path:

That is Mueller's category-page advice, arrived at from the other direction. He argues a real category page beats an indexable search page because it can carry context about why the term matters and which pages are best. The engines behave as though they agree: the top-cited domains in our corpus are directories that publish the static version — birdeye.com (142 answers), bbb.org (92), bestprosintown.com (82), medspascout.com (63), threebestrated.com (48). Every one of those is a listing page on a clean path.

The 1.11% figure sharpens it. Of 3,252 cited URLs, only 36 carry a functional parameter of any kind, and none of those is a search query — they are pagination (`?page=`, `?reviewsPage=`), sorting (`?sort=`), tabs (`?tab=`) and filters (`?fcountry=`). The engines cite clean canonical URLs almost without exception.

A side finding you can verify on your own server today

3,085 of the 3,086 URLs ChatGPT cited in this corpus carry `?utm_source=openai` — a tag OpenAI appends itself. We confirmed our own pipeline does not add it: the answer text is stored verbatim from the API, and `utm` appears nowhere in our engine code. The practical consequence is that ChatGPT-referred visits are identifiable in raw server logs with a `grep`, with no analytics product, no tag manager and no AI-visibility tool in the path. If you want to know whether ChatGPT is sending you anyone, that is the cheapest possible check.

Three implications

What this does not say

Do I still need to block internal search results pages from Google?

You are not required to. In the Search Off the Record episode published 2026-07-30, John Mueller confirmed the old webmaster-guideline item is no longer in Search Essentials and that indexable search pages are not treated as a spam signal. He still recommends blocking them, for two practical reasons: crawl efficiency, and the risk that irrelevant terms visitors search for get rendered into indexable pages and cause your site to be flagged as hacked.

Does blocking /search hurt my visibility in ChatGPT or Perplexity?

We found no evidence of it. Across 1,048 AI answers about local businesses containing 3,252 cited URLs on 1,140 domains, zero cited URLs were internal search results pages, and only 1.11% carried any functional URL parameter at all. The engines in this corpus cite clean static paths.

Why does ChatGPT link people to my search page if it never cites one?

Those are different events. A cited URL is a page the engine retrieved and attached as a source. A search-page link in your GA4 report is frequently a URL the model constructed because it resolved your domain but not which page answers the question. Previsible measured the second at 28.8% of ChatGPT referral sessions; we measured the first at 0%. Both can be true at once.

How do I write the robots.txt rule Google recommends?

Mueller's advice in the episode is one broad rule matched on the parameter, not the word: block the /search path followed by a question mark, so /search?q=… is covered while a real page such as search-for-cheese.php is not. One rule is easier to maintain and easier to reason about than a labyrinth of nuanced patterns.

How can I tell if ChatGPT is sending me traffic without any tools?

Grep your server logs for utm_source=openai. In our corpus 3,085 of 3,086 URLs ChatGPT cited carried that parameter, appended by OpenAI rather than by the publisher, so ChatGPT-referred visits are identifiable directly from raw request logs.

The takeaway

Google relaxed a rule and kept the advice. On the AI side the advice costs you nothing measurable: the engines were never citing your search URLs. What they cite is the page you would have to write anyway — a real, static, specific page about one thing, on a path a crawler can reach. That is the same answer the citation data has given every time we have asked it a different way.

The free 60-second check asks the engines your buyers' actual questions and shows you what they say and which URLs they cite. The $79 audit runs 25 questions across ChatGPT, Perplexity, Gemini and Claude, three times each, and returns the source map — the exact pages and domains the engines pulled from, which is the target list you execute against. Agencies auditing a book of clients can run five at once with the $249 Agency 5-pack.

See your number

A free 60-second check shows what AI says about you.

Running this for clients? The $249 agency 5-pack audits five businesses, white-labeled.

Run a free AI visibility check

Begin your check

Free · 60 sec

No account · No card · 3 buyer questions, 2 engines

By running a check you agree to our Terms and Privacy Policy.

Who runs this