AI search news ·
Google no longer requires you to block your internal search pages. We checked whether AI engines cite them: zero of 2,269
In the Search Off the Record episode Google published on 2026-07-30, John Mueller confirms that the 2007-era webmaster-guideline item telling you to block internal search results pages is no longer in Search Essentials — it is advice now, not a rule, and letting those pages be indexed is “not necessarily something where we would say we think your site is spam.” That reopens a question worth measuring rather than guessing: if you do block `/search`, are you hiding something the AI engines would have cited? We ran the check against our own production corpus — 1,048 completed answers, 177 audits, 4 local verticals, 3,252 cited URLs across 1,140 domains. Exactly zero of them is an internal search results page.
Primary source: Google Search Central — “Should you block your Search result pages?” (Search Off the Record, 2026-07-30)
What Google actually said
The primary source is Google's own channel: “Should you block your Search result pages?”, a 28-minute Search Off the Record episode with Martin Splitt and John Mueller, published 2026-07-30. The relevant passage begins around 17:10. Quotes below are taken from the episode's own transcript, not from a secondary write-up.
Mueller: “Way back in the early days, in our webmaster guidelines when they existed as webmaster guidelines, I think we also listed search results pages as something that you have to block from Google… Nowadays, we don't have that listed in the search policies, but it's still something that I think just purely for technical reasons makes sense. So it's not necessarily something where we would say we think your site is spam if you allow your search results pages to be indexed. It's just like — you're being very inefficient.” Splitt confirms the reframing directly: “So we are not seeing this as a quality indication or a quality issue.”
Two caveats stay firmly in place, and both are practical rather than punitive. Crawl waste: an unbounded search path is an infinite space, and Googlebot spending its budget there is budget it does not spend on your real pages. Hacked-content risk: Mueller's example is a site that lets visitors search for terms it has nothing to do with, renders those terms into an indexable page, and ends up ranking for them — “and when we see that happen, we might flag that as hacked.” That is the failure mode worth caring about, and it is unchanged by the guideline being dropped.
One implementation detail from the episode is more useful than the headline. Asked how to write the rule, Mueller recommends one broad rule, matched on the parameter rather than the word: block `/search` followed by `?`, so you catch `/search?q=…` without also catching a legitimate page like `search-for-cheese.php`. And he is explicit that an indexable search page is the second-best version of a better idea: “probably you could do a better job if you just created a real category page… you could provide more context — why is this term important, and which of these pages are actually the best on your website for this term?”
The question Google's episode does not answer
Search Off the Record is about Googlebot. It says nothing about whether blocking `/search` costs you anything with ChatGPT or Perplexity — and that is now the live worry, because five days ago we published Previsible's finding that 28.8% of ChatGPT's referral traffic lands on an internal search page. Read quickly, those two facts look like a conflict: Google says block them, and ChatGPT appears to be sending a third of its clicks to them.
So we measured the other side of it. Previsible measured where ChatGPT's clicks landed, in GA4, across 166 mostly-SaaS properties. We measured what the engines actually cite — the URLs attached to a recommendation answer — in our production corpus of local-business audits. Same phenomenon, opposite end of the pipe.
The result: 0 of 2,269
| Measure | Value |
|---|---|
| Completed answers in the cut (2026-07-31) | 1,048 across 177 audits, 4 verticals |
| Answers carrying ≥1 URL | 546 |
| Cited URLs (answer × distinct URL) | 3,252 |
| Distinct URLs | 2,269 |
| Distinct domains | 1,140 |
| Internal search results pages | 0 |
| URLs carrying any functional parameter | 36 of 3,252 (1.11%) |
Not one of the 2,269 distinct URLs the engines attached to a local-business recommendation is a search results page. The classifier is deliberately narrow and purely syntactic — it never fetches the page. A URL counts as search-shaped only when a path segment is exactly `search` (so `/search-engine-optimization` does not match) or a query parameter is one of `q`, `s`, `query`, `search`, `k`, `find_desc` and friends with a non-empty value. The code is `ops/internal_search_urls.py` in our repo.
A zero is only worth reading if the instrument can produce a non-zero, so this one ships with a positive control that must pass before it will print a number. Twelve fixed URLs: six that must classify as search pages (`yelp.com/search?find_desc=…`, `/?s=hvac+repair`, `amazon.com/s?k=…`, `/search/`, and two more) and six near-misses that must not (`/search-engine-optimization/`, `/researching-a-dentist`, `?q=` with an empty value, a real cited directory URL). All twelve pass. The run refuses to report anything if they don't.
The first version of this script was wrong, and the way it was wrong is worth stating. It read only the URLs written inline in the answer text and reported Perplexity as citing 13 URLs across 525 answers — a nonsense number, because Perplexity returns its sources out of band, in a separate `citations` array. Reading both channels recovers 153 Perplexity URLs. The zero survived the fix; the Perplexity denominator did not survive it as anything solid, and we are saying so: only 12 rows in this cut carry that array, so this is overwhelmingly a finding about ChatGPT (3,086 of the 3,252 cited URLs are its).
What the engines cite instead — and why it agrees with Mueller
The engines are not avoiding search results. They are avoiding search URLs. What they cite constantly is the pre-rendered equivalent — the same list of businesses a search box would have produced, published on a static, crawlable path:
- `medspascout.com/best/top-rated/nevada/las-vegas` — cited in 63 answers
- `counselindex.com/texas/top-personal-injury-lawyers-dallas-texas-2026`
- `lawleaderboard.com/personal-injury-lawyer/portland-or/`
- `discovermedspa.com/denver?sort=rating` — a listing page, sorted; still not a search query
That is Mueller's category-page advice, arrived at from the other direction. He argues a real category page beats an indexable search page because it can carry context about why the term matters and which pages are best. The engines behave as though they agree: the top-cited domains in our corpus are directories that publish the static version — birdeye.com (142 answers), bbb.org (92), bestprosintown.com (82), medspascout.com (63), threebestrated.com (48). Every one of those is a listing page on a clean path.
The 1.11% figure sharpens it. Of 3,252 cited URLs, only 36 carry a functional parameter of any kind, and none of those is a search query — they are pagination (`?page=`, `?reviewsPage=`), sorting (`?sort=`), tabs (`?tab=`) and filters (`?fcountry=`). The engines cite clean canonical URLs almost without exception.
A side finding you can verify on your own server today
3,085 of the 3,086 URLs ChatGPT cited in this corpus carry `?utm_source=openai` — a tag OpenAI appends itself. We confirmed our own pipeline does not add it: the answer text is stored verbatim from the API, and `utm` appears nowhere in our engine code. The practical consequence is that ChatGPT-referred visits are identifiable in raw server logs with a `grep`, with no analytics product, no tag manager and no AI-visibility tool in the path. If you want to know whether ChatGPT is sending you anyone, that is the cheapest possible check.
Three implications
- Blocking `/search` has no measurable AI-visibility cost in this corpus. If the reason you left internal search open was a worry about AI engines, that worry has a number attached to it now: 0 of 2,269. Follow Mueller's rule shape — match `/search` plus `?`, not the word `search` — and take the crawl-efficiency and hacked-content wins.
- Previsible's 28.8% and our 0% are not in conflict, and the difference is the whole lesson. A model that cannot resolve which of your pages answers a question can still construct a search URL and hand it to the user — that click shows up in GA4. It is not a page the engine retrieved and chose to cite. Traffic-side and citation-side instruments see different events, and a business reading only one of them will draw the wrong conclusion about what to build.
- The move is to publish the static page, not to open the parameterized one. For a med spa, a law firm or a home-services company, the version of this that matters is that directories are winning these citations with pre-rendered “best X in Y” pages while the businesses themselves are not. That is the same gap we measure elsewhere: the engines recognise a business in 98.8% of answers when asked by name and name it in 19.8% when asked the buyer's question.
What this does not say
- It is not a census of the web. It is 1,048 answers about local businesses in four verticals — dentists, med spas, personal-injury law, home services. A corpus of SaaS or publisher queries could look different, and Previsible's SaaS figure (34.6% of LLM referrals to internal search) is a reason to expect it might.
- The classifier undercounts by construction. A directory listing served on a pretty path is a search result in substance, and this method counts it as an ordinary page because the URL alone cannot prove otherwise. Every number here is a floor for “static”, not a ceiling.
- It measures citation, not causation. We are not claiming blocking `/search` improves anything. We are claiming the engines were not citing those URLs in the first place, so blocking them removes nothing we can observe.
- Google's AI Overviews and AI Mode are not in this corpus, and neither is Copilot. This is ChatGPT with a thin Perplexity sample.
Do I still need to block internal search results pages from Google?
You are not required to. In the Search Off the Record episode published 2026-07-30, John Mueller confirmed the old webmaster-guideline item is no longer in Search Essentials and that indexable search pages are not treated as a spam signal. He still recommends blocking them, for two practical reasons: crawl efficiency, and the risk that irrelevant terms visitors search for get rendered into indexable pages and cause your site to be flagged as hacked.
Does blocking /search hurt my visibility in ChatGPT or Perplexity?
We found no evidence of it. Across 1,048 AI answers about local businesses containing 3,252 cited URLs on 1,140 domains, zero cited URLs were internal search results pages, and only 1.11% carried any functional URL parameter at all. The engines in this corpus cite clean static paths.
Why does ChatGPT link people to my search page if it never cites one?
Those are different events. A cited URL is a page the engine retrieved and attached as a source. A search-page link in your GA4 report is frequently a URL the model constructed because it resolved your domain but not which page answers the question. Previsible measured the second at 28.8% of ChatGPT referral sessions; we measured the first at 0%. Both can be true at once.
How do I write the robots.txt rule Google recommends?
Mueller's advice in the episode is one broad rule matched on the parameter, not the word: block the /search path followed by a question mark, so /search?q=… is covered while a real page such as search-for-cheese.php is not. One rule is easier to maintain and easier to reason about than a labyrinth of nuanced patterns.
How can I tell if ChatGPT is sending me traffic without any tools?
Grep your server logs for utm_source=openai. In our corpus 3,085 of 3,086 URLs ChatGPT cited carried that parameter, appended by OpenAI rather than by the publisher, so ChatGPT-referred visits are identifiable directly from raw request logs.
The takeaway
Google relaxed a rule and kept the advice. On the AI side the advice costs you nothing measurable: the engines were never citing your search URLs. What they cite is the page you would have to write anyway — a real, static, specific page about one thing, on a path a crawler can reach. That is the same answer the citation data has given every time we have asked it a different way.
The free 60-second check asks the engines your buyers' actual questions and shows you what they say and which URLs they cite. The $79 audit runs 25 questions across ChatGPT, Perplexity, Gemini and Claude, three times each, and returns the source map — the exact pages and domains the engines pulled from, which is the target list you execute against. Agencies auditing a book of clients can run five at once with the $249 Agency 5-pack.
See your number
A free 60-second check shows what AI says about you.
Running this for clients? The $249 agency 5-pack audits five businesses, white-labeled.
Who runs this
- Built and operated by Sensara LLC, Atlanta, Georgia — about us and how the audit works.
- See what the report looks like before you run anything — score per engine, the competitors AI names instead of you, and a fix plan.
- We run the same audit on ourselves every week and publish the result: in the latest run AI named AskedAbout in 1 of 144 answers. We report our own numbers the way we report yours.