Someone finally ran the experiment: AI Overviews cut clicks 39.8% — and the clicks that vanished were not the worthless ones

Where this stands on September 27, 2026: the paper was revised again, and a second randomized experiment agrees on direction

We re-read the paper at the source on 2026-09-27. SSRN now lists the working paper as posted April 3, 2026 and last revised July 8, 2026; the version this post first reported was the June 17 revision. The July 8 PDF (41 pages) carries the same headline figures: 1,065 US participants, 68,089 searches between January 7 and February 10, 2026, outbound organic clicks down 39.8% and zero-click searches up 34.5% when an AI Overview appeared, no change in sponsored clicks or search frequency, and no measurable improvement in perceived search quality or ease of finding information. The abstract now states the comparison as "relative to the no-AIO counterfactual", and the title footnote gives the pre-registration, AEA RCT Registry AEARCTR-0017393. The AI Mode arm (28% of participants, 12,507 search instances) is reported as descriptive: fewer external clicks per search and more zero-click searches than the control, satisfaction "substantially lower", and higher attrition noted by the authors.

Since July a second randomized experiment has been posted, by a different team on a different quarter's data. Stephanie T. Wang, Jeffrey Gleason, Yakov Bart, Christo Wilson and Danaé Metaxa (University of Pennsylvania and Northeastern University) posted AI in Search Reduces Publisher Referrals Without Improving User Experience: Experimental Evidence on arXiv on August 18, 2026 (Search Engine Journal reported it on September 21). It is pre-registered, recruited US Chrome users on March 17 to 19, 2026, ran three baseline days and then seven treatment days, and analyses the 1,100 participants who made at least one search. Three arms: Google as it is, a no-AI arm that hid AI Overviews and redirected AI Mode to ordinary results, and an arm that routed every search to AI Mode.

The two experiments measure different quantities, a per-search click cut conditional on an AI Overview appearing against a per-user change in click-through rate, so the numbers do not add or convert into each other. They agree on direction on every outcome both measured: removing AI Overviews returns clicks to publishers, forcing AI Mode removes far more, and neither AI surface improved how users rated the experience. The second study adds one figure the first could not: AI Mode showed no ads during its field period, so the share of users clicking an ad fell 42.7 percentage points in that arm. Every figure above is read from the two PDFs on 2026-09-27; the file hashes and page reads are in the committed source record for this post.

What the study did, and what it found

The paper — The Impact of Google AI Overviews on Publisher Traffic and User Experience: Evidence from a Field Experiment (SSRN 6513059, posted April 3, 2026, last revised July 8, 2026 on our 2026-09-27 read) — is the first causal test of the feature rather than an after-the-fact traffic correlation. As PPC Land and Search Engine Journal both report:

• Method: a custom Chrome extension randomly assigned participants to see Google's normal results page, a version with AI Overviews stripped out, or AI Mode — then passively tracked what they actually did. 1,065 desktop Chrome users in the US, 68,089 searches, data collected January 7 – February 10, 2026 (two weeks per participant). AI Overviews fired on roughly 41% of queries.
• Outbound organic clicks fell 39.8% when an AI Overview appeared. Zero-click searches rose 34.5%.
• Sponsored clicks did not meaningfully change. The ads kept getting clicked; the organic links did not.
• Users were not happier. Survey results were, in the authors' words, “precisely estimated nulls” on satisfaction, perceived information quality, and ease of finding information — no measurable improvement.
• The lost clicks were not the low-quality ones. This is the finding from the revised version. Google's standard defense is that AI Overviews only absorb visits that were worthless anyway. The researchers checked: about 4 in 10 same-tab clicks bounced back to the results page in both groups, roughly 18% of visits ended within 10 seconds in both groups, and time-on-site was statistically indistinguishable. They conclude the data is “at odds with the view that AIOs primarily eliminate low-engagement website visits.”

Why this matters even if you never cared about publisher traffic

It closes an argument. Until now, a business owner could reasonably tell themselves that the AI-search panic was an SEO-industry story about media companies, and that the traffic decline was confounded by a dozen other things. A randomized experiment removes the confounds: when the AI answers, the click does not happen — 40% of the time — and the people who didn't click were not the tire-kickers. The traffic that AI absorbs is real traffic.

The second-order consequence is the one that hits a local business. If four in ten organic clicks evaporate at the answer layer, then the answer itself — the paragraph the model writes, and the businesses it names inside that paragraph — becomes the thing that determines who gets the customer. It is no longer a preview of your website. It is a substitute for it. And you cannot see it happening: an AI that answers in-place and names your competitor generates no lost click in your analytics, no impression, no bounce. Nothing. Just a booking that goes somewhere else.

What to do with it

The measurement most businesses reach for — Google Search Console — was never built to answer this, and the new AI report doesn't fix it: it reports AI impressions only, not whether the model recommended you, and covers Google's surfaces only. The number that survives this study is a different one: how often does each engine name your business when a real buyer asks a real buying question? In our own measurements that number is low and unstable — across 152 local businesses the average was named in 46% of the answers its own customers would see, and in repeat runs of the same question on the same engine only 31% of recommended brands survived every repeat. The instability runs deeper than brand names: the core of domains cited by two or more engines replaced a third of its members in six days, and even the #1 cited domain in our four-engine corpus — reddit.com — owes nearly 70% of its citations to one engine.

The takeaway

Agarwal and Sen have made the click loss a fact rather than a vibe, and taken away the excuse that the missing visitors didn't matter. If 40% of the clicks are gone at the answer layer, the only defensible thing to track is the answer. A free 60-second check asks ChatGPT and Perplexity (the $79 audit adds Gemini and Claude, 25 questions sampled three times each) the questions your customers ask and reports how often each one recommends you — the metric that keeps working after the click stops.