Every few weeks someone sends me a screenshot of an AI Overview getting something badly wrong and asks whether the whole thing is broken. Every few weeks someone else sends me Google's statement that Overviews rarely hallucinate and asks whether they can stop worrying.
Both are answering a question that does not matter much to a brand. Whether Google can be tricked into saying something absurd about pizza has nothing to do with whether the Overview describing your category is fair, or whether it will still be describing it that way tomorrow. Those are the questions I would ask, and both are measurable.
What Google claims about AI Overview accuracy
Google's own account of accuracy is more specific than the coverage suggests, and worth reading directly. In the write-up after the May 2024 launch, the company said that because "accuracy is paramount in Search," Overviews are built to show only information backed by top web results, so they "generally don't hallucinate or make things up in the ways that other LLM products might."
Then it named the ways they do fail: misinterpreting queries, misinterpreting a nuance of language on the web, or not having a lot of great information available.
The original launch announcement had promised Overviews would "do the work for you." Two weeks later the same blog was explaining what had gone wrong, which is a faster correction cycle than most vendors manage.
The pizza answer gets explained in that second post. It came from a forum, on a topic Google describes as a "data void," where almost no serious content existed and satire was most of what did. The system retrieved what was there.
I find that explanation more useful than the joke, because a thin category is the same condition. If very little good content exists about how your market works, the Overview gets assembled from whatever does, and that usually means review sites and forums rather than anyone who knows the product. Our methodology is built around measuring that rather than guessing at it.
Accurate is not the same as consistent
An Overview is generated when someone searches. It is not stored and served. Two checks ten minutes apart can name different brands, cite different pages, or show no Overview at all, and every one of those answers can be factually fine.
This catches people out constantly. A brand reads an Overview, finds nothing wrong in it, and assumes they are set. An hour later the same query names a competitor instead. Nothing broke. The answer was regenerated.
What this looks like in practice is less dramatic than it sounds. It is rarely one brand appearing and vanishing. More often three vendors are named every time and a fourth slot rotates between two others, week after week, with nothing changing on anyone's website. If you are the rotating slot, a single check tells you either that you have won or that you have lost, and both readings are wrong.
So "is it accurate" cannot be answered by looking once. It is a question about a distribution, which is why we report visibility as a rate with the number of checks beside it, and show a fraction rather than a percentage when there are fewer than five.
An AI Overview can be accurate and still wrong for your brand
There is a third kind of wrong that neither Google's statement nor the viral screenshots cover: an Overview that is entirely correct and still damaging.
"Reliable but expensive." "Powerful, though the setup takes a while." "A solid choice for larger teams." Each is true, sourced, and defensible. Each one costs you the deal with the buyer who reads it. No fact-check will ever flag them, because nothing in them is false.
That is why we rate how you are described as its own measurement, keep the sentence that produced each rating, and let you disagree with it. A sentiment score you cannot open is a number you should not act on. In our experience a recurring criticism traces back to two or three specific pages, which makes it a fixable problem rather than a reputational one.
Being cited is not the same as being read
The other belief worth dropping is that a citation delivers visitors.
Pew Research Center followed the browsing of 900 US adults through March 2025. Users clicked a link inside an AI summary on 1% of visits to pages that carried one. On pages with a summary, 8% of visits produced a click on any result at all, against 15% on pages without.
Pew is not alone. Ahrefs measured a 34.5% drop in clicks for the top-ranking page when an Overview appears. SparkToro's zero-click study found that only 374 of every 1,000 US Google searches end with a click to the open web at all. Seer Interactive tracked organic click-through falling year on year on queries where an Overview shows, and paid click-through falling with it.
Put those beside the accuracy question and the conclusion is uncomfortable. If the Overview is where the answer gets consumed, the wording of the Overview is the product experience, and your citation is a footnote almost nobody opens.
Citations still matter. Being cited means Google retrieved and trusted your page. But count it as retrieval evidence, not as traffic. And watch for the state that matters most: cited without being named. Google read you, and then described someone else.
How accurate is AI Overview on your own queries?
Google's guidance for site owners is that its generative features are "rooted in our core Search ranking and quality systems," and its documentation of the AI features themselves describes Overviews and AI Mode as surfaces built on the same index. So the fundamentals still apply and there is no separate Overview lever to pull. What changes is how you measure, not what you do.
A workable approach:
- Ask repeatedly. One check is a coin flip. Twenty is a measurement.
- Record "no Overview shown" as its own state. A query where none appears is not a loss. It is a query with nothing to win yet, and folding it into your percentage distorts the number in a random direction.
- Keep the full text. The wording is the evidence. A score derived from it and then thrown away cannot be audited by you or by us.
- Separate the four outcomes. Named, cited but not named, absent, and no Overview each mean something different. Averaging them destroys the one signal that tells you what to do next.
That list is what it takes to track AI Overviews properly. If you want to change the outcome rather than just measure it, that is how to rank in Google AI Overviews. And RankHound, the AI visibility platform brands use to get recommended by AI, runs all of this for you. Its AI Overview tracking checks again on a schedule and reports the four states separately, so you can see whether a change worked.
So, are they accurate?
Mostly, on the narrow definition. Google's claim that Overviews rarely fabricate matches what we see, and the documented failures cluster in thin topics and misread nuance rather than invention.
But "accurate" was always the wrong word for what brands worry about. An Overview can be accurate and inconsistent. It can be accurate and unkind. It can be accurate, cite your page, and still name a competitor in the sentence the buyer actually reads.
Those three are measurable. "Is it accurate" is not, at least not in a way you can act on, and that is the better reason to stop asking it.
