Like all the best stories in 2025, this one starts with a post in a Facebook Group.
I’m a proud regular at Hershey Park and have gotten a ton of helpful info (and some juicy gossip) from a few Facebook Groups. The scuttlebutt of late has been that a long standing ride, Wild Mouse, was closing at the end of the summer. The proof? A screenshot from Google showing confirmation in the AI Overview that the ride was closing.
As somebody who follows search results closely, I’ve seen plenty of strange things in the AI Overviews so I took a closer look.
Was the source info for five-year plan referenced in the AI Overview a press release from Hershey? A social post from the park? A news article?
Not quite – it was derived entirely from TheShaiv, a Reddit user who started a thread outlining his concept of what Hershey Park’s 5 year plan could be. Yes, a hypothetical 5 year plan from a random (and very creative) reddit user.
Google’s intent with serving an AI overview is clear: they want to drive better user experience by providing instant answers. But when things are just slightly off, you end up with a random Reddit user’s hypothetical five-year plan being the source of truth for your global brand in search results.
This isn’t an isolated incident. Since Google rolled out AI Overviews in May 2024, marketers have watched in horror as the feature confidently shares everything from dangerous health advice to completely fabricated business information. According to MIT Technology Review, within days of AI Overviews’ release in the US, users were sharing examples of responses that were strange at best. It suggested that users add glue to pizza or eat at least one small rock a day, and even confidently stated that historical figures achieved impossible academic milestones decades after their deaths.
What makes this especially dangerous for businesses? These aren’t just silly mistakes; they’re systematic misinterpretations that can damage brand reputations, spread false information about your company, and confuse potential customers about your products or services.
The Real Problem: Context Is Difficult For AI
The crux of the issue is this: Google’s AI doesn’t understand context the way humans do. Google has even admitted this, that sometimes the AI Overviews are embarrassingly wrong.
Here’s what’s actually happening under the hood.
Google’s AI is incredibly good at finding relevant information. When you search for “Hershey Park Wild Mouse closing,” it can scan millions of web pages and identify that TheShaiv’s Reddit post contains those exact keywords. The AI sees: “Hershey Park,” “Wild Mouse,” and “closing” all in one place…that’s relevant content.
But here’s the problem: it completely misses and disregards context clues: a random Reddit post and the mention of it being a concept.
Think of it like a hyper-efficient research assistant with little common sense. They’ll find exactly what you asked for, but they might hand you a college student’s fantasy football draft as official NFL roster changes.
Nailing the context requires understanding human intent, sarcasm, speculation, and authority in ways that current AI systems simply can’t master…at least yet. It’s the difference between finding information and understanding information.
Four Reasons AI Overviews Can Be Inaccurate
1. The Satire Problem
AI systems struggle to recognize satirical content, even when it’s obvious to humans. As Google explained in their post-launch analysis in 2024, “In other examples, we saw AI Overviews that featured sarcastic or troll-y content from discussion forums. Forums are often a great source of authentic, first-hand information, but in some cases can lead to less-than-helpful advice, like using glue to get cheese to stick to pizza.”
Business Impact: If someone creates satirical content about your business on Reddit or another forum, AI Overview might present it as factual information to searchers.
2. The Authority Confusion
The Hershey Park example perfectly illustrates this: AI Overview couldn’t distinguish between an official company announcement and fan speculation. In some cases, an LLM may extract statements from a source without considering its context, resulting in an incorrect conclusion. When AI systems retrieve information, they often lose the critical context cues that would help humans understand the source’s credibility or intent.
Business Impact: Speculation about your company can be presented as official announcements, potentially affecting stock prices, customer decisions, and brand reputation.
3. The Time Warp Effect
According to TechRadar, Google’s more-than-frequently-wrong AI Overviews search feature still thinks it’s 2024. It struggles to know if the most relevant information it finds is also the most recent piece of information. A quick example – a search on who wears a particular jersey number on a sports team may provide outdated information, say a player who used to wear a jersey number or who is known around the web for wearing a number, say a famous former player.
Business Impact: Outdated information about your products, services, or company might be presented as current facts, confusing potential customers about what you actually offer.
4. Lack Of Information
When there’s limited high-quality content available on a topic, AI systems may fill gaps with whatever information they can find, even if it’s completely off-base. Google uses the term ‘data void’ or ‘information gap’ to describe these instances.
Think about searching for information on a new product launch, a recent business acquisition, or a niche industry regulation. If authoritative sources haven’t written about it yet, AI might pull from speculation, rumors, or completely unrelated content that happens to contain similar keywords.
Marketing Impact: For new products or services where limited information exists online, AI might fill gaps with irrelevant content, outdated information from similar products, or even competitor information that happens to share keywords with your brand.
What Businesses Can Do to Keep Information Accurate
The reality is that AI Overview failures will continue happening. The technology’s fundamental approach – retrieving information without always fully understanding context – makes these mistakes inevitable. But you can minimize the damage:
Create Authoritative Content: The best defense against AI misinformation is comprehensive, clearly structured content that leaves no room for misinterpretation. Publish detailed FAQs that address common misconceptions, create regular company updates that clearly state current information, and establish your website as the definitive source for information about your business.
Monitor Your Narrative: Set up Google Alerts for your brand name plus terms like “closing,” “discontinued,” or “changes.” Regularly test AI Overview responses for your key business terms and document instances of misinformation. Create authoritative press releases to counter false narratives when they emerge.
Clarify Context: Help AI systems understand the context of information about your business by using clear, unambiguous language in all official communications. Include publication dates and “last updated” timestamps on important pages, and create clear distinctions between official announcements and customer discussions.
Establish Credibility: Build your brand authority all around the web and on lots of 3rd party websites and social platforms. Optimize your Google Business Profile with current, accurate information, earn high quality mentions from reputable industry sources, and maintain consistent business information across all platforms.
The Competitive Reality Behind AI Search
Getting accurate information about your business into AI Overviews is paramount, as Google keeps showing more and more of them in results.
Why? Google is navigating an intensely competitive search landscape. With ChatGPT, Perplexity, and other AI tools providing instant responses to user questions, there’s natural pressure to show AI-generated answers for more queries to remain competitive.
Our research at WebFX showed that users increasingly expect immediate, AI-powered answers rather than having to click through multiple search results. This creates a challenging balance between providing comprehensive AI responses and maintaining accuracy, especially for edge cases and uncommon queries where high-quality sources may be limited.
Google has stated that AI Overviews are designed to surface when their systems are most confident that an AI Overview will be high-quality and helpful. However, the business reality of competing with other AI systems means finding the right threshold between being helpful and being accurate…a challenge every AI company faces today.
But here’s what businesses need to understand: the vast majority of AI Overviews provide high-quality information and the strange, puzzling answers are likely to be less frequent as Google’s technology improves.
Increased frequency of AI Overviews (even the strange ones) represent a fundamental shift in how people consume information online. Instant answers aren’t going away. The companies that thrive in this new environment will be those that proactively create authoritative content, monitor AI-generated information about their business, and implement systematic approaches to combat misinformation.
Interested in making sure your business is visible across all Google and LLMs like ChatGPT? Check out our OmniSEO offering.
Track Rankings Beyond Search
Your prospective customers are searching in AI engines beyond traditional search. Are you ranking?
Future-Proof Your SEO Strategy with OmniSEO™
Goodbye search engine optimization, hello search everywhere optimization.
Future-Proof Your SEO Strategy with OmniSEO™
Goodbye search engine optimization, hello search everywhere optimization.
Writers
