Overview

AI Overviews (AIOs), a feature that offers a Generative AI summary of information above organic search results for certain queries, have had a volatile rollout starting in May 2024.

Here's an AIO example:

To understand the initial impact of Google's AI Overviews on organic keyword rankings, we asked our clients to send us a sample of the keywords they were most concerned about in terms of the potential effects of AIO.

Our early research, based on 2,700 keywords, provides an initial look at the findings, segmented by industry, keyword intent, and keyword type (e.g., YMYL, local, and buying guides).

Key Takeaways

Prevalence of AI Overviews: Currently, AI Overviews appear in 7.6% of the searches (down from 17.4%).
>99% of the instances of AI Overviews, it was sourced from top 10 web results.
Limited Initial Release: The initial release of AI Overviews is highly limited, predominantly affecting informational queries.
Future Tracking: We will continue to track the number of queries that display AI Overviews as more data becomes available.
Immediate Testing: As soon as you have access to AI Overviews, we suggest testing the queries that matter most to your business.

Of the thousands of queries we tracked before May 30, 2024, 17.4% resulted in an AI Overview response.

On May 30, 2024, however, Google released an AIO pullback announcement stating that they made over a dozen technical system improvements to address concerns over "odd, inaccurate, or unhelpful AI Overviews."

Since this announcement, diverse types of client sites that all had over 20% AIO for their keywords before May 30 saw an overall drop of 20% of those keywords showing AIO.

For clients we analyzed post-May 30, the average appearance of AIO is now down significantly to 7.6%.

We will continue tracking AIO and updating this figure as we gather more data.

AIO Featured Only 6% Mobile than Desktop

Based on a US keyword data set in seoClarity's Research Grid, as of September 2024, data shows that AIOs are featured only slightly more on mobile (4.2%) vs. desktop (3.9%).

In short, mobile appears 6% higher than desktop.

We've built a complete workflow within seoClarity that enables them to take full advantage of this data to see the impact of AIO on the traffic and to measure the CTR impact this could have.

AI Overviews in seoClarity

Top 10 Web Results Feature in AIO Sources 99.5% of the Time

We analyzed 36,000 keywords and compared the URLs showing up in AIOs to those ranking in the top 10 organic results.

Our findings revealed that one or more of the top 10 web results were included in the AI Overview's sources a staggering 99.5% of the time.

This marks a substantial change from what we saw in SGE and the early days of AIO where there was very little correlation between ranking sites and what appeared in AIO results.

Digging deeper into the data, we also found that nearly 80% of the AI Overview results contain a link to one or more of the top 3 ranking results. And when looking at just the top 1 position, the AI Overview contained a link to it almost 50% of the time.

We speculate that this shift is in response to widespread criticism of the frequent inaccuracy of AIOs, prompting Google to incorporate some traditional ranking factors in selecting AIO sources.

The bottom line? One of the best ways to get referenced in an AIO now is to rank highly in Google.

Some additional background on these results:

These are early results for 36K keywords. As AIOs keep expanding we may see a decline in the percentage of sources from top 10 results. Nevertheless, even this early glimpse indicates a significant change from Google.
The keywords analyzed are those tracked by our clients, so they tend to be "money" keywords (the informational and transactional keywords these clients care about most). As such, results may vary for more obscure, long-tail searches.
These results also highlight a substantial advantage that Google has over standard LLMs — its understanding of relevant and authoritative sites — which it is leveraging to improve the AI results. Google understands website authority and has a lot more signals than any standard LLM like ChatGPT or Perplexity would have.

77% of AIOs Include Referenced Links Exclusively From the Top 10 Web Results

Looking at the data a bit differently, we also sought to uncover how many AI Overviews had ALL the referenced links from the Top 10 web results.

Incredibly, 77% of all keywords triggered AI Overviews only included references from the Top 10 results.

This further solidifies our hypothesis that Google is confident in its existing rankings system's ability to generate high-quality AI Overview summaries.

Transactional: AI Overviews do not currently appear for most transactional head terms but may show when there's a mix of commercial and informational intent.

Informational: Although informational queries produce AI Overviews, the frequency is lower than what we observed in SGE (Search Generative Experience) – the experimental version of AI overviews available only to users who opted into it in Google Labs since May 2023.

Compared to SGE, informational queries deserving summaries (QDS) have decreased, but AI Overviews still appear primarily for keywords with strong informational intent, especially where users seek to perform a task or obtain detailed steps (as of May 30, 2024).

AI Overviews Impact 44% of YMYL Keywords

YMYL (Your Money/Your Life) queries are those that could affect a person’s physical or financial well-being.

Based on our analysis of 50 YMYL keywords tested on June 3, 2024, AI Overviews appear in 44% of these queries. Specifically, 58% of "Life" keywords (15 out of 26) and 24% of "Money" keywords (7 out of 24) now show AI Overview results.

AIO YMYL - June

This analysis was conducted just four days after Google announced significant changes to enhance the accuracy and reliability of AI Overview results.

Notably, all source links in these AI Overviews now come from reputable sources such as the Mayo Clinic and CDC, and most are from pages already ranking in the top 10 organic search results.

AI Overviews Don’t Appear for Local Keywords

Previously, Search Generative Experience (SGE) appeared for local results. But, based on our sample, AI Overviews are no longer shown for these queries.

Recommended Reading: 5 Research-Backed SGE Insights All SEOs Should Know

We tested numerous searches that typically trigger Local Map results and the Three-Pack, but none displayed AI Overviews (as of May 30, 2024).

AI Overviews Appear Inconsistently For Buying Guide Keywords

SGE results also used to appear for short queries like "child’s bike buying guide," but AIO for such queries are now less common.

AI Overviews may still show for longer queries like "guide to choosing the right bike for a child," though not consistently (as of May 30, 2024).

Research Methodology

These insights are based on the analysis of 2,700 keywords from diverse clients, each submitting their most important keywords through June 2024.

As of August 15, 2024, Google rolled out AI Overviews to non-logged in users. Within seoClarity's Research Grid of 500+ million keywords in the United States, it's reporting that AIOs triggered for nearly 1 million keywords in the last ~10 days.

The newest insights that analyzed 36,000 of these keywords are from August 15, 2024 to August 22, 2024.

More data and insights to come as this continues to rollout.

Note: The figures in this study may be skewed slightly high due to the complex informational nature of the queries these clients value, which tend to produce AI Overviews more frequently.

Next Steps

To monitor the impact of AI overviews on your rankings and traffic, we’ve outlined several ways to do so using the seoClarity platform – including using our new AI Overviews filter in SERP Features which tracks AIO summaries generated as a "Top Insights" panel.

In addition, clients now have the ability to:

See which keywords trigger AI Overviews
Check whether they or their competitors are linked/referenced
Review the AIO content itself (through SERP Previews)

This makes seoClarity the first and only Enterprise SEO platform to provide AIO tracking at scale with in-depth details.

Not currently an seoClarity client? Learn more about how our platform can help you monitor and strategize for AIO by scheduling a quick demo.

About the Author: Mark Traphagen

Mark Traphagen is Vice President of Product Marketing and Training for seoClarity. His previous careers as a salesman and then a teacher were excellent preparations for what he does now. Mark's deep dive into online marketing began about 14 years ago when he was tasked with turning around a failing independent bookstore. He took the store online and developed practices of content marketing, influencer relationships, exceptional customer service, and community building that brought the store to profitability in two years. Since that time he has dedicated himself to learning what really works for online business success. He is a sought after speaker at conferences such as PubCon, State of Search, and SMX, and a featured writer for major online publications such as the Moz Blog, Search Engine Land, Marketing Land, and a VIP Columnist for Search Engine Journal.

Full Enterprise Platform + Support

Rankings

Keyword Research

Content Marketing

SEO Reporting

Technical SEO

Competitor Insights

SEO Execution

Data Services

seoClarity Research Study

Impact of Google’s AI Overviews: SEO Research Study

Contents

Overview

Key Takeaways

AI Overviews Appear in 7.6% Of Searches (Down From 17.4%)

AIO Featured Only 6% Mobile than Desktop

Top 10 Web Results Feature in AIO Sources 99.5% of the Time

77% of AIOs Include Referenced Links Exclusively From the Top 10 Web Results

AIO Impact by Industry

Impact By Intent: AIO Primarily Appear for Informational Queries