Strategies for Scaling Contextual Targeting Segments in Programmatic Advertising

Last week, my team at Gourmet Ads faced an intriguing challenge: building a Custom Contextual Targeting Segment for a new advertiser, working with an Agency Trading Desk we’ve never worked with before. That initially seemed straightforward but quickly revealed its complexity given the many moving parts of the programmatic advertising ecosystem.

Given the increasing importance of precision in programmatic advertising and the growing popularity of contextual targeting due to the deprecation of cookies, I believe sharing our approach to scaling contextual segments can provide valuable insights to others in the industry. It’s not just about the insights, but also the passion we’ve developed for building segments for our advertiser clients. Before I go into how we methodically solved the problem its important to under the campaign, the approach and the issues.

First the Deal Request

In this case the Agency Trading Desk is using Amazon DSP and wanted a series of deals from us to advertise 2 similar products (there were 2 different ASINs) the brand was selling on Amazon. The ATD’s goal was simple, show to the advertiser how providing off Amazon Contextual Deals could not only costs less from a CPM basis, but bring volumes of highly relevant audiences to their two product pages thus increasing Detailed Pages Views and thus increase ROAS. Our Contextual Targeting deals would run alongside other campaign strategies they were using like Amazon In-market Audiences and ASIN Targeting the agency was running.

Now the Problem

Now remember this is a newly created and custom segment not one of our 400+ off the shelf contextual segments sitting, built ready to go in our platform. This is a custom segment just for the advertiser and included their brand, competitor brands, industry terms and localized language.

The contextual topic itself was not the issue; instead, it was the massive disparity in our SSP (Xandr) between Impressions Matched and Bid Requests, which stood at a significant 7%. To put it simply, of all the relevant pages our contextual engine had identified and grouped into the segment, we were only sending 7% of the pages to the advertiser to bid on. In other words, the Bidder Profile used by the advertiser (or in this case, we suspect it was at the DSP level) was limiting us to just 7%. When we started this process, I recall our team saying we had accrued 64,000 URLs in the contextual segment which from our side showed the segment to be large enough for the advertiser. But it wasn’t. This meant that, in order to provide the Agency Trading Desk with a viable chance to scale the Programmatic Deal, we needed to significantly increase this figure by at least tenfold.

When we first started building contextual segments, we though 10,000 URLs would be enough for a segment. The first ever segment we built was Recipe Targeting for Banana Bread Recipes and was about 5,000 URLs. But over time we found that due to variety of limitations we need to be at 50,000 or more for deals to scale.

Getting back to the challenge, you might be asking why would Bid Requests be limited or why would a Bidder Profile be limiting what they can bid on? There are several valid reasons, such as limiting bid requests at the country level, device level, users with cookies, media type level or an approved domain list etc. We don’t know for sure, and what my team have learned over the last few years it's VERY difficult to change this and 9 times out of 10 the people we are working with have no idea how to change this. It’s important to mention, at this stage the Agency Trading Desk hadn’t even implemented any 3rd Party verification services like IAS or DoubleVerify yet. This is straight Publisher to SSP to DSP.

We’ve seen deals throttling from time to time which is simply limiting the bid requests from being sent, especially on deals that haven’t been bid on for a few days. In this scenario, the SSP will automatically limit the bids to say a small sample like 5%. Then when the SSP sees bid responses flowing the throttling is automatically removed and the full volume of bid requests are sent. The additional challenge in this specific situation was that the end advertiser (ie the brand) would not approve the campaign until the Agency Trading Desk could get an accurate forecast in their DSP. The only DSP we know that doesn’t throttle bid requests is DV360.

So, we had to work around these issues to scale the Deal.

How we Scaled the Contextual Segment  

Given this unique problem, I thought it might be interesting to share some of the processes our operations team went through to scale this segment for the client. At the beginning of the process, we started with 64,000 URLs and ended up with 749,000 URLs—an increase of nearly 12 times. Our goal was to strike a balance between scale and specificity, ensuring that the expanded segment could deliver the desired results without diluting the targeting criteria for the Brand.

When we started building our own contextual engine in 2019 and whilst it has several manual processes that are input, most of the work handled by our system is automated processes.

Understanding the Target Audience

We start this by doing basic Audience Research. We conduct research to understand the interests, preferences, and behaviours of the advertiser’s target audience. This starts with asking the right questions to the advertiser about their target audience. It is super important to understand the audience because it forms the foundation of any effective targeting strategy.

Essentially we need to know:

• Who is the target audience? Understanding the demographic profile is crucial. We ask the advertiser to specify traits such as gender (male or female), age range, occupation, and other relevant demographic information.
• What are their interests and preferences? Delving into the interests and preferences helps us understand what kind of content will resonate with the audience. This includes hobbies, lifestyle choices, media consumption habits, and online behaviours.
• What other demographic information is relevant? This could include location, income level, education level, and family status, all of which can influence content preferences and online behaviour.

Once we gather this information, we can start to build a detailed persona of the audience the advertiser wants to target. This persona helps us make informed assumptions about the audience's behaviour, preferences, and potential responses to different types of content. By understanding who the target audience is, we can better tailor our contextual segments to align with their interests and behaviours, ensuring higher engagement and effectiveness in our programmatic campaigns.

Seek Out 100 Websites on the Topic

We then work to identify a list of approximately 100 websites that are highly relevant to the topic at hand. Ideally these will be websites which accept advertising ie we’ll look for sites that have an ads.txt. We have a huge database of over 500,000 domains but then the subset of around half have an ads.txt approx. 270,000 websites. These 100 websites will become the foundation of our contextual targeting segment, and we’ll use it as reference point going forward. At this stage, it's crucial to ensure that these top 100 websites align closely with the advertiser's objectives and the persona we’ve identified for the target audience.

• Conducting an Old-School Search Engine Mission: This process might seem reminiscent of the early days of search engines, but it's incredibly effective. We start by performing manual searches using keywords and phrases that resonate with the audience and the campaign’s goals. This helps us uncover websites that are not just popular but also highly relevant to the topic. So yes, we go site by site looking at the content.
• Evaluating Content Quality and Relevance: It's not enough to simply list websites; each site must be evaluated for content quality, authority, and relevance. We ensure that the content aligns with the audience's interests and that the sites have a reputable standing in the industry. This often involves checking metrics like domain authority, audience engagement, and content frequency. The basic litmus test is would the advertiser be happy to advertiser on all the pages of this website ?
• Creating a Diverse Mix: We aim for a diverse mix of websites, from well-known industry leaders to niche sites that may have more specialized audiences. This diversity helps ensure that the contextual segment captures a broad spectrum of potential consumers.

Competitor Websites & Similar Visitation 

Years ago (way before the term MFA was coined) AppNexus founder Brian O'Kelley had showed me how website co-visitation could help understand the journey of real users online.

Put simply, a user who visited the New York Times was likely (with high probability) to also visit the Washington Post, but low probability to visit a say a low-quality News Aggregator in Spain that was 3 months old. Taking the theory that a web user goes to several similar sites when reviewing or reading a topic we leverage the 100 Websites, and we start to build a build a list of organic competitors’ websites either via relationships or via common keywords. So, we begin an automated process to look at the top 10 search terms of the top 100 Websites we had identified. This process is great for discovering content and more importantly keyword patterns that generally hidden in plain sight. It also finds single pages on major sites that are highly relevant to the contextual segment.  

A great example here is if you are building a segment on say “home loans” the top 100 sites will all be banks and financial institutions. However, if you searched for “home loan reviews” you’ll more likely get websites with advertising as a bank is never going to “review” themselves.

Do a deep Dive on Keyword Research

We love it when clients give us Pay Per Click (PPC) Keyword data (particular insights to what works and what doesn’t is always fantastic data sets) and we try and build out 100 two-word phrases to help. Then from the top 100 we look at the search popularity of these terms and then what websites rank highly for them. We’ll then begin uploading them into our system and indexing all the pages on the site. We’ll add categories (similar to IAB Categories but much more detailed and labels).   

We use a variety of search engine optimization tools to help build a keyword profile for the segment. We’ll also look for negative words here or words that might inadvertently include bad content or irrelevant content. This was particularly relevant in the campaign here, and we had over 100 campaign specific negative keywords implemented.

Localization and Regional Relevance

Understanding the importance of localization is critical to building a truly effective contextual segment. Different regions and countries may have varying preferences, language nuances, and cultural specifics that need to be considered.

• Incorporating Localized Keywords: While crawling and analyzing content, we pay attention to localized keywords that may differ from standard or global terms. This can include regional slang, culturally specific phrases, or terms that have particular significance in a certain area. These localized keywords are essential for ensuring the content resonates with audiences in specific geographic locations.
• Understanding Regional Content Trends: Different regions may prioritize different types of content based on local interests, holidays, or events. We strive to identify these trends and incorporate them into our contextual segments to increase relevance and engagement. This involves monitoring regional news, social media discussions, and popular content within those areas.
• Adapting Content to Local Context: Beyond keywords, we also consider the broader cultural context. This includes understanding regional preferences for media formats, tone, and even visual styles that may be more appealing in certain locations. Tailoring content in this way helps ensure that the segment feels personalized and relevant to each audience.

Crawling Websites and Individual Pages

There are two parts to this. We already have massive index of URLs in our dataset. Last check it was around 7.7 billion URLS on all range of topics. Think of this as our view of the Open Web. But we’ll nearly 100% always run a new crawl and sometimes recrawl sites that we are interested in to build a segment.

Once we have our initial list of websites and got a subset of keywords, we’ll set a fresh crawl. This involves scanning existing and new websites, getting the latest URLs and then crawling individual pages to gather data that will be used to build the contextual segment.

• Automated Crawling Tools: We employ advanced crawling tools to scan the websites systematically. These tools help us gather URLs, extract keywords, and identify patterns in the content that are relevant to the target audience.
• Analysing Content Depth and Keywords: As we crawl through the websites, we focus on analysing the depth of the content and the keywords used. This involves looking at article titles ie H1, H2, H3, meta descriptions, and body content and in some cases sentiment to ensure they align with the interests and behaviours of the target audience.
• Data Collection and Categorization: The data collected is then categorized based on themes, topics, and relevance. This helps us create a structured database of URLs that will form the backbone of our contextual segment.

The URL Contextual Segment Gets Built and Pushed

When the web crawls have finished, and our system has meticulously gathered and analysed the relevant URLs we deploy these into the segment. This is really an automated decisioning process based on our algorithm of what to include and more importantly what not to include.

In this case of the Deal above, the Agency Trading Desk had asked us to go broader to get scale. In this case not only did we need scale, but the client wants larger scale to reach as many consumers as possible. This helped as we could find more content than we had original built.

When the contextual segment is finished being built by the system, this is automatically uploaded to into Microsoft Advertising 's Xandr platform and we then can start building the Deals leveraging this segment. As long as the DSP is integrated with Microsoft Advertising, we can create a deal for the Advertiser. Each DSP has different integration models. Some are built at the Bidder level others are mapped to a Buyer Seat IDs. Some Deals are auto sync’d across to their buyer seat such as Google DV360, The Trade Desk and Amazon Ads DSP. We generally see “Impressions Matched” about an hour or so later and the advertiser can start buying the deal.

Forecasting Challenges

Every advertiser wants to know the size of the segment and the size of the audiences. For newly created contextual segments this is a massive challenge. Typically we need 7 to 10 days of the segment being created in order to understand the size of the audience. One work around is to simply create a deal and send it to the advertiser’s DSP. This will quickly establish the number of Matched Impressions in a 24 hour period and can be used to get a rough idea of the size of the audience.

Rebuilds & Recency

Once the segment is built, we have system called “Automated Rebuilds” and depending on the frequency required by the advertiser we can refresh or rebuild a segment. This is exactly how it sounds and we can rebuild the segment at any frequency from 1 day to 365. Most active contextual targeting segments with active deals are rebuilt daily to weekly depending on the size of the segment, advertiser requests. We have a few techniques when it comes to learning about new content with around 45-50k new pages of content ingested each day.

Some contextual segments rarely change like there is not a lot going on when it comes to say Chicken Recipes. Sure, there is new content being created daily but its not mission critical to be updated daily. Whereas a topic like the GLP1 class of medications (like Ozempic or Wegovy) has volumes of new content created daily right around the world and its important to make sure we are doing daily updates as we want advertisers to be recently created and relevant content.

Timeframe

We often say to advertisers that our team can build a brand new custom contextual targeting segment in 24 hours, which is true. We have approx. 2-3 hours of manual build time and process and then a few hours of automation. But the creation of the segment can take like 20 mins before its available in the segment and we could create the Deal shortly after this.

However, the reality is that we’d like a few rebuild processes to occur before we say it’s complete. We typically like advertisers to give as 3-7 days before the campaign needs to go live so segment includes most of the URLs we need to scale it.

The end result

In this case, rerunning the contextual targeting build process above from scratch allowed us to gain a total of 749,000 URLs. The Agency Trading Desk had allowed us to also go broader on the topic that we initially did on the first build which did help. I think if we had not been able to broaden the topic base, we would have likely just doubled the size of the URLs to approximately 120,000 URLs.

The result was we ended up with 749,000 URLs in the segment —an increase of nearly 12 times our original size. Whilst the Impressions Matched to Bid Request ratio did increase, the underlying issue here was there was enough impressions being sent to Amazon DSP to allow the Agency Trading Desk to get a reasonable forecast for Advertiser.  

The Agency Trading Desk is planning on pushing the deals live this week and thanks to the rebuild will have the ability to layer on whatever targeting they want or need (Viewability maybe?) to ensure campaign success.

#contextualtargeting #programmaticadveritsing

All the Images within have been created by ImageFX