18 months ago, Google started encouraging advertisers to upgrade their Dynamic Search and Google Display campaigns to – you guessed it! – PMax. They promised conversion uplifts of 15% and 20% respectively, and now they're asking you to put it to the test. But wait, how does testing actually work in an AI environment? Let's have a look. 𝗣𝗠𝗮𝘅𝗶𝗳𝗶𝗰𝗮𝘁𝗶𝗼𝗻 Since the launch of PMax three years ago, Google has been swallowing legacy campaign types alive. First there were involuntary migrations of Smart Shopping and Local campaigns. And later followed announcements for DSA, GDA, and Smart campaigns. Google has even focus-grouped the idea of migrating keyword Search campaigns to PMax. We call this "PMaxification." It's a strategy of incubating PMax as a new AI ad platform within the old one. It helps cement PMax as the future of Google Ads. 𝗱𝗔𝘁𝗔-𝗱𝗥𝗶𝗩𝗲𝗡 𝗱𝗘𝗰𝗜𝘀𝗜𝗼𝗡𝘀 Google has long used testing and experiments to help sell their technology. In the age of data-driven decision making, it seems like a perfectly fair approach – perhaps even unquestionable to some. However, not all tests are created equal, and we should not permit data to become an "article of faith" that overrules our critical thinking. Google's Performance Max Experiments tool is an owned, blackbox testing environment with labeled tests. Without sounding too conspiratorial, just thinking critically: there is a conflict of interest, the means to unobservably cheat are given, and it cannot be ruled out that the testing environment is not stacked. This is why we prefer to use tools like Google's Matched Markets python library. It has the robustness of Google data science, but the code is open-source and visible, and Google is agnostic to the test. 𝗘𝗻𝘁𝗲𝗿 "𝗱𝗶𝘃𝗲𝗿𝗴𝗲𝗻𝘁 𝗱𝗲𝗹𝗶𝘃𝗲𝗿𝘆" A recent article by the American Marketing Association highlighted a phenomenon called divergent delivery. This is when the distribution of user types becomes uneven across different test groups, leading to wrong inferences and outcomes. It is exacerbated in AI advertising, where AI can dynamically alter the creative AND algorithmically select the audience. Moreover, AI also decides the bids and placements. This is executed independently and with minimal to no reporting. Even in a case where the test design is fair – and let's please assume it is 🫶 – the modern ad platforms are so multivariate, so non-deterministic, and so confounded, that it's just very hard to be data-driven and not "dAtA-dRiVeN." 𝗦𝗼 𝘄𝗵𝗮𝘁? We're not saying "don't test." We are asking, reminding, imploring you to be please be mindful about testing. It's 2025. Set testing conditions that are robust and transparent. Speak openly about conflicts and incentives. Be aware that testing was never perfect. Be aware that testing is even less perfect in AI environments. Don't forget your brain, your judgement, your experience.
Vladimir Petkov
2mo
I would be testing with removing Pmax and relocating to actually controllable and more transparent campaign types.
Full professor of Marketing, Audencia Business School.
2moGuilhem Bodin