Human vs. algorithm: Should you interfere with the Facebook ad delivery AI?

As Growth Lead at Bower Collective, an online direct-to-consumer (D2C) eco cleaning brand, learnings from experiments come in thick and fast. Every two weeks, I'll be sharing a bite-sized post that outlines a marketing challenge, the experiment I conducted to overcome the challenge, and the learning I took forward into our growth marketing strategy.

Context for beginners

When you click on a Facebook or Instagram ad that takes you to a website where you buy something, Facebook Ads Manager displays that information to the owner of that website in real time. This is very powerful because as an advertiser you can see exactly how much revenue is being driven by each of your ads. Advertisers will often pay close attention to this information to see which ads resonate well with people and drive a lot of revenue, and which ads have less impact and drive much less revenue.

Using a metric called ‘return on ad spend’ (ROAS), Advertisers can see how much revenue an ad is generating relative to the amount they’ve paid to Facebook to show people that ad.

The challenge

Below is an example of the data an advertiser might see when analysing performance of three creatives. In this example, ad 1 has performed the best, but Facebook spent most of the budget on ads 2 and 3 which saw much lower performance. I experienced this recently when a high-performing creative (4x higher than the others) was neglected by Facebook.

Why did Facebook do this if it's meant to optimise spend by creative to get the best results? What’s it playing at?

Often it's because the Facebook ad delivery AI knows that not all creative will scale linearly. In other words, Facebook has predicted that while one creative may initially have great performance at a low spend level, that trend won't continue as spend increases. For that reason, Facebook doesn't overcommit budget. Typically, there will be another creative (e.g. ad 2 and 3 in the table above) that it has predicted will see a more consistent performance trend as spend increases.

Essentially, to get you the best results, Facebook is always looking holistically at all your creatives to work out the optimal level of spend between them.

The experiment

So, do you trust that Facebook knows best and leave it be, or do you intervene and start turning off the other creatives to force Facebook to give your high-performing ad more budget?

I decided to use Facebook's A/B testing feature to split an audience so that group A (control) contained all the existing creative (business as usual) and group B (test) contained just the top-performing creative.

Interestingly, after just one week group B (test) saw a 67% performance improvement vs. group A (control) and had a 95% confidence level, meaning that Facebook was confident that Group B (test) was the best performing strategy by a large margin.

Armed with this new insight, I paused the group A ad-set and kept the group B ad-set with the top-performing creative running. Performance has remained consistently higher within that ad-set since then.

In this instance, I was right to question the Facebook ad delivery AI. By intervening, I managed to generate significantly more revenue than I would have done had I not taken any action.

The learning

That said, I’m not suggesting you should always second guess the Facebook ad delivery AI and manually intervene to force spend into single high-performing creatives. This is unlikely to be a successful strategy, as I've always seen best performance where there are multiple creatives in each ad-set (ideally a combination of video and images).

However, if you do happen to see low spend on a very promising creative, then an option could be to run an A/B experiment – as described above – to know more concretely whether an intervention is necessary.

Have you come across this challenge before? What was your approach in understanding and overcoming it? Comment in the post to let me know!

Like learning about Growth Marketing?

Join the Growth Marketers Facebook group. It's a safe space for Growth and Digital Marketers to support and learn from one another. Members can share tips, strategies, experiences, learnings, or simply ask a question to the community around something they’d like to know more about.