Marginal Return v. Average Return

At this point in the paid media world, most programs and agencies have good ideas on how to avoid basic errors. Stuff like budget capping in search, avoiding LI audience network, aggregating data in Facebook to get out of learning phase, etc., etc. There are a million simple critical things to keep an eye on. Within the last 5 years, more advanced topics like understanding incrementally by using holdout tests are becoming more widespread. I would argue that true incremental measurement is still criminally under-adopted but that is another post. What I want to talk about is one error that I see in virtually every account that I audit, not optimizing toward a marginal return.

Optimizing toward average CPA, not marginal CPA, is a massive problem!

This is a tricky one.  Most of the time when you are given a goal of $10 CPA people will bid up/invest more until you hit those $10 dollars and then stop pushing. The problem is that half of those conversions are under $10 and half are over $10. Really you want to push up until the margin CPA equals your goal.  Here is a visualization of the different ways to look at the exact same performance. Left is average CPA, right is marginal.

Why is it difficult?

It means that in some cases you would invest in a channel/campaign/KW that has a higher average CPA over a lower one and this is fundamentally counterintuitive.  The easiest way to find marginal performance is to slice your data by time but you can also create split tests with different bids to get cleaner results. Let’s take a look at some real data that is sorted by spend to show the relationship between cost and conversions. Our goal is $5 CPA, and if you look at average CPA this campaign is doing great until the very highest spend. We expect efficiency to drop as we scale so usually what happens is we’ll stop somewhere around 3k.

BUT if we are only looking at the average CPA we are forgetting the marginal CPA. The next steps is to graph the data and match it to a model. In this case we are looking at 303*LN(Spend)) - 1755 = expected Conversions, I just use excel to find the equation. From there we can calculate the cost of the incremental conversions every time we make a budget change. When we do this we see that most of our conversions are higher than our goal .

OK so maybe you are saying “Fine Jay, perhaps some of those conversions are a little more expensive than I would like but I am cool with the average CPA and we just need to get as many conversions as posssible.” Well fine but let me show you something else. Same situation where our goal is $5 CPA. Look at these two campaigns. The first one is our friend from above, second is a new poor performing campaign. Since the very worse performance in our good campaign is better than the best of #2 it might be tempting to never turn on Campaign 2, but since we know that marginal CPA is going to go up as we spend more we need to make sure we are tracking the right data. Let’s see what the right distribution of budget across these two campaigns if we have $5,000. If we put everything in Campaign 1 we would expect to get 829 conversions at a $6.03 CPA. Please note that this number is better than the starting point of Campaign 2.

To figure this out we are going to need a little modeling and a bit of calculus.

Starting Equations  

Campaign 1 Model

303*LN(X1)) – 175

Campaign 2 Model

50*LN(X2) - 236.04

Budget Constraint

X1 + X2 =$5000

Take the derivatives of both equations.

Campaign 1

303/X1  

Campaign 2

50/X2

Set the derivatives equal to each other and substitute X2 using X2 =$5000 - X1. This makes sure that our marginal CPAs are always going to be the same in both campaigns.  If we only looked at only average CPA we would not be able to see the point where Campaign 2 is a better investment.

303/X1 = 50/(5000-X1)

303(5000-X1) = 50X1

1,51,5000 – 303X1 = 50X1

1,515,000 = 353X1

X1 = $4,291, Campaign One Budget

X2 = $709 Campaign Two Budget

When we use this distribution we move the final conversions from 829 to 876.

And there is your budget distribution to maximize conversions for $5,000 in these two campaigns. Now the real goal to think about this across the entirety of your program. You will end up with something that looks like below. For extra credit also incorporate LTV and a true incremental lift modifier.