Incrementality: the number, and what it hides
The conversions that happened because of the ad, rather than alongside it.
Why incrementality matters
It is the only measure that answers the question the business is actually asking. Attribution tells you what to credit; incrementality tells you what to keep paying for.
What this looks like with real numbers
Hold out 10% of a geography for four weeks. If the held-out region's revenue falls 3% while the treated region holds flat, roughly 3% of revenue was incremental to the advertising, not the 22% the platform attributed.
What it does not tell you
Measuring it properly requires withholding advertising from part of the audience, which costs money and nerve. Most accounts never do it, which is why most accounts overstate paid's contribution.
The mistake people actually make
Skipping the holdout because it costs revenue. Not measuring it costs more, and it costs it quietly.
Questions, answered.
How do you calculate incrementality?
Conversions with advertising − conversions without, measured by holdout. The conversions that happened because of the ad, rather than alongside it.
What goes wrong with incrementality?
Skipping the holdout because it costs revenue. Not measuring it costs more, and it costs it quietly.
What does incrementality not tell me?
Measuring it properly requires withholding advertising from part of the audience, which costs money and nerve. Most accounts never do it, which is why most accounts overstate paid's contribution.
Change the creative, not the dashboard
The conversions that happened because of the ad, rather than alongside it.