A/B test: the number, and what it hides
A comparison where one variable changes and everything else is held constant.
Why a/b test matters
It is the difference between learning and guessing. When only the hook differs, a winner tells you something that transfers to the next ad.
What this looks like with real numbers
Four ads, same footage, same product, same audience, same budget, and the only difference is the opening line. One returns a $38 CPA against the others at $55 to $61. That result transfers: you now know something about openings that applies to the next ten ads.
What it does not tell you
Most creative 'tests' change the hook, the edit, the length and the offer at once. Something wins and nothing is learned, so the next round is as blind as the last.
The mistake people actually make
Changing the hook, the edit, the music and the length at once. Something wins, nothing is learned, and the next round starts from zero.
Questions, answered.
How do you calculate a/b test?
Same audience, same budget, same placements, one difference. A comparison where one variable changes and everything else is held constant.
What goes wrong with a/b test?
Changing the hook, the edit, the music and the length at once. Something wins, nothing is learned, and the next round starts from zero.
What does a/b test not tell me?
Most creative 'tests' change the hook, the edit, the length and the offer at once. Something wins and nothing is learned, so the next round is as blind as the last.
Change the creative, not the dashboard
A comparison where one variable changes and everything else is held constant.