A Geo-Lift Test That Surprised Us More Than the Campaign Did

The platform dashboard said the campaign was driving a 22% lift in sign-ups. The geo-lift test said closer to 4%. Nobody on the team believed the geo-lift number at first, including me, because the platform number had months of consistency behind it. The geo-lift test turned out to be right, and the reason was almost comically simple.

What the test actually measured

We held out a set of matched DMAs from the campaign for five weeks and compared sign-up trends against the active DMAs, controlling for the pre-period baseline. The 4% result was consistent across two re-runs with different holdout splits, which is usually a sign you've found something real rather than noise.

Platform-reported lift measures what the platform's model attributes to itself. A geo-lift test measures what would have happened without it. Those are not the same question.

Why the gap was so big

The campaign was retargeting an audience that was already converting at a high baseline rate — people who'd visited the pricing page in the last 14 days. The platform's attribution model saw the ad, saw the conversion, and credited itself, without any way to know that a large share of that audience would have signed up anyway. The geo-lift test, by comparing whole regions rather than individual users, naturally captured that baseline behavior and netted it out.

• Heavily retargeted, high-intent audiences are the exact scenario where platform attribution overstates lift the most.
• A geo-lift test on a narrow retargeting audience is harder to run cleanly, but it's precisely where you need it most.
• The 4% wasn't nothing — it justified keeping the campaign, just at a much smaller budget than the 22% number had justified.
• We re-ran the same test methodology on a prospecting campaign afterward and the platform and geo-lift numbers were much closer, which itself was useful confirmation.

The real finding wasn't the specific percentage. It was learning exactly which type of campaign the platform's attribution lies to you about most, which turned out to be the deep-retargeting end of the funnel — and now we test that category first on every new client.