A Causal Inference Analysis That Changed a Real Budget Decision

A client's marketing dashboard showed one paid channel with a consistently strong ROAS, and the budget had been growing toward it for two quarters based on that number alone. The dashboard was doing simple correlational attribution — if a customer saw an ad on that channel before converting, the channel got credit. The number was real. Whether it meant what everyone assumed it meant was a different question entirely.

What the causal analysis actually tested

We ran a geo-based causal analysis — comparing matched regions with and without spend on that channel over a defined period, rather than just attributing conversions to whoever showed an ad before them. The result was uncomfortable: a large share of the 'conversions' the dashboard was crediting to that channel were customers who would have converted anyway, through organic search or direct traffic, and the channel was simply good at targeting people who were already about to buy.

The dashboard wasn't lying. It was answering a different question than the one the budget decision actually needed answered.

The channel's true incremental lift, once you control for who would have converted regardless, was meaningfully lower than its reported ROAS implied — while a different channel the dashboard had been underselling turned out to be driving conversions that genuinely wouldn't have happened otherwise.

• Correlational attribution answers 'who converted after seeing this ad,' not 'who converted because of this ad' — these are frequently different answers.
• A geo holdout or matched-market test gives a far more honest read on incremental lift than any attribution dashboard.
• Channels that target already-high-intent customers will always look artificially strong in correlational attribution.
• Budget decisions built on correlational metrics alone are, more often than people assume, rewarding the wrong channel.

The client reallocated a meaningful chunk of budget based on the causal results, away from the channel the dashboard had been praising. It's an uncomfortable reallocation to make when a familiar number has been driving decisions for two quarters, but the incremental revenue numbers after the shift backed it up within the next quarter.