Uplift Modeling

9 min read

Learn

Quick Reading

Estimated 9 mins

Prereq

Intermediate

Basic ML concepts helpful

Interactive

Static Playbook

Static guide & reference tables

The question uplift modeling answers

A standard predictive model answers: "how likely is this user to convert?" Uplift modeling answers a different, more actionable question: "how much does treating this user — sending them a promo, showing them an ad, calling them — change their probability of converting, compared to leaving them alone?" This quantity is the individual treatment effect (ITE): $\tau_i = Y_i(1) - Y_i(0)$, the difference between a unit's outcome under treatment and its outcome under control. We never observe both for the same person (the fundamental problem of causal inference), so uplift models estimate $\tau_i$ from a randomized or quasi-randomized experiment covering many units.

Why this matters more than response modeling

Imagine four types of customers: Persuadables (convert only if treated), Sure Things (convert regardless), Lost Causes (never convert regardless), and Sleeping Dogs (convert less often if treated — the intervention backfires). A response model that just predicts P(convert | treated) will rank Sure Things highest, because they have high conversion probability — but treating them wastes budget since they'd have converted anyway. Uplift modeling specifically targets Persuadables, where the marginal value of treatment is highest, and can even detect Sleeping Dogs to avoid making things worse.

Customer response segments

Segment	P(convert \| treated)	P(convert \| control)	Uplift	Should you treat?
Persuadables	High	Low	High positive	Yes — highest ROI
Sure Things	High	High	~Zero	No — wasted spend
Lost Causes	Low	Low	~Zero	No — won't convert anyway
Sleeping Dogs	Low	High	Negative	No — treatment backfires

Two-model approach and its weakness

The simplest uplift estimator trains two separate models: one on the treated group predicting $P(Y=1 | X, T=1)$, one on the control group predicting $P(Y=1 | X, T=0)$, then estimates uplift as $\hat{\tau}(x) = \hat{P}(Y=1|X=x, T=1) - \hat{P}(Y=1|X=x, T=0)$. This is easy to implement with any off-the-shelf classifier, but it has a critical flaw: each model is optimized to predict its own outcome well, not to predict the *difference* well. Small, correlated errors in both models can produce a noisy, biased uplift estimate, especially when the true uplift signal is much smaller than the baseline conversion rate — which is the usual case.

Class transformation and single-model approaches

A more robust family of methods models uplift directly in a single model. The class transformation trick (Lai's method / the modified outcome method) relabels the target as $Z_i = Y_i \cdot T_i + (1-Y_i)(1-T_i)$ for a randomized 50/50 split, so that $E[Z|X] = \frac{\tau(X) + 1}{2}$ — a single classifier on $Z$ directly recovers a monotonic transform of uplift. Uplift trees (Rzepakowski and Jaroszewicz) go further, modifying the splitting criterion of a decision tree to maximize the *divergence* in uplift between child nodes — such as the Kullback-Leibler or Euclidean divergence between the treatment and control outcome distributions — rather than maximizing information gain for a single outcome. The causal forest (Wager and Athey) extends random forests similarly, building trees that split to maximize heterogeneity in estimated treatment effect, with honest splitting to keep effect estimates unbiased.

uplift_two_model_and_qini.py

python

import numpy as np
from sklearn.ensemble import GradientBoostingClassifier
from sklearn.model_selection import train_test_split

rng = np.random.default_rng(0)
n = 20000
X = rng.normal(size=(n, 5))
treat = rng.binomial(1, 0.5, n)

# True uplift depends on X[:,0]; baseline depends on X[:,1]
baseline = 1 / (1 + np.exp(-(0.5 * X[:, 1])))
true_uplift = 0.15 * (X[:, 0] > 0)
p_outcome = np.clip(baseline + treat * true_uplift, 0.01, 0.99)
y = rng.binomial(1, p_outcome)

X_train, X_test, t_train, t_test, y_train, y_test = train_test_split(
    X, treat, y, test_size=0.3, random_state=1
)

# Two-model approach
model_treat = GradientBoostingClassifier().fit(X_train[t_train == 1], y_train[t_train == 1])
model_ctrl = GradientBoostingClassifier().fit(X_train[t_train == 0], y_train[t_train == 0])

uplift_hat = model_treat.predict_proba(X_test)[:, 1] - model_ctrl.predict_proba(X_test)[:, 1]

def qini_curve(uplift_scores, treatment, outcome):
    order = np.argsort(-uplift_scores)
    t_sorted, y_sorted = treatment[order], outcome[order]
    cum_treat_resp = np.cumsum(y_sorted * t_sorted)
    cum_ctrl_resp = np.cumsum(y_sorted * (1 - t_sorted))
    n_treat = np.cumsum(t_sorted)
    n_ctrl = np.cumsum(1 - t_sorted)
    ratio = np.where(n_ctrl > 0, n_treat / np.maximum(n_ctrl, 1), 0)
    return cum_treat_resp - cum_ctrl_resp * ratio

qini = qini_curve(uplift_hat, t_test, y_test)
print("Qini curve, first/last values:", qini[0], qini[-1])

Evaluate with Qini and uplift curves, not accuracy

You cannot score an uplift model with standard classification metrics because true individual uplift is never observed. The standard evaluation is the Qini curve: sort units by predicted uplift descending, then at each cutoff compute (cumulative treated responders) minus (cumulative control responders, rescaled by the ratio of treated to control group sizes). A good uplift model produces a curve that rises well above the diagonal random-targeting baseline. The Qini coefficient (area between the curve and the diagonal) is the single-number summary used to compare models.

Uplift modeling needs randomized or quasi-randomized treatment

If treatment assignment in your training data was not random (e.g., you only ever targeted customers your old model deemed promising), any uplift model you train will inherit that selection bias, and its estimates will be unreliable precisely in the regions you care about most. Uplift modeling generally requires a held-out randomized control group — often a deliberately 'untreated' slice of traffic — to produce trustworthy individual-level treatment effect estimates.

What's Next?

Uplift modeling tells you who to target once you already have a treatment and a randomized or near-randomized rollout to learn from. The natural next question is how to design and size that rollout correctly in the first place — how many units you need in treatment and control to detect an effect of the size you care about, and how to think about the statistical power of the experiment before you run it.

I build these systems professionally.

Whether it's a RAG pipeline, analytics migration, or AI workflow — let's talk.

Work With Me →