Causal Inference Fundamentals
Correlation is not causation — but what exactly is causation?
Every statistics course repeats 'correlation is not causation,' but few give you the machinery to tell them apart. Causal inference is that machinery: a formal framework for asking and answering questions of the form 'what would have happened if X had been different,' using data where X wasn't actually randomized.
The field matters enormously for machine learning in production. A model that predicts churn well might be riding on a feature that's merely correlated with churn (say, 'number of support tickets') rather than causing it — and if you intervene on that feature (reduce support tickets by making it harder to contact support), churn won't actually go down. Causal inference is what separates 'useful predictor' from 'useful lever.'
Potential outcomes and the fundamental problem
The dominant formalism, due to Jerzy Neyman and later developed by Donald Rubin, is the potential outcomes framework. For each unit $i$ (a person, a customer, a patient) and each treatment level, imagine a potential outcome: $Y_i(1)$ is what would happen if unit $i$ received treatment, and $Y_i(0)$ is what would happen if it didn't. The individual causal effect is $Y_i(1) - Y_i(0)$.
The catch, known as the fundamental problem of causal inference: for any given unit, you only ever observe one of these two outcomes — whichever treatment they actually received. You never see both the treated and untreated version of the same person at the same time. All of causal inference is, in some sense, a set of clever strategies for estimating what the unseen potential outcome would have been, using information from other units.
Confounding: why naive comparisons mislead
The reason we can't just compare the average outcome of treated units to untreated units is confounding: a variable that affects both who gets treated and the outcome itself. Classic example: people who exercise more (treatment) tend to be healthier (outcome) — but people who exercise more also tend to be younger and wealthier, both of which independently predict better health. A naive comparison conflates the true effect of exercise with the effect of being young and wealthy.
Formally, if $Z$ is a confounder, then $Z$ causes both treatment assignment and $Y$, creating a 'backdoor path' between treatment and outcome that has nothing to do with the treatment's actual causal effect. Randomized controlled trials (RCTs) solve confounding by construction — if treatment is assigned by a coin flip, it can't be correlated with anything, confounders included. Most of causal inference outside of RCTs is about approximating what randomization would have given you.
Z (confounder: e.g. income)
/ \
v v
T (treatment: e.g. ad exposure) --> Y (outcome: e.g. purchase)
The backdoor path T <- Z -> Y means a raw comparison of Y across
levels of T mixes the true causal effect T -> Y with the spurious
Z -> Y pathway. "Adjusting for Z" (e.g. stratifying, regression,
matching, or weighting) blocks this backdoor path.Three identification strategies, one goal
The Average Treatment Effect and its variants
Since individual causal effects are unobservable, causal inference usually targets population-level quantities. The Average Treatment Effect (ATE) is $E[Y(1) - Y(0)]$ averaged over everyone. The Average Treatment Effect on the Treated (ATT) restricts this average to units that actually received treatment — often the more policy-relevant quantity, e.g. 'how much did this subsidy help the people who took it,' as opposed to 'how much would it help a random person.' The Conditional Average Treatment Effect (CATE), $E[Y(1) - Y(0) \mid X = x]$, lets the effect vary by covariates — the basis of modern heterogeneous-treatment-effect and uplift modeling work.
import pandas as pd
import statsmodels.formula.api as smf
# df has columns: outcome, treatment, and confounders age, income
df = pd.read_csv("observational_data.csv")
# Naive (likely biased) comparison
naive_effect = df.loc[df.treatment == 1, "outcome"].mean() - \
df.loc[df.treatment == 0, "outcome"].mean()
# Regression adjustment: control for observed confounders
model = smf.ols("outcome ~ treatment + age + income", data=df).fit()
print(model.summary())
adjusted_effect = model.params["treatment"]
print(f"Naive diff: {naive_effect:.3f}, Adjusted ATE: {adjusted_effect:.3f}")
# If these differ substantially, confounding by age/income was real.Adjustment only works for observed, correctly specified confounders
What's next?
These fundamentals — potential outcomes, confounding, DAGs, and the ATE/ATT/CATE distinctions — are the vocabulary the rest of causal inference is built from. The next step is turning identification strategies into concrete estimators: matching similar units on observed covariates is one of the most intuitive and widely used, and a natural place to go deeper.
I build these systems professionally.
Whether it's a RAG pipeline, analytics migration, or AI workflow — let's talk.