Counterfactual Explanations
The question people actually ask
When a loan application is denied or a resume is filtered out, the person affected rarely wants a feature-importance bar chart — they want to know: 'what would I need to change to get a different outcome?' A counterfactual explanation answers exactly that: it finds a new, minimally different input $x'$ such that the model's prediction for $x'$ flips to the desired class, e.g., 'if your annual income were $4,200 higher and everything else stayed the same, this application would have been approved.'
Formalizing 'minimal change'
Wachter et al. (2017) framed counterfactual search as an optimization problem: find $x'$ minimizing $\lambda \cdot (f(x') - y')^2 + d(x, x')$, where $f(x')$ is the model's prediction, $y'$ is the desired outcome, and $d(x, x')$ is a distance function penalizing how far $x'$ strays from the original input $x$. The first term pushes toward the target outcome; the second keeps the suggested change small and plausible. $\lambda$ trades off how strictly the target outcome must be hit versus how minimal the change must be.
import numpy as np
from scipy.optimize import minimize
def find_counterfactual(model_predict_proba, x, target_class=1, lam=1.0):
x = np.array(x, dtype=float)
def objective(x_prime):
pred = model_predict_proba(x_prime.reshape(1, -1))[0][target_class]
pred_loss = (pred - 1.0) ** 2 # want pred -> 1 (confidently target class)
distance = np.sum((x_prime - x) ** 2) # stay close to original input
return lam * pred_loss + distance
result = minimize(objective, x0=x, method="Nelder-Mead")
return result.x
# Usage with any sklearn-style classifier:
# cf = find_counterfactual(model.predict_proba, applicant_features, target_class=1)
# delta = cf - applicant_features # the "what changed" vector
Plausibility and actionability matter as much as minimality
Counterfactuals vs. SHAP: different questions entirely
SHAP and permutation importance explain *why* a model made a decision by attributing credit across existing features. Counterfactuals instead describe *what alternate world* would produce a different decision — they don't need to open the model's internals at all beyond querying predictions, making them applicable even to fully black-box, API-only models. Regulatory frameworks like GDPR's 'right to explanation' have specifically pointed toward counterfactual-style explanations because they're directly actionable for an affected individual in a way attribution scores are not.
| Explanation type | Answers | Needs model internals |
|---|---|---|
| SHAP / permutation importance | Why did the model predict this? | SHAP: helpful but not required; permutation: no |
| Partial dependence | How does prediction trend with a feature? | No — just predict/score access |
| Counterfactual | What minimal change flips the outcome? | No — black-box query access suffices |
What's next?
Counterfactuals are one specific technique within a much broader toolbox aimed at making opaque models answerable to the people affected by their decisions — a toolbox that spans everything from attribution to surrogate modeling to inherently transparent architectures, which is worth surveying as a field rather than technique by technique.
I build these systems professionally.
Whether it's a RAG pipeline, analytics migration, or AI workflow — let's talk.