Partial Dependence Plots
Visualizing the marginal effect of one feature
A Partial Dependence Plot (PDP) shows how a model's average predicted output changes as one feature varies, while marginalizing out the effect of all other features. It answers: 'holding everything else about the data distribution fixed, what does the model's prediction look like as a function of just this one feature?' This is the natural next step after a global importance ranking (which feature matters) or a SHAP summary (how much, per instance) — a PDP shows the actual *shape* of the relationship: linear, monotonic, U-shaped, or flat beyond a threshold.
How it's computed
For a feature $x_S$ of interest and the remaining features $x_C$, the partial dependence function is defined as $\hat{f}_S(x_S) = \frac{1}{n}\sum_{i=1}^n \hat{f}(x_S, x_C^{(i)})$ — fix $x_S$ to a specific value, keep every other feature at its *actual* value from each training row, predict, and average across all rows. Repeating this for a grid of values of $x_S$ traces out the curve. In practice, this means re-running the model $n$ times (once per training row) for every single grid point, which is why PDPs are computationally more expensive than permutation importance for large datasets.
from sklearn.inspection import PartialDependenceDisplay
from sklearn.ensemble import GradientBoostingRegressor
from sklearn.datasets import fetch_california_housing
import matplotlib.pyplot as plt
data = fetch_california_housing()
X, y = data.data, data.target
model = GradientBoostingRegressor(n_estimators=200, max_depth=3).fit(X, y)
# Feature index 0 = MedInc (median income)
PartialDependenceDisplay.from_estimator(model, X, features=[0], feature_names=data.feature_names)
plt.savefig("pdp_medinc.png")
# The curve shows how predicted house value changes as income varies,
# averaged over the realistic distribution of every other feature.
PDPs can be misleading under correlated features
2D PDPs reveal interactions
Extending the same idea to two features at once produces a 2D partial dependence surface, which can reveal interaction effects a 1D PDP would miss entirely — for example, a feature that only matters when another feature exceeds some threshold. This is directly related to Friedman's H-statistic, which quantifies interaction strength by measuring how much the 2D partial dependence deviates from what you'd expect if the two features' effects were purely additive.
| Method | Shows | Risk with correlated features |
|---|---|---|
| PDP | Average marginal effect, one/two features | Can extrapolate into unrealistic combinations |
| ICE | Per-instance curves (PDP's building blocks) | Reveals heterogeneity PDP averages away |
| SHAP dependence plot | Per-instance SHAP value vs. feature value | More robust, but interpretation differs from PDP |
What's next?
PDPs require access to the model itself to repeatedly re-predict across a grid. A simpler, model-agnostic question — how much does shuffling a feature's values hurt overall performance — doesn't need that grid-based machinery at all, and is where permutation-based importance measures come from.
I build these systems professionally.
Whether it's a RAG pipeline, analytics migration, or AI workflow — let's talk.