Explainable AI (XAI) Overview
Why XAI became its own field
As machine learning moved from academic benchmarks into lending, hiring, medical diagnosis, and criminal justice, 'the model said so' stopped being an acceptable answer — both ethically and, increasingly, legally (GDPR's right to explanation, the EU AI Act's transparency requirements for high-risk systems). Explainable AI (XAI) is the umbrella field covering every technique for making a model's behavior understandable to humans, spanning wildly different approaches depending on *who* needs the explanation and *why*.
Two axes that organize the whole field
Almost every XAI technique can be placed on two axes. Intrinsic vs. post-hoc: intrinsic interpretability comes from using a model simple enough to understand directly (linear regression coefficients, shallow decision trees); post-hoc explanation applies a separate technique *after* training a complex model to approximate or probe its behavior (SHAP, LIME, counterfactuals). Global vs. local: global explanations describe overall model behavior across the whole input space (feature importance, partial dependence); local explanations describe one specific prediction (SHAP values for a single row, a single counterfactual). Most of the techniques covered elsewhere in this track — SHAP, permutation importance, PDPs, counterfactuals — are post-hoc; they differ mainly in whether they're global or local.
| Technique | Intrinsic or post-hoc | Global or local |
|---|---|---|
| Linear regression coefficients | Intrinsic | Global |
| Decision tree structure | Intrinsic | Global (and traceable per-instance) |
| Permutation / MDI importance | Post-hoc | Global |
| Partial dependence plots | Post-hoc | Global |
| SHAP values | Post-hoc | Local (aggregable to global) |
| Counterfactual explanations | Post-hoc | Local |
LIME: a model-agnostic local surrogate approach
LIME (Local Interpretable Model-agnostic Explanations, Ribeiro et al. 2016) explains a single prediction by sampling perturbed versions of the input nearby, getting the black-box model's predictions for them, and fitting a simple, interpretable model (usually sparse linear regression) to approximate the black box *locally*, in the neighborhood of that one instance. Unlike SHAP, LIME has no strong theoretical fairness guarantees, but it's conceptually simple, works for any model including images and text, and was historically important in popularizing post-hoc local explanation as a practical technique.
def choose_xai_technique(question, audience, model_type):
"""A rough decision guide, not a library call."""
if "why this one decision" in question:
if model_type == "tree_ensemble":
return "SHAP (TreeSHAP) -- exact, fast, local"
return "LIME -- model-agnostic local surrogate"
if "what feature matters overall" in question:
return "permutation importance (held-out set)"
if "what would change the outcome" in question and audience == "affected_individual":
return "counterfactual explanation"
if audience == "regulator_or_auditor" and "must be inherently transparent" in question:
return "use an intrinsically interpretable model (linear, shallow tree, GAM)"
return "partial dependence plot -- for general trend understanding"
Explainability is not the same as trustworthiness
What's next?
The intrinsic/post-hoc distinction raises an obvious practical question many teams face directly: when is it worth sacrificing some raw predictive accuracy to use a model that's transparent by construction, rather than a black box paired with post-hoc explanations layered on top?
I build these systems professionally.
Whether it's a RAG pipeline, analytics migration, or AI workflow — let's talk.