Outlier Detection Techniques

9 min read

Learn

Quick Reading

Estimated 9 mins

Prereq

Intermediate

Basic ML concepts helpful

Interactive

Static Playbook

Static guide & reference tables

Outliers aren't always errors — and that distinction matters

An outlier is a data point that deviates substantially from the rest of the dataset, but *why* it deviates matters for what you do next. Some outliers are data entry errors or sensor glitches (a negative age, a temperature reading of 9999) that should be corrected or removed. Others are genuine, rare events — a fraudulent transaction, a catastrophic equipment failure — that are often precisely the thing you're trying to detect, and removing them would destroy the signal you care about. Outlier detection techniques are the same regardless of intent, but what you do with a flagged point depends entirely on which case you're in.

Statistical methods: Z-score and IQR

The simplest approach flags points beyond a threshold number of standard deviations from the mean: $z = \frac{x - \mu}{\sigma}$, commonly flagging $|z| > 3$. This assumes roughly Gaussian data and is itself distorted by the very outliers it's trying to detect, since $\mu$ and $\sigma$ are both outlier-sensitive. The IQR method is more robust: compute the interquartile range $IQR = Q_3 - Q_1$, and flag points outside $[Q_1 - 1.5 \times IQR, \; Q_3 + 1.5 \times IQR]$ — the same convention used to draw box-plot whiskers. Because quartiles are far less influenced by extreme values than mean/std, IQR-based flagging degrades more gracefully as outlier severity increases.

outlier_methods.py

python

import numpy as np
from sklearn.ensemble import IsolationForest
from sklearn.neighbors import LocalOutlierFactor

def iqr_outliers(x, k=1.5):
    q1, q3 = np.percentile(x, [25, 75])
    iqr = q3 - q1
    lower, upper = q1 - k * iqr, q3 + k * iqr
    return (x < lower) | (x > upper)

# Isolation Forest: isolates points that need fewer random splits to separate
# (anomalies are "easier" to isolate because they're sparse and distinct)
iso = IsolationForest(contamination=0.02, random_state=0)
preds = iso.fit_predict(X)   # -1 = outlier, 1 = inlier

# Local Outlier Factor: flags points whose local density is much lower
# than their neighbors' -- good for outliers that are only "weird" locally
lof = LocalOutlierFactor(n_neighbors=20, contamination=0.02)
lof_preds = lof.fit_predict(X)

Isolation Forest: anomalies are easy to isolate

Isolation Forest (Liu, Ting & Zhou, 2008) takes a fundamentally different approach: rather than modeling what 'normal' looks like, it exploits the fact that anomalies are few and different, so they get separated from the rest of the data in fewer random partition splits than typical points. The algorithm builds many random trees that recursively split the feature space, and a point's anomaly score is based on the average path length needed to isolate it — short paths indicate anomalies. This scales well to high dimensions and doesn't require assuming any particular data distribution, unlike Gaussian-based statistical methods.

Local Outlier Factor: density relative to neighbors

Local Outlier Factor (LOF) compares a point's local density to the local density of its neighbors, rather than to the dataset as a whole. This matters when 'normal' isn't uniform across the whole feature space — a point can be a clear outlier relative to its local cluster while still falling within the global range of the data (e.g., a transaction that's unusually large *for that specific customer* but unremarkable compared to the dataset's overall transaction sizes). LOF catches this kind of context-dependent anomaly that global statistical methods and even Isolation Forest can miss.

Method	Assumes distribution?	Sensitive to local density variation?	Scales to high dimensions?
Z-score	Yes (Gaussian)	No	Poorly
IQR	No, but assumes unimodal	No	Poorly
Isolation Forest	No	Partially	Well
Local Outlier Factor	No	Yes (by design)	Moderately

Removing outliers can leak your target if done carelessly

Filtering outliers based on the target variable itself (e.g. dropping unusually high sale prices before training a price-prediction model) can quietly bias the model toward underestimating rare-but-real extreme cases, and if done using statistics computed across train+test, it also introduces the same kind of leakage discussed for scaling and imputation. Outlier handling decisions should be based on feature distributions understood from domain knowledge and training data only, applied consistently before any split-dependent statistic is computed.

Contamination rate is a real hyperparameter, not a guess

Both Isolation Forest and LOF require specifying an expected *contamination* rate — the approximate fraction of the dataset you expect to be anomalous. Setting this wrong systematically changes what gets flagged: too high, you flag normal points as anomalies; too low, you miss real ones. In production anomaly detection, this is often set via domain knowledge (known historical fraud rate) rather than cross-validation, since true positive outlier labels are frequently scarce or unavailable.

What's Next?

Outlier detection is closely related to, but distinct from, how you split data for training and evaluation — a poorly chosen split can make outliers (or their absence) create a misleading picture of how well a model will actually perform.

I build these systems professionally.

Whether it's a RAG pipeline, analytics migration, or AI workflow — let's talk.

Work With Me →