Most machine learning problems assume a roughly stable relationship between features and labels: the pattern that predicted churn last year still predicts churn this year. Fraud detection breaks that assumption, because the entity generating the "positive" class — fraudsters — is actively adapting to whatever defense is deployed against them. A fraud model isn't predicting a fixed natural phenomenon; it's playing an ongoing adversarial game where the other side studies the model's blind spots and routes around them.

This single property shapes almost everything distinctive about fraud systems: why they need frequent retraining, why they combine multiple models and rules rather than relying on one, and why a model that performs excellently in backtesting can degrade within weeks of deployment as fraud patterns shift in response to the very model now screening them.

Feature engineering: fraud lives in behavior, not single transactions

A single transaction in isolation often doesn't reveal fraud — a $40 purchase looks ordinary no matter who makes it. The signal usually lives in behavioral and contextual features built from history and context around the transaction: velocity features (how many transactions on this account in the last hour, compared to this account's own typical rate), deviation features (is this purchase amount, merchant category, or location unusual *for this specific user*, not unusual in general), network features (does this device, IP address, or shipping address link to other accounts previously flagged as fraudulent), and sequence features (does the order of recent actions — new device added, password changed, large transaction attempted — match a known fraud playbook).

This is why fraud feature pipelines tend to be unusually heavy on real-time aggregation infrastructure: computing "transactions in the last 10 minutes" or "distance from this user's last known location" requires maintaining fast-access rolling state per account, not just reading static profile fields.

Note

In many fraud contexts, well under 1% of transactions are actually fraudulent — sometimes far less. A model that simply predicts "not fraud" for every transaction can achieve 99%+ raw accuracy while catching zero fraud, which makes accuracy a nearly useless metric for this problem. Fraud teams instead optimize for precision and recall at specific, carefully chosen operating thresholds, use resampling or class-weighting techniques during training to keep the minority class from being ignored, and evaluate with precision-recall curves rather than the ROC curves and accuracy numbers that work fine for more balanced problems.

Metric choice under extreme class imbalance

Metric	What it measures	Why it's misleading or useful here
Raw accuracy	Fraction of all predictions correct	Misleading — trivially high by predicting the majority class
Precision	Of flagged transactions, what fraction are truly fraud	Directly tied to false-positive cost (blocking good customers)
Recall	Of actual fraud, what fraction was caught	Directly tied to losses from missed fraud
Precision-recall AUC	Overall tradeoff quality across thresholds	Far more informative than ROC-AUC under heavy imbalance

Real-time constraints shape the architecture

A card-payment fraud model often has a latency budget measured in tens of milliseconds, because the scoring decision has to complete before the payment authorization can proceed — users won't tolerate a noticeable delay at checkout. This hard constraint rules out some otherwise-attractive modeling choices: a very large ensemble or a heavy deep model might score marginally better offline but be unusable if it can't return a score within the latency budget at the required request volume.

Production fraud systems typically layer defenses rather than relying on one model: fast, cheap rule-based checks or a lightweight model handle the bulk of traffic in real time with a strict latency budget, flagging a smaller subset of ambiguous cases for a heavier model or even human review that can take longer, since only a small fraction of transactions need that deeper scrutiny. This tiered design lets the system meet real-time constraints on average while still applying more computation where it's actually needed.

text

Adversarial adaptation: the model's own success degrades it

Once a fraud model is deployed, successful fraudsters who get caught (or who probe the system and notice certain patterns get blocked) adapt their behavior to avoid the signals that triggered detection — splitting large transactions into several smaller ones to dodge velocity checks, rotating through new devices and IP addresses to avoid network-based flags, or mimicking normal user behavior more closely before attempting the fraudulent action. This means a fraud model's performance decays over time even without any change in the underlying data distribution in a statistical sense — the decay is driven by intelligent adversaries specifically reacting to the model.

This is fundamentally different from ordinary model drift caused by, say, seasonal shopping patterns changing gradually. It requires continuous retraining on recent data, active monitoring for emerging fraud patterns that don't match historical labels yet, and often a human fraud-analysis team whose job is explicitly to spot new attack patterns before enough labeled examples exist for the model to learn them automatically.

Note

A transaction isn't definitively labeled as fraud until a chargeback is filed or a customer reports it, which can take weeks after the transaction itself — meaning recent transactions in a training set are systematically under-labeled simply because not enough time has passed for the true label to arrive. This label latency has to be accounted for explicitly in how training data is assembled, or the model ends up trained on an artificially fraud-light recent window that looks safer than it actually is.

What's next

Fraud detection shares its class-imbalance and real-time scoring challenges with credit scoring — another high-stakes financial ML application, but one shaped more by regulatory fairness requirements than by an actively adversarial population, which is where the next lesson picks up.

I build these systems professionally.

Whether it's a RAG pipeline, analytics migration, or AI workflow — let's talk.

Work With Me →

Need production AI or MarTech expertise? I build custom LLM workflows, server-side GTM tagging, and analytics infrastructures professionally.

Need custom AI or MarTech setup? Let's build together.

Let's talk →