Lambda Architecture for ML Systems
Two conflicting demands: speed and correctness
Production ML systems face a tension: users want predictions that reflect very recent data (low latency, 'what just happened'), but the most accurate, reconciled view of data often only exists after batch processing completes (deduplication, late-arriving corrections, joins against slowly-updated reference tables). Lambda architecture, originally articulated by Nathan Marz for big-data systems, resolves this by running two parallel processing paths over the same raw data and reconciling their outputs — rather than forcing one engine to do both jobs adequately.
The three layers
Batch layer: the full, immutable event log is reprocessed periodically (hourly, daily) to produce a complete, accurate, recomputed view — feature tables, aggregates, retrained models. It's slow but authoritative; if a bug is found, you can always recompute from the raw log. Speed layer: a stream processor handles only recent events (e.g., the last few minutes/hours not yet covered by the latest batch run) to provide a low-latency, approximate view that fills the gap. Serving layer: merges the batch view and the speed layer's incremental view at query time, so a request for 'current feature value' returns batch-layer accuracy for everything older than the last batch run, plus speed-layer freshness for anything since.
| Layer | Latency | Accuracy | Failure recovery |
|---|---|---|---|
| Batch | Minutes to hours | High — full reprocessing, corrections included | Trivial — rerun from immutable log |
| Speed | Milliseconds to seconds | Approximate — may miss late data/corrections | Harder — in-flight state can be lost |
| Serving (merged) | Low (reads from both) | Converges to batch accuracy over time | Depends on both layers |
def get_feature(user_id, feature_name, now):
"""Serving-layer read: merge authoritative batch value with speed-layer delta."""
batch_value, batch_as_of = batch_store.get(user_id, feature_name)
# speed layer only holds events AFTER the batch's cutoff
delta_events = speed_store.get_events_since(user_id, feature_name, batch_as_of)
if feature_name == "transaction_count_30d":
return batch_value + len(delta_events)
elif feature_name == "last_seen_ts":
return max([batch_value] + [e.ts for e in delta_events])
else:
raise ValueError(f"No merge rule defined for {feature_name}")
# As soon as the next batch run completes, batch_as_of advances and the
# speed layer's window of responsibility shrinks back down — bounding
# how much approximate data can ever accumulate.Why not just use one engine for everything?
The obvious alternative — the Kappa architecture — runs everything through the stream processor alone, treating batch recomputation as 'just replay the log from the beginning through the same streaming code.' This eliminates the duplicated logic between batch and speed layers (a major source of the training-serving skew discussed in streaming pipeline design) and is the preferred approach when the stream processing engine can genuinely handle full historical reprocessing efficiently (modern Flink and Kafka Streams increasingly can). Lambda architecture remains preferred when batch and streaming genuinely need different algorithms — e.g., a batch layer running a complex iterative join or a full model retrain that has no sane streaming equivalent, while the speed layer only needs simple incremental counters.
The real cost of Lambda is maintaining two codebases
Late-arriving data breaks the clean mental model
What's Next?
These architectural patterns apply across data types, but text data in particular — unstructured, high-dimensional, and requiring its own feature engineering vocabulary — deserves dedicated treatment.
I build these systems professionally.
Whether it's a RAG pipeline, analytics migration, or AI workflow — let's talk.