Model Registry & Versioning

The question a model registry exists to answer

At 2am during an incident, the question that matters most is often deceptively simple: 'what model is actually running in production right now, what data and code produced it, and can we get back to the previous one in the next five minutes.' Without a system designed to answer this, teams end up reconstructing the answer from scattered sources — a Slack message about a deploy, a training script someone ran from their laptop, a model file sitting in a shared folder with an ambiguous filename. A model registry exists specifically to make that question answerable instantly and reliably, the same way a package registry or container registry does for software artifacts.

What gets tracked: lineage, not just a file

A registered model entry is more than a weights file — it's a record of lineage: which exact code commit trained it, which exact dataset snapshot it trained on, which hyperparameters were used, what its evaluation metrics were (including slice-based metrics, not just an aggregate score), who or what pipeline registered it, and when. This lineage is what makes a model auditable and debuggable months later — if a production issue surfaces, lineage lets you trace backward from 'this model is misbehaving' to 'this is exactly what produced it' without guessing.

Stages: the gates between training and real traffic

Registries typically organize model versions into stages such as staging (candidate, undergoing further validation — shadow traffic, A/B test, manual review) and production (currently serving real traffic). Promoting a model from staging to production is a deliberate, usually gated action — not something that happens automatically the moment training finishes — and the registry is what records exactly when and by whom that promotion happened. Older production versions typically move to an archived stage rather than being deleted, precisely so they remain available as a rollback target.

Without a model registry vs. with one

Rollback strategy beyond 'just revert'

A full rollback strategy is more nuanced than simply swapping the model artifact back. Teams need to decide: does the previous model's feature pipeline still match the current production data schema (if upstream data changed since that version was retired, a straight rollback might break)? Should rollback happen instantly for all traffic, or gradually, mirroring how the bad model was rolled out? And critically, rollback needs monitoring of its own — confirming the reverted model is actually behaving as expected again, not just assuming it is because it used to work. These are exactly the kinds of decisions that are easy to make calmly in a design review and hard to make well during an active incident, which is why they're worth deciding in advance rather than improvising.

What's next

A model registry tracks versions once a model is already trained — but keeping multiple models organized and comparable earlier in the process, during experimentation itself, is just as important. Revisiting ML Reproducibility Engineering and MLOps: Monitoring & Maintaining Models in Production together with this lesson completes the picture of how a model moves safely from a training run to a trustworthy production deployment.