Back to notes

January 28, 2026 · 5 min read

An API Versioning Mistake That Broke a Client Integration Silently

We shipped a 'non-breaking' API change that broke a client's integration for nine days before anyone noticed, because our definition of non-breaking didn't match how their code actually parsed our response.

PostmortemEngineering

We added a new optional field to an API response and changed one existing field's value from a string to a nested object with a label and value key, reasoning that since the new structure still had a .toString()-friendly shape and no required fields were removed, it counted as additive and non-breaking under our own versioning policy. A client's integration broke silently the day it deployed and stayed broken for nine days before anyone on either side noticed.

Where our definition of 'non-breaking' diverged from reality

Our policy defined breaking changes narrowly: removing a field, renaming a field, or changing a field's type from one primitive to another. Changing a string field into an object wasn't technically covered by that last clause the way we'd written it, because we read it as being about primitive-to-primitive changes specifically. The client's code did response.status.toUpperCase(), assuming status was always a string. It became an object. .toUpperCase() on an object doesn't throw — it just returns garbage silently, because JavaScript coerces it into a string representation that passed their minimal validation but was meaningless downstream.

A breaking change doesn't need to throw an error to be breaking. It just needs to violate an assumption someone's code was quietly built on.

How it finally got caught

Not by any monitoring on our side — we had no visibility into how clients' code consumed our response shapes. The client's own customer support team noticed a cosmetic bug (a status label rendering as '[object Object]'.toUpperCase() nonsense) in their UI and traced it back nine days later. Nine days of a visibly broken field in production, on our change, that neither side's monitoring caught.

  • Rewrote our breaking-change policy to explicitly cover any change to an existing field's shape, type, or structure — not just primitive-to-primitive type swaps.
  • Added a contract test suite that snapshots the exact JSON shape of every documented response and fails CI on any unannounced structural change, including nested object changes.
  • Started shipping genuinely additive-only changes as new field names rather than mutating existing fields in place, with deprecation windows for anything that needs to change shape.
  • Added a changelog and a required major-version bump for any field structure change, even ones we'd previously have called 'basically non-breaking.'

The actual fix to the client's code took an hour once found. The nine days of silent breakage happened entirely because 'non-breaking' was defined from our side of the API, based on what we thought was safe to assume about how it'd be consumed, instead of being defined by what consumers actually depend on. Those are not the same definition, and the gap between them is exactly where this kind of bug lives.