LLM Output Validation Caught a Bad Deploy Before Users Saw It

We bumped a model version in a pipeline that extracted structured fields from support tickets — category, priority, a short summary — and shipped it the way we'd shipped the last four version bumps, with a quick manual spot check on a handful of sample tickets. Everything looked fine. What we didn't catch in the spot check was that the new model version had started occasionally wrapping its JSON output in a markdown code fence, something the previous version never did.

Why the spot check missed it

The code-fence behavior wasn't consistent — it showed up on maybe one in twenty outputs, usually ones where the ticket content itself referenced code or technical formatting. Our sample set of a dozen tickets didn't happen to hit that pattern. A five percent failure rate on a parsing step sounds small until you remember it compounds: at ticket volume, that's dozens of silently dropped or malformed records a day, each one a support ticket that doesn't get routed correctly.

A spot check tells you the model behaved on the examples you thought to check. A validation layer tells you whether it behaved on the ones you didn't.

What actually caught it

We had a schema validation step sitting between the model call and the downstream write — strict JSON parsing, required fields checked, type coercion rejected rather than silently applied. It had felt like overkill when we built it, mostly there because a client's compliance requirements demanded an audit trail. The day of the version bump, the validation failure rate on that step jumped from a near-zero baseline to five percent, and the alert on that metric is what got us looking before any tickets actually got misrouted in front of a customer.

• Strict schema validation on every structured output, not just a sample during QA.
• An alert on validation failure rate, not just total failure count — rate catches subtle regressions faster.
• A fallback path for failed validations (retry with a stricter prompt, or route to human review) instead of silently dropping the record.
• Version bumps treated as deploys with a rollback plan, not as a drop-in replacement.

The fix itself was trivial once we found it — a stripping step for markdown fences before parsing. The actual lesson is that the validation layer paid for its entire build cost in one afternoon, and it's the kind of infrastructure that looks unnecessary until the exact day it isn't.