Structured Outputs Changed How We Build Agent Pipelines, Quietly

Before structured outputs were reliable, a huge share of our agent pipeline's failures weren't reasoning failures at all — they were parsing failures. The model would return something 95% in the shape we asked for, and that 5% deviation would throw a JSON parse error somewhere downstream, usually in production, usually at the worst time. We built an embarrassing amount of defensive regex and retry logic just to handle a model almost following instructions.

The failure mode nobody talks about enough

Reasoning failures get the attention because they're interesting — a model hallucinates a fact, draws a wrong conclusion, misunderstands intent. Parsing failures are boring, so they get less attention even though in our logs they were, for a long stretch, the single largest category of production incidents. The model was often reasoning fine. It was the glue code trying to interpret its output that kept breaking.

Most of our agent's 'intelligence' bugs turned out to be parsing bugs wearing a more interesting disguise.

What changed once schemas became enforceable

Once we could force output against a strict JSON schema at the API level instead of hoping a prompt instruction was followed, that entire category of incident dropped close to zero. Not because the model got smarter — because we stopped asking it to also be a reliable string formatter on top of being a reasoner, and let the API enforce the contract instead of our regex.

• Retry logic for malformed JSON went from a load-bearing part of the pipeline to something we barely need anymore.
• Downstream code could finally assume a shape instead of defensively checking every field's existence and type.
• Multi-step agent pipelines got more reliable overall, because each step's output stopped being a potential parsing landmine for the next step.
• We were able to delete roughly 400 lines of output-sanitization code that existed purely to work around unstructured generation.

It's a quiet change because it doesn't produce a demo-able new capability — the agent doesn't do anything it couldn't do before. It just does it reliably enough that you can build a second and third step on top of it without the whole chain being fragile. That reliability is most of what separates an agent pipeline that survives contact with production from one that doesn't.