Integration Testing Caught a Vendor's Bug Before It Reached a Client

A payment vendor pushed a routine SDK version bump — the kind of dependency update that normally gets merged on autopilot with a quick skim of the changelog. Our integration test suite, run automatically against every dependency bump before it could merge, failed on a single assertion: a test transaction converting 100 USD to EUR returned a value off by about 4%. Not zero, not an error — just quietly, plausibly wrong.

What the vendor had actually changed

Digging into their changelog, the bump included an internal refactor to how they cached exchange rates, and that refactor introduced a bug where a stale rate from a previous currency pair occasionally got applied to a different pair under specific caching conditions. It wasn't in their changelog as a known issue — we found it ourselves, reported it, and they confirmed and fixed it within two days. Nothing about the SDK bump looked risky on paper. It was a 'minor version, bug fixes and internal improvements' release.

A vendor's changelog tells you what they think changed. An integration test tells you what actually did.

Why this specific test existed

This wasn't a generic smoke test — it was a deliberately constructed integration test that ran real currency conversions against known, hand-verified expected values for a handful of currency pairs, specifically because currency conversion had bitten us before on a different vendor years earlier. The test suite treated every dependency bump, including ones from a trusted payment vendor, as something to verify against real expected outputs rather than just checking that the SDK still loaded without throwing.

• Tests asserted on known correct values, not just 'did the call succeed' — a currency conversion returning a plausible-looking wrong number passes a smoke test and fails a value-assertion test.
• Dependency bumps for anything touching money, PII, or compliance-relevant logic were gated through this suite automatically, with no manual override to skip it even for a 'routine' patch version.
• The suite ran in a staging environment with values close to real transaction amounts, not synthetic round numbers that might mask a percentage-based miscalculation.

If that SDK bump had gone straight to production, a 4% currency miscalculation on live transactions would have run for however long it took someone to notice a reconciliation mismatch — likely weeks, given how subtle the error was per-transaction. The entire save came down to one test asserting a real expected value instead of just checking that nothing crashed. That's the whole argument for value-based integration tests over smoke tests: a vendor bug that behaves exactly like correct behavior is the one you actually need to catch.