February 13, 2026 · 5 min read
A Reinforcement Learning Pilot That Didn't Ship, and Why That Was Fine
We spent six weeks on an RL pilot for dynamic pricing that never made it to production. Calling it a failure misses what the project actually accomplished.
A client wanted to explore reinforcement learning for dynamic pricing on a marketplace, with the long-term goal of a policy that could learn optimal pricing from live market feedback rather than static rules. We built a pilot in a simulated environment using historical transaction data. Six weeks in, we recommended not deploying it, and the client agreed.
What actually killed it
The simulation worked. The policy it learned looked plausible and performed well against the historical replay data. The problem was the gap between a simulated environment and a live market: the policy had learned to exploit specific statistical quirks of the historical dataset that had no guarantee of holding in live conditions, and testing that honestly would have required a live exploration phase with real pricing experiments on real customers — meaning real revenue risk during the learning period, on a business where pricing mistakes are visible and costly immediately.
A reinforcement learning policy that only exists in simulation hasn't been validated, it's been validated against itself.
We could have pushed for a live pilot with a small traffic allocation and strict guardrails. We recommended against it instead, because the expected value of the learning didn't clearly outweigh the downside risk during the exploration phase, for a business that didn't have the pricing margin to absorb a bad few weeks of experimentation.
- A policy that performs well in simulation against historical data has validated nothing about live performance — simulations reward exploiting dataset quirks.
- RL's exploration requirement is a real cost, not a footnote — someone has to absorb the downside of a policy still learning what works.
- Recommending against shipping something you built isn't a failure of the project if the decision is the right one for the business.
- A six-week pilot that correctly identifies 'this isn't ready for live risk yet' is a successful use of six weeks, not a wasted one.
The client walked away with a clear-eyed understanding of what RL-based pricing would actually require before it could be trusted live — and decided, correctly, that the required exploration risk wasn't worth taking on yet. That's not every pilot's story ending in a shipped feature. It's still a good outcome.