An AI Agent's Cost Crept Past the Pilot Budget Within a Month

The pilot was scoped for a modest few hundred dollars a month, based on expected query volume from a limited internal rollout to about thirty employees. By the fourth week, the bill was north of two thousand dollars, and the first instinct — more people are using it than we planned — turned out to be wrong. Usage was almost exactly on target.

Finding the actual cause

The agent had a self-correction loop: if a tool call failed or returned something it judged unsatisfactory, it would retry with a revised approach, up to a limit we'd set generously at five attempts, assuming failures would be rare. They weren't rare. One particular tool — a flaky internal API with an undocumented rate limit — was failing intermittently, and the agent dutifully burned through all five retries nearly every time it hit that tool, each retry carrying the full accumulated conversation context in the prompt.

Expected: 1 tool call per query, occasional retry
Actual:   avg 3.8 retries per query hitting the flaky endpoint

Each retry re-sends full context → token cost compounds, not adds
30 users * ~15 queries/day * 3.8x multiplier ≈ budget blown in days

A retry loop doesn't just multiply your cost by the retry count — it multiplies it by the retry count applied to a growing context window, which is a much uglier curve.

The fix and the lesson

We fixed the flaky endpoint's rate limit handling, capped retries at two with exponential backoff, and — the more durable fix — added a hard per-session token budget that kills a runaway conversation instead of letting it retry indefinitely. The lesson that stuck with me: cost estimates for agentic systems need to model the failure path, not just the happy path, because the failure path is often where the actual spend lives.

• Set a hard ceiling on retries and total tokens per session, not just a soft expectation.
• Alert on cost anomalies daily during a pilot, not monthly when the invoice lands.
• Model the cost of your worst-behaving dependency, not your average one.