A Caching Layer Fixed a Scaling Problem We Almost Solved With More Hardware

A client's database was consistently pegged at high CPU during business hours, and their plan was to upgrade to the next instance tier up — roughly double the monthly cost — to buy headroom. Before signing off on that, we pulled query logs to see what was actually driving the load, because 'upgrade the instance' is the move you make when you haven't looked yet.

What was actually causing the load

One query — a lookup of a user's subscription tier, used to gate feature access — accounted for about 40% of total query volume. It was a simple, cheap-looking query on its own, but it ran on nearly every API request across the whole platform, and subscription tier data changes maybe once a month per user. We were running tens of millions of near-identical, cheap-per-query reads a day against data that was, for all practical purposes, static.

The database wasn't struggling with a hard query. It was struggling with volume on a query that didn't need to hit the database at all.

What we built instead of upgrading hardware

A Redis cache in front of that specific lookup, keyed by user ID, with a short TTL and an explicit invalidation hook fired whenever a subscription actually changed — so the cache was never stale for longer than the time between a legitimate change and the webhook that fired on it, rather than relying purely on TTL expiry for correctness.

Before: every API request -> DB query for subscription_tier (40% of DB load)
After:  every API request -> Redis GET (cache hit ~97%)
                           -> on miss: DB query, populate cache, TTL 10min
                           -> on subscription change webhook: explicit cache invalidate

• Identified the specific high-volume, low-change-rate query before reaching for more hardware — not every scaling problem is actually a capacity problem.
• Used explicit invalidation on the write path, not just a TTL, since stale subscription tier data has real product consequences (wrong feature gating) if left to expire naturally.
• Measured cache hit rate in production before declaring victory — it landed at 97%, which meant the database saw roughly a 40% reduction in total query volume.
• Left the instance size untouched, since CPU utilization dropped from a sustained 85% to around 45% with the cache in place, well within existing headroom.

The caching layer took about a week to build and test properly, including the invalidation logic, against a hardware upgrade that would have been live in an afternoon but would have cost roughly double, every month, forever, while not actually addressing why the database was under load in the first place. More hardware is the fix when you've genuinely run out of capacity. Most of the time, it's the fix people reach for because finding the actual query is more work than clicking the upgrade button.