Notes From Adopting an AI Gateway Across Multiple Projects

I started routing every client project's LLM calls through a single AI gateway instead of calling providers directly, mostly to get consistent logging, rate limiting, and cost tracking across projects that otherwise each reinvented that layer badly. Six months in, the notes are mostly positive with one sharp edge I didn't anticipate.

What it actually solved

• Unified spend tracking across providers and projects, visible in one dashboard instead of five separate vendor billing pages.
• Centralized rate limiting and retry logic, written once instead of reimplemented slightly differently in every project.
• A single place to swap or fall back between providers, which made the multi-provider resilience work from an earlier project dramatically simpler the second time around.
• Consistent request/response logging for debugging, which previously varied project to project based on who'd set it up.

The gateway fixed the problem of five inconsistent integrations. It created the problem of one thing that, when it has a bad day, takes every project down with it.

The sharp edge

A gateway misconfiguration — a bad rate-limit threshold pushed during routine maintenance — caused a cascading failure across three unrelated client projects simultaneously, something that structurally couldn't have happened when each project called its provider directly. Centralizing the integration also centralizes the blast radius of a mistake in that one piece of shared infrastructure.

I still think the tradeoff is worth it, but I've added monitoring and staged rollouts for gateway config changes that I wouldn't have bothered with if the gateway only affected one project. Shared infrastructure earns the same operational seriousness as production code, which is an easy thing to under-invest in right up until the day it isn't.