It started with a glance at the cloud billing page. The number for the last couple of days was higher than it had any right to be — a sharp, unexpected jump, concentrated almost entirely in one line item: the LLM provider.

Nothing was broken. No alert had fired. The app was healthy. And yet the bill had clearly spiked, and I had no idea why. That gap — spend went up and I can’t say which operation caused it — is the whole story.

Forensics: grouping the bill until it confessed

The billing console is blunt, but it’s honest if you ask the right question. I switched the report from “group by service” to group by SKU, and the picture sharpened immediately: the cost wasn’t text generation. It was image generation — output tokens, billed per generated image, which add up far faster than people expect.

The next question was who. And here’s where it got uncomfortable: the answer was “a pile of local scripts.”

We’d accumulated dozens of one-off generation scripts — character concept rounds, episode rebuilds, asset variations. Each of them did the same thing: read an API key from a local env file and call the model provider’s HTTP endpoint directly. They worked. They were convenient. And they were completely invisible:

• No request was traced anywhere.
• No cost was attributed to any operation, script, or person.
• Nothing capped how many images a buggy loop could generate.

An episode rebuild script that re-renders every scene, run a few times during iteration, plus several rounds of character variants — and the only evidence it ever happened was the bill, days later.

The lesson landed hard: we had monitoring for our services, but the expensive thing wasn’t going through our services at all.

The root cause wasn’t a bug — it was an open door

We already had an internal AI service. It had a proper provider abstraction, a commercial-use model allowlist for licensing, and tracing hooks. The problem was that it was optional. Anything could bypass it by calling the provider directly with a raw key, and so the highest-volume, highest-cost workload did exactly that.

You can’t observe or control what doesn’t pass through you. So the fix wasn’t a patch — it was a principle:

Every LLM call goes through the AI service. No exceptions, no direct provider calls anywhere else.

One door. Everything that wants to generate has to walk through it. Then — and only then — does observability and control actually mean something, because there’s a single place to add them.

Making the door worth walking through

A mandate without ergonomics just gets worked around. So before enforcing anything, we made the gateway genuinely better than a direct call.

Cost attribution you can actually read

We enriched every trace the service emits with four pieces of context, passed as request headers:

• source — which system made the call (a script, the scheduler, the web app)
• user — the operator or service behind it
• session — one per job/run
• tags — free-form facets (which script, which model, which purpose)

The session is the trick. A script that generates ninety images sets one session ID at the top and reuses it for every call. In the tracing UI, that whole run collapses into a single session with a summed cost. The question that started this whole mess — which operation cost what? — now has a one-click answer.

We also fixed the embarrassing detail that image generations were being recorded with zero cost. The provider returns token usage on every response; we’d been discarding it. Now we thread it into the trace, register the model’s price once, and let the tracing layer compute real per-generation cost. Sessions roll it up. The bill stopped being a mystery.

Control, not just visibility

The same chokepoint is where control lives. The service enforces a model allowlist — which doubles as a cost guard: the cheap, default model is allowed; the premium image model that quietly drove most of the spike is blocked outright and returns a clear error. If you want it, you have to ask for it on purpose.

And a shared client means scripts no longer hand-roll a fetch to a provider. They import one function, pass their session context, and get back an image. The right way became the easy way.

Enforcement: making the wrong thing impossible

Here’s the part I’m most happy with, because it’s where good intentions usually die. A rule that lives only in a docs file rots. We wanted the rule to defend itself.

A CI guard that scans only what changed

We added a CI check that fails any pull request introducing a direct provider call — a fetch to the provider’s domain, or a raw provider key in code. The twist that made it practical: it scans only the files changed in the diff.

That decision matters. We had two dozen legacy scripts still full of direct calls, and we deliberately chose not to migrate them all at once. With a changed-files scan:

• Untouched legacy scripts stay green. No busywork.
• The moment someone edits one, CI demands they migrate it first.
• No new code can ever add a direct call.

It’s lazy migration, and it’s exactly right: you pay the migration cost only when you’re already in the file for another reason.

Removing the key so the old path fails loudly

The CI guard stops new code at merge time. But what about running an old script? We closed that too — by removing the provider key from the environment the scripts read from. The service keeps its own key, in its own place. Any legacy script that tries to call the provider directly now fails immediately with a missing-key error.

Fail-loud beats fail-silent. A script that errors gets migrated. A script that quietly works keeps spending.

And the seatbelts

Finally, the boring guardrails that should have existed from day one: a budget alert so a spike is never again discovered days late, and — more importantly — a per-day quota on the model itself, which actually blocks further calls once the daily ceiling is hit and resets the next day. An alert tells you the house is on fire; the quota is the fire door.

What I’d tell past me

Three things, in order of how much they’d have saved:

1. A chokepoint is a feature, not a bottleneck. The single gateway isn’t about elegance — it’s the one place where tracing, cost attribution, licensing, and rate limits can exist at all. Optional chokepoints get bypassed by exactly the traffic you most need to see.
2. Monitoring is not control. We had dashboards for everything that went through our services. The lesson is that the dashboard is worthless for the workload that doesn’t.
3. Make the wrong thing impossible, not just discouraged. A rule in a doc is a suggestion. A rule enforced by CI plus a missing key is a wall. The bill spiked because the easy path and the wrong path were the same path. Now they aren’t.

The surprise on the billing page was annoying. But it bought us something we’d been missing for a long time: every AI call we make is now visible, attributed, capped, and routed through one door we actually control. Worth it.