Model API access control: per-key limits, budgets, and alerts

Most teams manage model API access the way they manage the office coffee machine: one shared key, everyone pours. The key works, the bill arrives, and nobody can say which workload drank the budget. When the bill doubles, the investigation starts — but that key was used by five projects in parallel, and provider dashboards only show usage per key, not per purpose. The delay is the real problem: an anomaly matters most in the window before you notice it.

Per-key access control turns the question “who spent what” from archaeology into a lookup. Three pieces make it work: one key per workload, limits that cap each key, and alerts that fire before the cap.

One key per workload, not one per team

Issue a separate key for each agent, CI job, and environment, and name it after its owner: agent-codegen-prod, ci-preview, etl-nightly. A legible key name upgrades the provider dashboard from a wall of anonymous tokens to a per-service report. Where the provider supports it, tag or annotate the key with a purpose so downstream tooling can group on it.

Least privilege applies to model access the same way it applies to files: an agent that summarizes code should not carry credentials for your most expensive model. Scope each key to the models its workload actually calls, and you remove the most common leak path — a general-purpose key that any prompt can route to any model.

Per-key issuance also makes rotation surgical. A key that shows up in a log or a prompt gets revoked without reissuing credentials for everyone else, and replacing one workload’s key never touches another’s.

Limits: rate, quota, and budget

Three granularities exist, and you want all three. A rate limit (requests per minute) stops an agent stuck in a retry loop from hammering the endpoint. A daily quota (tokens per day) smooths consumption so one workload cannot starve the others. A spend cap (monthly budget) is the backstop against surprises. Set them in that order — the budget alone is a weak primary control because it speaks once a month.

Where the cap sits matters. A hard cap cuts the workload off mid-run; a soft limit alerts first and lets the run finish. For agents, fail closed on rate and quota: silently degrading a blocked workload to a fallback model can route work to a model nobody approved.

Alerts: watch deltas, not absolutes

Absolute-spend alerts are noise. With a $200 monthly budget, day ten already looks normal, so a threshold on total spend never fires when it should. Alert on deltas instead: this week versus the trailing baseline, a spike in 401 errors (usually a rotated key nobody updated), the first use of a model that has been idle, or requests outside working hours. Each event is actionable: a suddenly quiet key is a rotated one; a suddenly loud key is a leak or a runaway loop. Set alert thresholds below the hard caps so there is still headroom when one fires.

Where the enforcement point lives

All three controls converge at the hop every request passes through. A local reverse proxy in front of the provider endpoint can hold per-workload keys, attach them per host, and record every injection and denial to an append-only audit log that never contains token values — which is also what turns the monthly bill into a per-workload report. trustless is a single-binary credential broker that does exactly this for OpenAI-compatible endpoints via trustless serve, emitting proxy.inject and proxy.deny events for each request (see the command reference). The mechanics are deliberately boring: one key per workload, a cap per key, an alarm per anomaly.

Decide the controls before the next bill surprises you. When you open this month’s provider invoice, will you be able to name which workload spent each line?