LLM budget enforcement
Enforce LLM budgets before another request spends tokens.
Budget alerts tell a team that usage changed. In-path enforcement decides whether the next request may reach the provider. Halvr applies both to rolling micro-USD usage without scattering limit logic across applications.
Summary
LLM budget enforcement checks recorded token usage before forwarding a model request. When a project, API key, customer, or feature has consumed its rolling allowance, the proxy returns HTTP 402 with scope and usage headers instead of calling the provider.
Budget window
Rolling 30 days
Enforcement response
HTTP 402 before upstream
Scopes
Project + key + customer + feature
Alerts and enforcement solve different failure modes.
An alert is asynchronous. It can reach an operator quickly, but requests continue while the message is read, investigated, and acted on. That delay is acceptable for forecasting and anomaly review; it is not a reliable ceiling for runaway agents or batch jobs.
Enforcement is synchronous. The budget decision runs in the same request path as provider routing. Once recorded usage reaches the configured allowance, later requests stop before the provider receives them.
Use a hierarchy that matches operational ownership.
A project budget caps total exposure. An API-key budget isolates a service, environment, or workload. Customer and feature budgets apply commercial or product-specific boundaries without creating a separate key for every dimension.
Halvr evaluates the project first, then the calling API key, followed by customer and feature dimensions attached to the request. The first exhausted scope becomes the blocked reason and is written to the audit trail.
- Project: the maximum provider-cost exposure for one Halvr workspace.
- API key: a boundary for a service, team, environment, or background worker.
- Customer: an allowance aligned to an account or contract.
- Feature: a limit for an experiment or expensive product workflow.
Understand what a rolling spend window guarantees.
Halvr stores provider cost as integer micro-USD in UTC daily Redis buckets and sums the trailing 30 days. Before an upstream call, one Lua operation checks every project, API-key, customer, and feature scope and reserves the estimated cost in all scopes or none of them.
After the provider responds, Halvr atomically replaces the estimate with actual cost. If actual usage exceeds the estimate, the completed request is recorded as an overage and later requests observe the updated balance. Output ceilings remain important because the reservation estimate is the pre-spend guard.
Make the blocked response useful to the application.
Halvr returns HTTP 402 with a short JSON error and headers that identify the exhausted scope, target, micro-USD limit, and recorded spend. Product code can use that stable response to downgrade, queue work, disable an optional action, or show an account-specific limit state.
HTTP/1.1 402 Payment Required
content-type: application/json
x-halvr-budget-scope: api_key
x-halvr-budget-limit-micro-usd: 5000000
x-halvr-budget-used-micro-usd: 5001842
{
"error": "API key monthly spend budget exceeded."
}Send threshold events once, then preserve delivery history.
The warning, exhaustion, and blocked events represent different operational states. Deliver each event once per budget scope in its 30-day deduplication window, retry failed deliveries with bounded backoff, and expose the last result in the dashboard.
Halvr uses the project webhook first, then configured email or Slack channels. Outbound webhooks are signed so receivers can verify the raw body before triggering an incident workflow.
Test the product fallback before enabling the hard limit.
Set a temporary low budget in a non-production project, consume it with a known request, and verify the next call returns the expected 402 without appearing in provider usage. Confirm that warning, exhaustion, and blocked events are not repeated on every retry.
- Verify the correct scope wins when several limits are configured.
- Check streaming and non-streaming callers handle the same blocked response.
- Confirm the user-facing fallback does not automatically retry forever.
- Raise the limit and verify traffic resumes without replacing the API key.
Try Halvr
Route a real request through Halvr.
Keep your current SDK. Change the base URL, attach customer and feature metadata, and set a spend limit.