LLM cost monitoring
LLM cost monitoring that can stop spend, not just chart it.
Production LLM cost monitoring should explain unit economics and control the next request. Halvr combines request-level attribution with budgets, routing, caching, alerts, and hard blocks in one proxy path.
Summary
An LLM cost monitoring tool records token usage, model price, latency, and product metadata for each request. A complete system also evaluates budgets before forwarding the request, so teams can warn at 80%, block at 100%, and preserve an audit trail of the decision.
Attribution
Customer + feature + session
Budgets
Project + API key + dimension
Alert window
Deduplicated over 30 days
What useful LLM cost monitoring measures.
Total monthly spend is necessary but insufficient. It cannot show whether an AI feature is profitable, which customer caused a spike, or whether a smaller model would have met the same product requirement.
The useful unit is one request with its product context. Record input and output tokens, calculated cost, model, provider, latency, cache status, customer, feature, session, API key, and final routing or policy decision.
- Customer cost answers whether account-level pricing covers model usage.
- Feature cost shows which product workflows consume margin.
- API-key cost separates environments, services, and teams.
- Session and tool cost expose long-running agent loops.
Monitoring and cost control are different jobs.
Monitoring is retrospective: it records what the provider already processed. Cost control is synchronous: it checks whether the request should proceed before billable tokens are generated.
Halvr keeps both in the same path. The proxy authenticates the Halvr API key, resolves attribution metadata, checks zero-provider-cost cache hits, validates model pricing, and atomically reserves estimated micro-USD across every applicable budget before it sends an allowed request upstream.
Use explicit metadata instead of guessing from prompts.
Do not infer customers or features from prompt text. Attach stable identifiers your application already knows. This avoids leaking business context into prompts and gives analytics a durable join key.
{
"metadata": {
"customerId": "customer_123",
"feature": "ai_writer",
"sessionId": "agent_run_8f2"
}
}Design budgets around operational ownership.
Set a project budget for the global ceiling and API-key budgets for service-level containment. Add customer or feature budgets where commercial plans or product experiments need independent limits.
Threshold events should be durable and deduplicated. A warning should fire once when a scope crosses 80%, exhaustion once at 100%, and a separate blocked event when the proxy rejects traffic. Repeating the same alert on every request trains teams to ignore it.
Evaluation checklist for an LLM cost monitoring tool.
Choose based on the control boundary you need, not the largest feature list. A tracing platform, provider gateway, and budget enforcement proxy can overlap while solving different primary problems.
- Does it calculate cost from actual token usage and current model pricing?
- Can it attribute spend to your own customer and feature identifiers?
- Can it reject an over-budget request before the provider receives it?
- Can you choose metadata-only, redacted, or full payload retention?
- Are alerts signed, retryable, deduplicated, and visible in an audit log?
- Can you export or independently verify policy and cost decisions?
Try Halvr
Route a real request through Halvr.
Keep your current SDK. Change the base URL, attach customer and feature metadata, and set a spend limit.