01 LLM cost controls in the request path
Halvr is a proxy-enforced spend control plane for teams shipping AI features. Route model calls through one endpoint, attach product metadata, and block budget overruns before a provider receives the request.
Acme production
Live spend control
MTDMonth-to-date
$7,241BlockedBlocked spend
$844CacheCache hit
42%P95p95 overhead
4.1 msRequest path
before provider spend
customer_982
gpt-4o
support_copilot
Blocked
team_enterprise
claude-3-5
agent_review
Routed
customer_123
gpt-4o-mini
ai_writer
Allowed
customer_771
gpt-4o-mini
research_agent
Routed
metadata.customerId=customer_123 / metadata.feature=ai_writer / policy=project_budget
Integration
One base URL
Attribution
Customer + feature
Enforcement
Spend budgets
Alerting
Webhook first
02 Built for real AI traffic
Production AI traffic does not arrive as neat dashboard rows. It arrives as bursts, retries, agents, cache hits, and one customer that can burn the entire allowance.
Halvr turns each request into an attributed control decision. The dashboard stays useful because the proxy already resolved ownership, cost, policy, and provider route at the point where spend could still be stopped.
Budget envelope
$7,244
of $10,000
The chart shows request load rising through the month while an eighty percent budget threshold remains visible as a dashed warning line.
Peak request load
18.2k/min
Budget window
rolling 30d
Warn threshold
80%
Block mode
pre-provider
Works with
03 How it works
Halvr is not a reporting SDK after the fact. It is an instrument in the request path, so it can decide, route, meter, and reject before spend happens.
Keep your SDK, payload shape, and provider mental model. The only required change is routing through Halvr.
Send stable customer and feature identifiers so spend maps to product ownership instead of raw provider accounts.
Project, key, customer, and feature budgets are evaluated before the upstream provider call can spend tokens.
Usage events, alerts, dashboards, and audit logs make the next cost decision more precise than the last one.
OpenAI-compatible base URL
No SDK rewrite.
const openai = new OpenAI({
apiKey: process.env.HALVR_API_KEY,
baseURL: "https://api.halvr.io/v1"
});
await openai.chat.completions.create({
model: "gpt-4o-mini",
messages,
metadata: {
customerId: "customer_123",
feature: "ai_writer"
}
});
// Attribution now follows the request.04 Savings calculator
The forecast uses conservative assumptions, integer micro-USD math, and the same catalog-backed Performance billing rule used by the dashboard.
Traffic assumptions
Default values are moderate for a production AI feature and avoid inflated counterfactuals.
5,400,000
monthly calls
Estimated net customer savings
$34,836Net savings subtract the projected Halvr fee, so this is closer to what finance will see after provider spend and Halvr billing.
Cache
$18,662Routing
$13,997Model selection
$10,88605 Pricing
Free proves the savings path. Performance charges from verified savings. Managed-Spend gives finance teams predictable pricing against the LLM spend Halvr controls.
For one production path that needs real savings features before a paid rollout.
10,000 requests or $1k managed spend per month, whichever comes first.
The default paid plan for teams that want Halvr paid from savings it can prove.
$99 monthly minimum, credited against the 20% verified-savings share.
The graduation plan for mature, optimized accounts that need predictable spend-under-management pricing.
Sales-only annual contract for teams averaging $25k+/mo spend under management.