Product definition

Halvr sits between your application and its model providers.

It is one request path for cost attribution, budget enforcement, routing, caching, alerts, and audit records. Product services do not need to implement those controls separately.

Updated July 12, 20267 min guide

Summary

Halvr is a proxy-enforced LLM spend control plane for applications using OpenAI, Anthropic, Gemini, or compatible model endpoints. Product code routes requests through Halvr and attaches customer or feature metadata. Halvr can serve an eligible cache hit, enforce policy, reserve estimated provider cost across rolling budgets, forward an allowed request, and reconcile the reservation against actual token usage.

Cost ownership and enforcement

Provider dashboards organize usage by provider account, model, and time. AI products need to know which customer, feature, API key, agent session, or tool call created the cost. Halvr joins provider usage to that product context on each request.

Attribution alone explains completed spend. Halvr also keeps budgets in the request path, where a control decision can stop a new provider call instead of reporting the overrun after it appears on an invoice.

The request path

The proxy authenticates the Halvr project key, resolves product metadata, selects the intended provider and model, evaluates project policy, and checks exact or semantic cache eligibility. A cache hit returns without consuming provider-spend budget.

For a cache miss, Halvr requires verified model pricing and estimates a bounded maximum cost. One Redis Lua operation checks project, API-key, customer, and feature scopes and reserves the estimate in all applicable scopes or none. The provider response then supplies actual tokens for atomic reconciliation.

Halvr and observability tools

An observability product is usually strongest after execution: traces, prompts, evaluations, latency, and debugging history. Halvr is deliberately centered on the financial decision before execution. It still records request evidence, but the defining capability is synchronous cost policy.

The categories overlap. Teams should choose Halvr when customer-level unit economics, rolling dollar budgets, hard blocks, and auditable spend decisions are the primary requirement.

Data Halvr uses

Halvr needs the provider request, the selected model, and the stable product identifiers the application chooses to attach. Full prompt retention is not required for cost attribution. Metadata-only retention keeps usage, routing, policy, latency, and attribution fields without storing prompts or responses.

{
  "metadata": {
    "customerId": "customer_123",
    "feature": "ai_writer",
    "sessionId": "agent_run_8f2"
  }
}

Providers and clients

Halvr has native routing and pricing support for OpenAI, Anthropic, and Gemini models and can connect configured OpenAI-compatible endpoints with explicit custom pricing. OpenAI-compatible clients usually integrate by changing the base URL and using a Halvr project key.

Unknown or retired standard models are blocked before provider spend until Halvr has a verified active price. This prevents a new model identifier from silently bypassing dollar budgets as zero-cost traffic.

Halvr architecture

Inspect the hot path, durable event pipeline, and failure semantics.

Halvr pricing

Compare Free, Performance, and Managed-Spend terms.

Halvr security

Review credential protection, retention, SSRF controls, and audit evidence.

Try Halvr

Route a real request through Halvr.

Keep your current SDK. Change the base URL, attach customer and feature metadata, and set a spend limit.

Start free