Cost Accounting and Spend Records
The gateway represents calculated cost as integer micro-dollars: 1 USD = 1,000,000 µ$. The calculator uses integer arithmetic and pricing values from the configured model catalogue. This avoids floating-point rounding in the gateway's accounting calculation; it does not guarantee that provider pricing metadata or reported token usage is complete or current.
Cost calculation
For a model with pricing metadata, CalculateCostMicros (internal/finops/calculator.go) accounts for:
- Non-cached prompt tokens (
input_cost_micros_per_1m) - Cached prompt read tokens (
cached_input_cost_micros_per_1m) - Cache-write tokens (
CacheCreationTokensbilled atcache_write_cost_micros_per_1madded in migration0012, falling back toinput_cost_micros_per_1mif unset) - Regular completion tokens (
output_cost_micros_per_1m) - Reasoning tokens (
reasoning_cost_micros_per_1m, falling back tooutput_cost_micros_per_1mif unset) - Modality-specific units (migrations
0007and0012): rerank search units (rerank_cost_micros_per_search_unit), image generation base/HD (image_cost_micros_base,image_cost_micros_hd), audio STT per second (audio_stt_cost_micros_per_second), audio TTS per 1k characters (audio_tts_cost_micros_per_1k_chars), and extensiblemodality_pricing_json, recorded withmodalityandmodality_unitsonspend_logs.
Each token rate is expressed in micro-dollars per million tokens, and the total numerator is divided by one million with ceiling division so a non-zero priced request does not round down to zero. Upper-bound pre-dispatch budget reservations (EstimateUpperBoundCostMicros) also factor in cache_write_cost_micros_per_1m.
The gateway returns cost telemetry on inference responses and records spend events for later queries. If a model has no pricing record (or unverified rates), the calculator returns zero cost and increments unpriced_requests and unpriced_tokens in spend summaries (/admin/v1/spend/summary) so unpriced usage is never silently hidden.
Durable Outbox, Budget Reservations & Period Rollover
- Pre-Dispatch Upper-Bound Reservations & Lease Renewal: Before dispatching upstream, the gateway reserves an upper-bound micro-dollar budget in Redis and renews the reservation lease for long-running or streaming requests.
- Durable Postgres Spend Event Outbox: Spend events are persisted through a retryable PostgreSQL outbox (
0008_add_spend_event_outbox.up.sql) and reconciled idempotently in Redis by event ID (gw:spend:applied:{org}), preventing double-counting on retries. - Unified Modality Budget Gating: Chat, embeddings, audio, and image requests share unified budget gating with request-sized modality estimates.
- Automatic Budget Period Rollover:
ResetExpiredBudgetsadvances expired budget windows, synchronizes entityspent_microsin PostgreSQL (0011_budget_reset_outbox.up.sql), and clears corresponding Redis spend counters.
Spend persistence
Spend events are queued to a buffered Go channel and processed by a background worker. The worker updates spend counters and batches durable writes to the configured repository. If the queue is full, the worker attempts a synchronous write for that event, so storage backpressure can affect a request in that exceptional case.
Batch size, flush interval, and channel size are configurable. The default values are in the gateway configuration; they are not a throughput guarantee.
Query spend data
The admin API provides spend summary and spend log endpoints under /admin/v1/spend. Authenticate with an admin key:
curl http://localhost:8080/admin/v1/spend/summary \
-H "Authorization: Bearer $GATEWAY_MASTER_ADMIN_KEY"
The exact summary fields follow the API response and storage implementation for the gateway version you run. See the admin API reference and virtual keys guide.