AI FinOps: Cost Management & Optimization
When several applications and teams use AI, it can be difficult to see where spend comes from or keep usage within limits. GoRoute records request costs, checks budgets before sending work to a provider, and can enforce request and token limits. The detailed guides explain how this accounting works and what it means for your setup.
Core Capabilities
| Capability | Implementation | Guide |
|---|---|---|
| 64-Bit Integer Micro-Dollar Ledger | All costs are computed in integer micro-dollars (1 USD = 1,000,000 micro-dollars) with intermediate pico-dollar precision and zero float32/float64 accounting. | Micro-Dollar Ledger |
| Cache-Write & Modality Pricing | Supports prompt-cache write pricing (CacheCreationTokens / cache_write_cost_micros_per_1m), audio/image/video modality rates (0007/0012), and immutable model_pricing_history. | Micro-Dollar Ledger |
| Unpriced Usage Tracking | Tracks unpriced_requests and unpriced_tokens explicitly so custom or uncatalogued models never silently report $0.00 without visibility. | Micro-Dollar Ledger |
| Pre-Flight Reservations & Lease Renewal | Reserves upper-bound estimated cost before upstream dispatch, renews leases during long streams, and reconciles exact token spend on completion. | Micro-Dollar Ledger |
| Durable PostgreSQL Outboxes | Persistent spend (0008) and budget-reset (0011) outbox tables ensure zero lost accounting events across restarts or transient DB blips. | Deployment & Storage |
| Single-Roundtrip Redis Lua Rate Limits | Atomic sliding-window RPM and TPM enforcement in Redis with fail-closed behavior and {org:<id>} hash-tag compatibility for Redis Cluster. | Sliding-Window Rate Limits |
| Pre-Flight Tokenizer | Local zero-network token estimation (POST /v1/tokenize, POST /anthropic/v1/messages/count_tokens) to enforce TPM/context budgets before dispatch. | Pre-Flight Tokenizer |