Model Management & Access Control
GoRoute AI Gateway separates physical upstream models from logical application endpoints. Through a 4-layer access control hierarchy, organizations can govern vendor entitlements, enforce approval policies, and restrict client keys with zero-latency hot-path verification.
4-Layer Model Access Control Hierarchy
Every inference request passes through four defensive evaluation boundaries:
Layer Breakdown
- Layer 1: Global Model Catalogue (
GlobalModel): Tracks model records known to this gateway deployment, including discovered and manually registered models. Catalog contents and lifecycle status can change as provider catalogs and discovery settings change. The catalog is not a promise that every upstream model is compatible with every route or enabled for every tenant. - Layer 2: Organization Provider Subscriptions (
OrgProviderSubscription): Enterprise vendor gating. An organization controls which upstream providers (openai,anthropic,gemini,cohere) are permitted for use by its developers and teams. Supportsauto_enable_newto automatically adopt new models discovered from an approved vendor. - Layer 3: Organization Model Policy Overrides (
OrgModelPolicy): Tenant model approval. Even if a provider is subscribed, security teams can explicitly approve or ban specific models (e.g. banning preview models, approving only HIPAA-compliant models). Recordsapproved_byandapproved_ataudit timestamps. - Layer 4: Virtual Key Allowlist (
VirtualKey.AllowedModels): Application-level restriction. Each virtual API key can define a scopedallowed_modelsarray (e.g.["fast", "text-embedding-3-small"]). If set, callers using that key are restricted exclusively to those models.
Tri-State Capability Verification
Model capabilities are notoriously prone to drift between vendor documentation and actual API support. To prevent runtime runtime errors, GoRoute replaces fragile boolean flags with an explicit Tri-State Capability Model (internal/domain/capabilities.go):
type CapabilityState string
const (
CapabilitySupported CapabilityState = "supported" // Verified functional
CapabilityUnsupported CapabilityState = "unsupported" // Verified lack of support
CapabilityUnknown CapabilityState = "unknown" // Unverified / experimental
)
The gateway tracks 13 distinct capability dimensions per model:
| Capability Field | Description |
|---|---|
tools | Function and tool calling support |
streaming | Server-Sent Events (SSE) streaming tokens |
vision | Multimodal image and diagram understanding |
reasoning | Extended thinking / reasoning effort parameters |
structured_outputs | Strict JSON Schema enforcement (response_format) |
embeddings | Vector embedding generation |
rerank | Document cross-encoder reranking |
moderation | Safety and policy classification |
image_gen | Text-to-image synthesis |
audio_stt | Audio transcription / speech-to-text |
audio_tts | Text-to-speech audio synthesis |
batches | 24-hour asynchronous batch processing |
realtime | Full-duplex WebSocket sessions |
[!IMPORTANT] If a client requests a feature (e.g.
stream: trueortools) for a model whose capability state isunsupported, the gateway immediately rejects the request withHTTP 400 Bad Requestprior to upstream dispatch, preventing wasted vendor charges.
Pricing Provenance & Fail-Closed Safety
To guarantee financial integrity across billing cycles, model pricing in GoRoute includes full cryptographic and audit provenance (internal/domain/modelmgmt.go):
- Pricing Status:
verified: Confirmed against official provider contracts (e.g.global-model-catalog-v1.0).custom: Overridden by organization enterprise discount agreements.unknown: Unverified pricing tier. Triggers fail-closed budget checks.unrecorded: Free or internal local models (e.g. Ollama).
- Pricing Provenance Source: Explicit reference tag indicating source documentation (e.g.
"global-model-catalog-v1.0","enterprise-agreement-2026-q1"). - Versioned History (
ModelPricingHistory): Every rate change incrementspricing_versionand stores the previous rates with effective timestamps and changing principal.
Responses echo provenance headers:
X-Gateway-Cost-Micros: 42
X-Gateway-Pricing-Status: verified
X-Gateway-Pricing-Version: 1
Declarative Model Groups (configs/config.yaml)
Define load-balancing tiers, weights, and cross-provider fallbacks:
model_list:
# Fast tier: Load-balanced between Gemini 2.5 Flash and GPT-4o-mini
- model_name: fast
routing_strategy: weighted-shuffle
model_params:
model: gemini/gemini-2.5-flash
api_key: os.environ/GEMINI_API_KEY
weight: 75
- model_name: fast
routing_strategy: weighted-shuffle
model_params:
model: openai/gpt-4o-mini
api_key: os.environ/OPENAI_API_KEY
weight: 25
# Reasoning tier: High-complexity models with automatic fallback
- model_name: reasoning
routing_strategy: least-busy
model_params:
model: anthropic/claude-sonnet-5-5
api_key: os.environ/ANTHROPIC_API_KEY
fallbacks: ["backup-reasoning"]
# Local tier: Private local LLM (Ollama)
- model_name: local
model_params:
model: ollama/llama3.2:1b
api_base: http://localhost:11434
Listing Available Models (GET /v1/models)
Clients can inspect configured aliases and the models permitted for their key. This tenant-filtered response is the deployment-specific check for available model IDs; it does not enumerate every model offered by an upstream provider:
curl http://localhost:8080/v1/models \
-H "Authorization: Bearer sk-gw-prod-7f9a2b8c4d1e"
Response:
{
"object": "list",
"data": [
{"id": "fast", "object": "model", "owned_by": "goroute-gateway"},
{"id": "reasoning", "object": "model", "owned_by": "goroute-gateway"},
{"id": "auto", "object": "model", "owned_by": "goroute-gateway"},
{"id": "gemini-2.5-flash", "object": "model", "owned_by": "gemini"},
{"id": "claude-sonnet-4-20250514", "object": "model", "owned_by": "anthropic"}
]
}