Skip to main content

Model Management & Access Control

GoRoute AI Gateway separates physical upstream models from logical application endpoints. Through a 4-layer access control hierarchy, organizations can govern vendor entitlements, enforce approval policies, and restrict client keys with zero-latency hot-path verification.


4-Layer Model Access Control Hierarchy​

Every inference request passes through four defensive evaluation boundaries:

Layer 1: Global Model Catalogue Layer 2: Org Provider Subscriptions Layer 3: Org Model Policy Overrides Layer 4: Virtual Key Allowlist Valid Unknown Model Subscribed Provider Disabled Approved Model Blocked Authorized Key Restricted Incoming Request (model: 'cerebras/llama-3.3-70b') HTTP 404 Model Not Found HTTP 403 Provider Not Subscribed HTTP 403 Model Policy Blocked Stage 2 Load Balancer & Dispatch HTTP 403 Key Not Authorized Verify model exists, active lifecycle status,tri-state capabilities & pricing provenance Check if Organization has enabled provider(OrgProviderSubscription.IsEnabled = true) Check if model is approved for the org(OrgModelPolicy.IsEnabled = true) Verify key allowed_models list(VirtualKey.AllowedModels)

Layer Breakdown​

  1. Layer 1: Global Model Catalogue (GlobalModel): Tracks model records known to this gateway deployment, including discovered and manually registered models. Catalog contents and lifecycle status can change as provider catalogs and discovery settings change. The catalog is not a promise that every upstream model is compatible with every route or enabled for every tenant.
  2. Layer 2: Organization Provider Subscriptions (OrgProviderSubscription): Enterprise vendor gating. An organization controls which upstream providers (openai, anthropic, gemini, cohere) are permitted for use by its developers and teams. Supports auto_enable_new to automatically adopt new models discovered from an approved vendor.
  3. Layer 3: Organization Model Policy Overrides (OrgModelPolicy): Tenant model approval. Even if a provider is subscribed, security teams can explicitly approve or ban specific models (e.g. banning preview models, approving only HIPAA-compliant models). Records approved_by and approved_at audit timestamps.
  4. Layer 4: Virtual Key Allowlist (VirtualKey.AllowedModels): Application-level restriction. Each virtual API key can define a scoped allowed_models array (e.g. ["fast", "text-embedding-3-small"]). If set, callers using that key are restricted exclusively to those models.

Tri-State Capability Verification​

Model capabilities are notoriously prone to drift between vendor documentation and actual API support. To prevent runtime runtime errors, GoRoute replaces fragile boolean flags with an explicit Tri-State Capability Model (internal/domain/capabilities.go):

type CapabilityState string

const (
CapabilitySupported CapabilityState = "supported" // Verified functional
CapabilityUnsupported CapabilityState = "unsupported" // Verified lack of support
CapabilityUnknown CapabilityState = "unknown" // Unverified / experimental
)

The gateway tracks 13 distinct capability dimensions per model:

Capability FieldDescription
toolsFunction and tool calling support
streamingServer-Sent Events (SSE) streaming tokens
visionMultimodal image and diagram understanding
reasoningExtended thinking / reasoning effort parameters
structured_outputsStrict JSON Schema enforcement (response_format)
embeddingsVector embedding generation
rerankDocument cross-encoder reranking
moderationSafety and policy classification
image_genText-to-image synthesis
audio_sttAudio transcription / speech-to-text
audio_ttsText-to-speech audio synthesis
batches24-hour asynchronous batch processing
realtimeFull-duplex WebSocket sessions

[!IMPORTANT] If a client requests a feature (e.g. stream: true or tools) for a model whose capability state is unsupported, the gateway immediately rejects the request with HTTP 400 Bad Request prior to upstream dispatch, preventing wasted vendor charges.


Pricing Provenance & Fail-Closed Safety​

To guarantee financial integrity across billing cycles, model pricing in GoRoute includes full cryptographic and audit provenance (internal/domain/modelmgmt.go):

  • Pricing Status:
    • verified: Confirmed against official provider contracts (e.g. global-model-catalog-v1.0).
    • custom: Overridden by organization enterprise discount agreements.
    • unknown: Unverified pricing tier. Triggers fail-closed budget checks.
    • unrecorded: Free or internal local models (e.g. Ollama).
  • Pricing Provenance Source: Explicit reference tag indicating source documentation (e.g. "global-model-catalog-v1.0", "enterprise-agreement-2026-q1").
  • Versioned History (ModelPricingHistory): Every rate change increments pricing_version and stores the previous rates with effective timestamps and changing principal.

Responses echo provenance headers:

X-Gateway-Cost-Micros: 42
X-Gateway-Pricing-Status: verified
X-Gateway-Pricing-Version: 1

Declarative Model Groups (configs/config.yaml)​

Define load-balancing tiers, weights, and cross-provider fallbacks:

model_list:
# Fast tier: Load-balanced between Gemini 2.5 Flash and GPT-4o-mini
- model_name: fast
routing_strategy: weighted-shuffle
model_params:
model: gemini/gemini-2.5-flash
api_key: os.environ/GEMINI_API_KEY
weight: 75

- model_name: fast
routing_strategy: weighted-shuffle
model_params:
model: openai/gpt-4o-mini
api_key: os.environ/OPENAI_API_KEY
weight: 25

# Reasoning tier: High-complexity models with automatic fallback
- model_name: reasoning
routing_strategy: least-busy
model_params:
model: anthropic/claude-sonnet-5-5
api_key: os.environ/ANTHROPIC_API_KEY
fallbacks: ["backup-reasoning"]

# Local tier: Private local LLM (Ollama)
- model_name: local
model_params:
model: ollama/llama3.2:1b
api_base: http://localhost:11434

Listing Available Models (GET /v1/models)​

Clients can inspect configured aliases and the models permitted for their key. This tenant-filtered response is the deployment-specific check for available model IDs; it does not enumerate every model offered by an upstream provider:

curl http://localhost:8080/v1/models \
-H "Authorization: Bearer sk-gw-prod-7f9a2b8c4d1e"

Response:

{
"object": "list",
"data": [
{"id": "fast", "object": "model", "owned_by": "goroute-gateway"},
{"id": "reasoning", "object": "model", "owned_by": "goroute-gateway"},
{"id": "auto", "object": "model", "owned_by": "goroute-gateway"},
{"id": "gemini-2.5-flash", "object": "model", "owned_by": "gemini"},
{"id": "claude-sonnet-4-20250514", "object": "model", "owned_by": "anthropic"}
]
}