Complete Configuration and Environment Reference
GoRoute AI Gateway supports a layered configuration architecture:
- Built-in Defaults: Hardcoded, zero-dependency defaults optimized for instant developer startup.
- Declarative YAML (
configs/config.yaml): Structured configuration defining storage backends, security boundaries, rate limits, model groups, and routing topologies. - Environment Variable Overrides (
GATEWAY_*/os.environ/VAR): Production overrides for Kubernetes ConfigMaps, container runtime secrets, and 12-factor cloud deployments. - Go SDK Options (
pkg/gateway): In-process configuration primitives for embedded architectures.
1. Quickstart Configuration Modes
Mode A: Zero-Config Environment Variables (Fastest)
Set your upstream provider API keys directly in your shell or container definition. The gateway automatically spins up an in-memory repository, provisions an AES-256-GCM vault, and configures default model aliases:
# Frontier LLM Providers
export OPENAI_API_KEY="sk-proj-..."
export ANTHROPIC_API_KEY="sk-ant-..."
export GEMINI_API_KEY="AIzaSy..."
export MISTRAL_API_KEY="mis_..."
# Enterprise Cloud Hyperscalers
export AWS_ACCESS_KEY_ID="AKIA..."
export AWS_SECRET_ACCESS_KEY="..."
export AWS_REGION="us-east-1"
export GCP_PROJECT_ID="my-gcp-project"
export GCP_REGION="us-central1"
# Gateway Master Secrets
export GATEWAY_MASTER_ADMIN_KEY="sk-gw-master-secret-passphrase-change-me"
export GATEWAY_MASTER_ENCRYPTION_KEY="0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"
# Launch Gateway
./ai-gateway
Mode B: Production Declarative YAML
Supply the path to your YAML configuration file via the -config CLI argument:
./ai-gateway -config /etc/goroute/config.yaml
The gateway resolves environment variable secret placeholders matching os.environ/VARIABLE_NAME inside strings dynamically during YAML parsing:
model_params:
model: "openai/gpt-4o"
api_key: "os.environ/OPENAI_API_KEY"
2. Complete 70+ Configuration Options Matrix
The table below documents every configurable parameter across all gateway subsystems.
| YAML Path | Environment Variable Override | Default | Subsystem & Description |
|---|---|---|---|
server.host | None | "0.0.0.0" | Server: HTTP network bind address. |
server.environment | ENV / GATEWAY_ENV | "development" | Server: Runtime environment ("development", "staging", "production"). Production rejects default secrets. |
server.port | GATEWAY_PORT | 8080 | Server: HTTP listen port (1–65535). |
server.grpc_port | GATEWAY_GRPC_PORT | 0 | Server: Optional A2A gRPC listen port (0 disables gRPC). |
server.cors_allowed_origins | None | [] | Security: Allowed CORS origins (empty disables CORS headers). |
server.cors_allow_credentials | None | false | Security: Enables Access-Control-Allow-Credentials (disallowed with *). |
server.enable_hsts | None | false | Security: Emits Strict-Transport-Security header on responses. |
server.max_file_storage_bytes_per_org | GATEWAY_MAX_FILE_STORAGE_BYTES_PER_ORG | 5368709120 | Files: Per-org storage quota in bytes (5 GiB default; 0 = unlimited). |
server.max_concurrent_file_uploads | GATEWAY_MAX_CONCURRENT_FILE_UPLOADS | 8 | Files: Maximum concurrent /v1/files uploads (0 = unlimited). |
server.read_timeout | None | 30s | Server: Maximum duration for reading the entire request body. |
server.write_timeout | None | 300s | Server: Maximum duration before timing out writes (sized for reasoning streams). |
server.idle_timeout | None | 120s | Server: Maximum time to wait for the next request when keep-alives are enabled. |
server.shutdown_timeout | None | 15s | Server: Graceful shutdown deadline before force-closing active HTTP streams. |
logging.level | GATEWAY_LOG_LEVEL | "info" | Logging: Log verbosity: "debug", "info", "warn", "error". |
logging.format | GATEWAY_LOG_FORMAT | "text" | Logging: "json" (cloud structured) or "text" (CLI). Auto-selects "json" if ENV=production. |
logging.add_source | GATEWAY_LOG_ADD_SOURCE | false | Logging: Annotates log lines with source file:line for developer debugging. |
storage.mode | GATEWAY_STORAGE_MODE | "memory" | Storage: Repository backend: "memory" or "postgres". |
storage.postgres_dsn | GATEWAY_POSTGRES_DSN | "" | Storage: PostgreSQL connection DSN (postgres://user:pass@host:5432/db). |
storage.redis_addr | GATEWAY_REDIS_ADDR | "localhost:6379" | Storage: Standalone Redis host:port for L2 auth caching, Lua rate limits, and pub/sub. |
storage.redis_addrs | GATEWAY_REDIS_ADDRS | [] | Storage: Comma-separated Redis Cluster or Sentinel seed nodes. |
storage.redis_master_name | GATEWAY_REDIS_MASTER_NAME | "" | Storage: Redis Sentinel master name. |
tracing.otlp_endpoint | None | "" | Observability: Optional OTLP/HTTP traces URL (empty disables export). |
tracing.service_name | None | "ai-gateway-go" | Observability: Service name attribute attached to exported OpenTelemetry spans. |
tracing.headers | None | {} | Observability: Optional HTTP headers sent with OTLP trace exports (e.g., auth tokens). |
storage.redis_pass | GATEWAY_REDIS_PASS | "" | Storage: Optional Redis authentication password. |
storage.redis_db | None | 0 | Storage: Redis logical database index (0–15). |
auth.master_admin_key | GATEWAY_MASTER_ADMIN_KEY | (auto-generated) | Auth: Master Bearer token for /admin/v1/* control plane APIs. |
auth.allow_master_key_data_plane | GATEWAY_ALLOW_MASTER_KEY_DATA_PLANE | false | Auth: Allows master admin key on data-plane routes (disabled by default for security). |
auth.master_encryption_key | GATEWAY_MASTER_ENCRYPTION_KEY | (auto-generated) | Auth: 64-character hex string (32-byte key) for AES-256-GCM vault encryption. |
auth.jwt_secret | GATEWAY_JWT_SECRET | (random) | Auth: HMAC secret used for signing internal short-lived JWT tokens. |
auth.default_org_id | None | "org_default" | Auth: Primary root organization identifier seeded on initial startup. |
auth.default_virtual_key | GATEWAY_DEFAULT_KEY | "" | Auth: Optional pre-seeded client virtual key for automated integration testing. |
auth.l1_cache_ttl | None | 15s | Auth: In-memory L1 cache TTL for virtual key validation and identity contexts. |
auth.l2_cache_ttl | None | 5m | Auth: Redis L2 cache TTL for virtual key validation across clustered pods. |
router.default_fast_alias | None | "fast" | Router: Virtual model alias routing to high-throughput, low-latency models. |
router.default_reasoning_alias | None | "reasoning" | Router: Virtual model alias routing to frontier deep-reasoning models. |
router.default_coding_alias | None | "coding" | Router: Virtual model alias routing to specialized code-generation models. |
router.complexity_threshold | None | 0.40 | Router: Prompt complexity heuristic score [0.0, 1.0] triggering "auto" $\rightarrow$ "reasoning". |
router.long_context_tokens | None | 32000 | Router: Input token threshold triggering automatic routing to long-context engines. |
router.default_max_retries | None | 2 | Router: Maximum intra-deployment retries before failing over to fallback chain. |
router.base_retry_delay | GATEWAY_ROUTER_BASE_RETRY_DELAY | 50ms | Router: Base backoff delay for full-jitter exponential retry calculations. |
router.max_retry_delay | GATEWAY_ROUTER_MAX_RETRY_DELAY | 2s | Router: Maximum ceiling delay between retry attempts. |
router.default_cooldown | GATEWAY_ROUTER_DEFAULT_COOLDOWN | 60s | Router: Duration a tripped deployment remains isolated in OPEN circuit breaker state. |
router.failure_window | None | 30s | Router: Sliding time window for tracking consecutive upstream provider errors. |
router.failure_threshold | None | 3 | Router: Number of consecutive errors required to trip deployment circuit breaker. |
rate_limit.enabled | None | true | Rate Limit: Enables distributed sliding-window rate limiting via Redis Lua scripts. |
rate_limit.window_seconds | None | 60 | Rate Limit: Window duration in seconds for RPM and TPM rate tracking. |
rate_limit.concurrency_ttl_sec | None | 300 | Rate Limit: TTL lease in seconds for tracking concurrent in-flight requests. |
rate_limit.default_estimated_tpm | None | 128 | Rate Limit: Heuristic token estimate reserved prior to parsing request payload. |
rate_limit.redis_timeout | None | 50ms | Rate Limit: Timeout for Redis rate check execution before failing closed. |
finops.channel_buffer_size | None | 16384 | FinOps: Asynchronous ring buffer capacity for in-flight spend ledger events. |
finops.batch_size | None | 250 | FinOps: Maximum spend records batched into a single database bulk insert. |
finops.flush_interval | GATEWAY_FINOPS_FLUSH_INTERVAL | 500ms | FinOps: Maximum interval between background spend ledger flushes. |
finops.max_unflushed_age | None | 5s | FinOps: Hard timeout forcing asynchronous spend ledger buffer flush. |
discovery.auto_approve_discovered_models | None | true | Discovery: Automatically activates newly discovered upstream models. |
discovery.poll_interval_hours | None | 24 | Discovery: Interval in hours between background upstream provider catalog syncs. |
discovery.enable_background_sync | None | true | Discovery: Enables background discovery daemon across enabled providers. |
security.network.egress_proxy_url | GATEWAY_EGRESS_PROXY_URL | "" | Network: Corporate HTTP/HTTPS egress proxy URL for enterprise firewalls. |
security.network.outbound_ssrf.allow_private_networks | GATEWAY_SSRF_ALLOW_PRIVATE | false | SSRF: Enables egress to private/internal IPs (must be false in production). |
security.network.outbound_ssrf.allowed_private_cidrs | None | [] | SSRF: Explicit CIDR allowlist (e.g. ["10.240.0.0/16"]) when egress is gated. |
security.network.outbound_ssrf.allow_insecure_http | GATEWAY_SSRF_ALLOW_INSECURE_HTTP | false | SSRF: Permits unencrypted HTTP upstream calls (prohibited in production). |
security.network.outbound_ssrf.allowed_schemes | None | ["https"] | SSRF: Permitted URL schemes for provider endpoints (https, http). |
security.network.outbound_ssrf.blocked_ips | None | ["169.254.169.254", ...] | SSRF: Cloud instance metadata IPs blocked from outbound routing. |
security.network.outbound_ssrf.dial_timeout | None | 10s | SSRF: Network connect and TLS handshake timeout for outbound calls. |
security.network.outbound_ssrf.max_redirects | None | 3 | SSRF: Maximum allowed HTTP redirect hops on outbound provider requests. |
security.network.inbound_caller.expose_on_public_internet | None | false | Network: Policy flag asserting whether gateway is exposed to public web. |
security.network.inbound_caller.allowed_client_cidrs | None | [] | Network: Client CIDR allowlist restricting incoming caller IP addresses. |
mcp.enabled | None | true | MCP: Activates Model Context Protocol virtual hub and bridge endpoints. |
mcp.allow_stdio_servers | None | false | MCP: Permits spawning local command-line subprocess MCP tool servers. |
mcp.session_ttl_minutes | None | 15 | MCP: Inactivity timeout for stateful 2025-11-25 MCP SSE sessions. |
mcp.execution_timeout_seconds | None | 30 | MCP: Maximum execution duration for individual MCP tool calls. |
mcp.max_payload_bytes | None | 10485760 | MCP: Maximum allowed tool request and response payload size (10 MB). |
mcp.handle_ttl | None | 1h | MCP: Cache TTL for 256-bit progressive discovery tool handles (tool_hdl_...). |
a2a.enabled | None | true | A2A: Activates Linux Foundation Agent-to-Agent v1.0 protocol router. |
a2a.allow_http_binding | None | true | A2A: Enables standard REST HTTP+JSON binding on /a2a/v1/*. |
a2a.allow_jsonrpc_binding | None | true | A2A: Enables JSON-RPC 2.0 protocol binding on /a2a/v1/*. |
a2a.allow_legacy_card_discovery | None | false | A2A: Allows unsigned or legacy uncanonicalized agent card discovery. |
a2a.max_task_lifespan | None | 24h | A2A: Retention period for active and completed A2A task records. |
a2a.stream_buffer_capacity | None | 256 | A2A: Ring buffer event capacity for SSE task streaming subscribers. |
a2a.stream_publish_interval | GATEWAY_A2A_STREAM_PUBLISH_INTERVAL | 50ms | A2A: Polling interval for background task stream publication daemon. |
a2a.max_push_retries | GATEWAY_A2A_MAX_PUSH_RETRIES | 5 | A2A: Maximum webhook push notification delivery retries before Dead Lettering. |
a2a.signing_key_id | GATEWAY_A2A_SIGNING_KEY_ID | "gw-a2a-signer-v1" | A2A: Key ID for RFC 8785 JCS canonicalization and JWS Agent Card signing. |
a2a.allowed_signing_algs | None | ["ES256", "RS256", "EdDSA"] | A2A: Permitted cryptographic algorithms for incoming Agent Cards. |
a2a.trusted_card_jwks | None | {} | A2A: Map of upstream agent slug to JWKS URL for signature verification. |
files.storage_dir | GOROUTE_FILES_DIR | os.TempDir()/goroute-files | Files: Local disk directory for stateful tenant assets and batch files (0700 dir, 0600 file mode). |
providers.custom_openai_compatible[].id | None | Required | Custom Provider: Unique string identifier (e.g. "inhouse-vllm"). |
providers.custom_openai_compatible[].display_name | None | Required | Custom Provider: Human-readable label for logs and UI. |
providers.custom_openai_compatible[].base_url | None | Required | Custom Provider: Target endpoint base URL (e.g. http://vllm-cluster:8000/v1). |
providers.custom_openai_compatible[].supports_streaming | None | "true" | Custom Provider: Streaming capability override ("true", "false", "unknown"). |
providers.custom_openai_compatible[].supports_tools | None | "true" | Custom Provider: Function/tool calling capability override. |
providers.custom_openai_compatible[].supports_structured_outputs | None | "true" | Custom Provider: Structured outputs (response_format: json_schema) support. |
providers.custom_openai_compatible[].supports_list_models | None | true | Custom Provider: Whether upstream endpoint implements GET /models. |
providers.custom_openai_compatible[].default_headers | None | {} | Custom Provider: HTTP headers automatically attached to outbound calls. |
3. Model List & Root-Level Fallback Nesting
The model_list array defines the virtual routing topology presented to clients.
Root-Level fallbacks: [...] Nesting Architecture
Each entry in model_list defines:
model_name: The virtual alias or model identifier requested by the client (e.g.,"fast","reasoning","gpt-4o").routing_strategy: The balancing algorithm across sibling deployments ("weighted-shuffle","least-busy","latency-based","usage-based-headroom","lowest-cost","round-robin").fallbacks: Explicit root-level array of fallback model names triggered when this model group experiences an outage, exhausted retries, or rate limit rejections.model_params: Upstream target configuration (model,api_key,weight,rpm,tpm,azure_deployment_name, etc.).
Declarative Example: Multi-Tier Failover
model_list:
# Primary Tier: GPT-4o with cross-cloud fallbacks
- model_name: "gpt-4o"
routing_strategy: "weighted-shuffle"
fallbacks:
- "claude-3-5-sonnet"
- "gemini-2.5-pro"
model_params:
model: "openai/gpt-4o"
api_key: "os.environ/OPENAI_API_KEY"
weight: 100
rpm: 1000
tpm: 80000
# Fallback Tier 1: Claude 3.5 Sonnet
- model_name: "claude-3-5-sonnet"
routing_strategy: "latency-based"
fallbacks:
- "gemini-2.5-pro"
model_params:
model: "anthropic/claude-3-5-sonnet-20241022"
api_key: "os.environ/ANTHROPIC_API_KEY"
weight: 100
# Fallback Tier 2: Gemini 2.5 Pro
- model_name: "gemini-2.5-pro"
routing_strategy: "weighted-shuffle"
model_params:
model: "gemini/gemini-2.5-pro"
api_key: "os.environ/GEMINI_API_KEY"
weight: 100
# High-Throughput "fast" Group: Multi-Provider Load Balancing
- model_name: "fast"
routing_strategy: "least-busy"
fallbacks:
- "fast-backup"
model_params:
model: "gemini/gemini-2.5-flash"
api_key: "os.environ/GEMINI_API_KEY"
weight: 70
- model_name: "fast"
routing_strategy: "least-busy"
model_params:
model: "openai/gpt-4o-mini"
api_key: "os.environ/OPENAI_API_KEY"
weight: 30
# Emergency Local Backup
- model_name: "fast-backup"
routing_strategy: "weighted-shuffle"
model_params:
model: "ollama/llama3.3"
api_base: "http://localhost:11434"
weight: 100
[!IMPORTANT] The
fallbacks: [...]slice is declared at the root level of eachmodel_listentry alongsidemodel_nameandmodel_params. It must never be nested inside themodel_paramsobject. The router traverses fallbacks strictly in listed order, respecting circuit breaker states for each candidate before dispatching.