Skip to main content

Chat Completions API

The GoRoute Data Plane provides a 100% OpenAI-compatible Chat Completions endpoint standardizing request parameters, tool calls, and real-time SSE streaming across all upstream providers (OpenAI, Anthropic Claude, Google Gemini, Mistral, Cerebras, Groq, DeepSeek, and local Ollama models).


Endpoint Specification​

  • Method: POST
  • Path: /v1/chat/completions (also supports unversioned /chat/completions)
  • Authentication: Bearer sk-gw-... (Virtual Key or Master Admin Key)
  • Content-Type: application/json

Key Capabilities & Enterprise Extensions​

1. Dynamic Provider & Model Routing​

Beyond predefined logical aliases (fast, reasoning, coding), GoRoute supports direct dynamic wildcard dispatch via provider/model syntax:

{
"model": "cerebras/llama-3.3-70b",
"messages": [{"role": "user", "content": "Explain kernel bypass networking."}]
}

Other direct provider routing examples:

  • anthropic/claude-3-5-sonnet
  • gemini/gemini-2.5-flash
  • groq/llama-3.3-70b-versatile
  • deepseek/deepseek-chat

[!NOTE] Direct provider routing evaluates your organization's Layer 2 Provider Subscriptions, Layer 3 Org Model Policies, and Virtual Key allowlists before dynamically provisioning or dispatching to the target provider.

2. Reasoning Effort & Thinking Budgets​

For reasoning models (OpenAI o1/o3/o4, Anthropic Claude 3.7 Sonnet, Google Gemini 2.5), GoRoute accepts standard reasoning_effort:

{
"model": "reasoning",
"messages": [{"role": "user", "content": "Solve this distributed systems puzzle."}],
"reasoning_effort": "high"
}

The gateway standardizes reasoning across providers:

  • Anthropic Claude: Maps "low", "medium", and "high" to explicit thinking: {"type": "enabled", "budget_tokens": N} (1024, 4096, or 16384 tokens).
  • Google Gemini 2.5: Injects thinkingConfig.thinkingBudget accordingly.
  • OpenAI o1/o3: Passes native reasoning_effort.
  • Pre-Flight Estimation: Reserves an additional 1,024 tokens during pre-flight rate limiting when reasoning_effort is detected.

3. Strict Role Alternation & Tool Consolidation​

Anthropic and Google Gemini mandate strict turn alternation between user and assistant. In multi-tool calling flows, OpenAI returns multiple consecutive role: "tool" messages for parallel calls.

When routing to Anthropic or Gemini, GoRoute automatically consolidates consecutive tool results into a single user turn with multiple tool_result content blocks. You never need to hand-craft vendor-specific turn structures.


Code Examples​

# 1. Synchronous Chat Completion with direct provider routing
curl -X POST http://localhost:8080/v1/chat/completions \
-H "Authorization: Bearer sk-gw-prod-7f9a2b8c4d1e" \
-H "Content-Type: application/json" \
-d '{
"model": "cerebras/llama-3.3-70b",
"messages": [
{"role": "system", "content": "You are a concise systems architect."},
{"role": "user", "content": "Why is DPDK faster than traditional kernel socket I/O?"}
],
"temperature": 0.2,
"max_tokens": 256
}'

# 2. Hybrid Reasoning with thinking effort on Claude 3.7
curl -X POST http://localhost:8080/v1/chat/completions \
-H "Authorization: Bearer sk-gw-prod-7f9a2b8c4d1e" \
-H "Content-Type: application/json" \
-d '{
"model": "anthropic/claude-3-5-sonnet",
"messages": [
{"role": "user", "content": "Prove that Raft election safety holds in asynchronous networks."}
],
"reasoning_effort": "high"
}'

# 3. Server-Sent Events (SSE) Streaming
curl -X POST http://localhost:8080/v1/chat/completions \
-H "Authorization: Bearer sk-gw-prod-7f9a2b8c4d1e" \
-H "Content-Type: application/json" \
-d '{
"model": "fast",
"messages": [{"role": "user", "content": "Stream a haiku about distributed consensus."}],
"stream": true
}'

Gateway Telemetry Response Headers​

Every chat completion response includes detailed telemetry headers for end-to-end tracing and auditing:

HeaderDescriptionExample
X-Gateway-Request-IdUnique UUID assigned to the request lifecyclereq_fb502e5e-b39a-4d9a
X-Trace-IdW3C / OpenTelemetry distributed trace ID8d380107e03c29fd8d979bbc660a10e9
X-Gateway-ProviderThe winning upstream provider selectedcerebras, anthropic, gemini
X-Gateway-ModelThe physical upstream model executedllama-3.3-70b, claude-3-5-sonnet
X-Gateway-Cost-MicrosMicro-dollar spend incurred ($0.000001 USD)13 ($0.000013 USD)
X-Gateway-Routing-TimeGateway Stage 1 + Stage 2 routing overhead12 µs
X-Gateway-CacheCache evaluation resultHIT or MISS
Retry-AfterCooldown period sent on HTTP 429 rate limit45 (seconds)

[!TIP] When X-Gateway-Cache: HIT is returned, X-Gateway-Cost-Micros is 0, confirming that the response was served from cache without upstream provider token charges.