Skip to main content

Gateway Data Plane API Reference

GoRoute AI Gateway exposes an OpenAI-compatible data plane supporting 24 primary /v1 paths across 73 ProviderType profiles (plus CustomOpenAICompatible). All routes execute with sub-2ms gateway overhead, automated circuit breaking, cross-provider fallbacks, and real-time micro-dollar accounting.

[!NOTE] All data plane routes accept standard OpenAI SDKs (Python, TypeScript, Go) and raw HTTP clients. Set your client base_url to http://localhost:8080/v1 and supply a Virtual API Key (sk-gw-...) via the Authorization header.


Authentication & Standard Telemetry Headers​

Every data plane request requires a tenant-scoped Virtual API Key:

Authorization: Bearer sk-gw-prod-7f9a2b8c4d1e

Standard Gateway Response Headers​

Every response returned by GoRoute includes rich tracing, routing, and FinOps telemetry:

HeaderDescriptionExample
X-Gateway-Request-IdUnique UUID assigned to the request lifecyclereq_991efda4-7fab-4b03
X-Trace-IdW3C / OpenTelemetry distributed trace identifierbad98e7deea34d7c810a963d9ed2c4ef
X-Gateway-ProviderThe winning upstream provider selected for executiongemini, anthropic, openai
X-Gateway-ModelThe physical upstream model that processed the requestgemini-2.5-flash, claude-3-5-sonnet-20241022
X-Gateway-Cost-MicrosExact micro-dollar spend incurred ($0.000001 USD)45 ($0.000045 USD)
X-Gateway-Routing-TimeGateway Stage 1 + Stage 2 routing overhead14 µs
X-Gateway-CacheExact-match cache evaluation statusHIT or MISS
Retry-AfterCooldown period sent on HTTP 429 rate limit errors45 (seconds)

Direct Dynamic provider/model Routing​

In addition to logical aliases (fast, reasoning), GoRoute allows clients to bypass model groups and target upstream models directly using provider/model syntax (e.g., anthropic/claude-3-5-sonnet-20241022, cerebras/llama-3.3-70b, bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0, vertex/gemini-2.5-pro):

  • Layer 2 Provider Subscription Check: The caller's organization must have an active subscription for the target provider.
  • Layer 3 Model Policy Check: The requested model must not be disabled by the organization's model availability policy.
  • Tenant Credential Isolation: The organization must have an active credential for the target provider stored in the vault. Host environment variables are never leaked across tenant boundaries.
  • Catalog Provisioning: The gateway may provision model records from its catalog when needed. Catalog contents and discovery behavior depend on the gateway build and configuration; a provider-side model ID is not automatically available to every tenant or key.
  • Fail-Closed Boundary: If repository queries fail or the model is unconfigured/unauthorized, GoRoute fails closed and returns HTTP 404 model_group_not_found (explicit model requests never silently fall back to unrelated tenant model groups).

Summary of All 24 Primary /v1 Data Plane Paths​

HTTP MethodRouteDescription
POST/v1/chat/completionsMulti-provider synchronous & SSE streaming chat inference
POST/v1/completionsLegacy prompt text completions
POST/v1/embeddingsDense vector embeddings generation
GET/v1/modelsTenant-filtered model catalogue and pricing inventory
POST/v1/rerankDocument semantic reranking via Cohere v2 / Jina AI
POST/v1/moderationsAutomated content moderation and safety scoring
POST/v1/messagesNative Anthropic Messages API ingress
POST/v1/messages/count_tokensAnthropic-compatible preflight token counting
POST/v1/tokenizeIn-process fast heuristic token estimation
POST/v1/audio/transcriptionsSpeech-to-text transcription with Whisper disk spooling
POST/v1/audio/translationsSpeech-to-text translation to English
POST/v1/audio/speechText-to-speech synthesis (TTS)
POST/v1/images/generationsImage generation (DALL-E 2/3, Imagen)
POST/v1/images/editsImage inpainting with mask uploads
POST/v1/images/variationsImage variation generation
POST/v1/filesMultipart file upload with SHA-256 deduplication
GET/v1/filesList tenant files
GET/v1/files/{file_id}Retrieve file metadata
GET/v1/files/{file_id}/contentDownload file stream
DELETE/v1/files/{file_id}Delete stateful tenant file
POST/v1/batchesCreate asynchronous 24-hr batch job (50% discount)
GET/v1/batchesList tenant batch jobs
GET/v1/batches/{batch_id}Inspect batch processing state and item counts
POST/v1/batches/{batch_id}/cancelAbort in-flight batch execution
POST/v1/responsesOpenAI Responses API translation bridge
GET/v1/realtimeFull-duplex OpenAI Realtime WebSocket session
POST/v1/guardrails/applyStandalone Pipeline v2 guardrail evaluation without upstream LLM call

1. Chat Completions (POST /v1/chat/completions)​

Standardized endpoint supporting Google Gemini, Anthropic Claude, OpenAI, Mistral, and local Ollama models in synchronous JSON or real-time Server-Sent Events (SSE) streaming mode.

Synchronous Request​

curl -X POST http://localhost:8080/v1/chat/completions \
-H "Authorization: Bearer sk-gw-prod-7f9a2b8c4d1e" \
-H "Content-Type: application/json" \
-d '{
"model": "fast",
"messages": [
{"role": "system", "content": "You are a concise Go systems engineer."},
{"role": "user", "content": "How do goroutines manage stack growth?"}
],
"temperature": 0.7,
"max_tokens": 512,
"stream": false
}'

Synchronous Response (HTTP 200 OK)​

{
"id": "chatcmpl-req_991efda4-7fab-4b03",
"object": "chat.completion",
"created": 1731081600,
"model": "gemini-2.5-flash",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "Goroutines start with a small stack (typically 2KB in modern Go). When stack space is exhausted, the runtime allocates a contiguous block twice the size and copies the existing stack over."
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 24,
"completion_tokens": 42,
"total_tokens": 66
}
}

Real-Time SSE Stream (stream: true)​

When stream: true is set, GoRoute initiates a zero-buffer Server-Sent Events stream:

HTTP/1.1 200 OK
Content-Type: text/event-stream
Cache-Control: no-cache
Connection: keep-alive

data: {"id":"chatcmpl-req_1","object":"chat.completion.chunk","created":1731081600,"model":"gemini-2.5-flash","choices":[{"index":0,"delta":{"role":"assistant","content":"Goroutines"},"finish_reason":null}]}

data: {"id":"chatcmpl-req_1","object":"chat.completion.chunk","created":1731081600,"model":"gemini-2.5-flash","choices":[{"index":0,"delta":{"content":" start"},"finish_reason":null}]}

data: {"id":"chatcmpl-req_1","object":"chat.completion.chunk","created":1731081600,"model":"gemini-2.5-flash","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: [DONE]

Extended Reasoning & Parameter Normalization​

GoRoute transparently normalizes requests targeting frontier reasoning models:

  • Reasoning Effort (reasoning_effort: "low" | "medium" | "high"):
    • OpenAI Reasoning (o1, o3, o3-mini): Converts role: "system" to role: "developer", translates max_tokens to max_completion_tokens, and automatically strips unsupported sampling parameters (temperature, top_p).
    • Anthropic Claude: Enables extended thinking with token budgets (1024, 4096, 16384) and omits temperature/top_p as mandated by Anthropic.
    • Google Gemini: Sets thinkingConfig.thinkingBudget for Gemini 2.5 thinking models.
  • Anthropic Role Alternation on Tools: When returning multiple parallel tool outputs (role: "tool"), GoRoute automatically consolidates consecutive tool results into a single user message with multiple tool_result content blocks to satisfy Anthropic's strict alternating turn protocol.

2. Legacy Text Completions (POST /v1/completions)​

OpenAI-compatible prompt completion endpoint supporting single string prompts or string arrays.

cURL Request​

curl -X POST http://localhost:8080/v1/completions \
-H "Authorization: Bearer sk-gw-prod-7f9a2b8c4d1e" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-3.5-turbo-instruct",
"prompt": "Write a haiku about Go channels",
"max_tokens": 64,
"temperature": 0.5
}'

Response (HTTP 200 OK)​

{
"id": "cmpl-req_7721a9c4",
"object": "text_completion",
"created": 1731081600,
"model": "gpt-3.5-turbo-instruct",
"choices": [
{
"text": "\nMessages flow like streams,\nGoroutines pass without lock,\nSync in harmony.",
"index": 0,
"logprobs": null,
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 7,
"completion_tokens": 19,
"total_tokens": 26
}
}

3. Vector Embeddings (POST /v1/embeddings)​

Generates dense vector embeddings across single strings or string arrays, with optional encoding formats (float or base64) and dimension truncation.

cURL Request​

curl -X POST http://localhost:8080/v1/embeddings \
-H "Authorization: Bearer sk-gw-prod-7f9a2b8c4d1e" \
-H "Content-Type: application/json" \
-d '{
"model": "text-embedding-3-small",
"input": ["GoRoute AI Gateway is ultra-fast", "Pure Go architecture"],
"encoding_format": "float"
}'

Response (HTTP 200 OK)​

{
"object": "list",
"data": [
{
"object": "embedding",
"index": 0,
"embedding": [0.0123, -0.0456, 0.0891, -0.0012]
},
{
"object": "embedding",
"index": 1,
"embedding": [0.0341, -0.0128, 0.0712, 0.0054]
}
],
"model": "text-embedding-3-small",
"usage": {
"prompt_tokens": 14,
"total_tokens": 14
}
}

4. Model Catalogue Inventory (GET /v1/models)​

Returns the tenant-filtered inventory of models accessible to the caller's Virtual API Key, evaluated against the 4-layer model access control hierarchy. Use this endpoint to check which model IDs are exposed to your caller. Provider catalogs change over time, and this response is deployment-specific; it is not a universal list of all models offered by upstream providers. The response may include capability and pricing metadata where available.

cURL Request​

curl -X GET http://localhost:8080/v1/models \
-H "Authorization: Bearer sk-gw-prod-7f9a2b8c4d1e"

Illustrative response (HTTP 200 OK)​

The model and pricing values below are sample data, not a current provider catalog or an availability guarantee.

{
"object": "list",
"data": [
{
"id": "openai/gpt-4o",
"object": "model",
"created": 1731081600,
"owned_by": "openai",
"display_name": "OpenAI GPT-4o",
"max_input_tokens": 128000,
"max_output_tokens": 16384,
"capabilities": {
"tools": "supported",
"streaming": "supported",
"vision": "supported",
"reasoning": "unsupported",
"structured_outputs": "supported",
"embeddings": "unsupported"
},
"supports_streaming": true,
"supports_tools": true,
"supports_vision": true,
"supports_reasoning": false,
"supports_embeddings": false,
"input_cost_micros_per_1m": 2500000,
"output_cost_micros_per_1m": 10000000,
"cached_input_cost_micros_per_1m": 1250000,
"pricing_status": "verified",
"pricing_provenance_source": "official-vendor-pricing-2025",
"pricing_effective_date": "2025-05-01",
"lifecycle_status": "active"
}
]
}

5. Document Reranking (POST /v1/rerank)​

Calculates semantic relevance scores between a search query and a list of text documents. Integrates with Cohere v2 API and Jina AI rerankers. Aliased at /rerank and /v2/rerank.

cURL Request​

curl -X POST http://localhost:8080/v1/rerank \
-H "Authorization: Bearer sk-gw-prod-7f9a2b8c4d1e" \
-H "Content-Type: application/json" \
-d '{
"model": "rerank-v3.5",
"query": "What is the capital of France?",
"documents": [
"Carson City is the capital city of Nevada.",
"The capital of France is Paris.",
"Tokyo is the capital of Japan."
],
"top_n": 2,
"return_documents": true
}'

Response (HTTP 200 OK)​

{
"id": "rerank-req_4ebe7b01",
"model": "rerank-v3.5",
"provider": "cohere",
"results": [
{
"index": 1,
"relevance_score": 0.9984,
"document": { "text": "The capital of France is Paris." }
},
{
"index": 0,
"relevance_score": 0.1245,
"document": { "text": "Carson City is the capital city of Nevada." }
}
],
"usage": {
"search_units": 1,
"total_tokens": 42
}
}

6. Content Moderation (POST /v1/moderations)​

Evaluates text against safety categories using upstream moderation models (e.g. OpenAI omni-moderation-latest). Aliased at /moderations.

cURL Request​

curl -X POST http://localhost:8080/v1/moderations \
-H "Authorization: Bearer sk-gw-prod-7f9a2b8c4d1e" \
-H "Content-Type: application/json" \
-d '{
"model": "omni-moderation-latest",
"input": ["GoRoute AI Gateway is safe and reliable."]
}'

Response (HTTP 200 OK)​

{
"id": "modr-req_865bcf43",
"model": "omni-moderation-latest",
"results": [
{
"flagged": false,
"categories": {
"hate": false,
"violence": false,
"self-harm": false,
"sexual": false
},
"category_scores": {
"hate": 0.00001,
"violence": 0.00002,
"self-harm": 0.00001,
"sexual": 0.00003
}
}
]
}

7. Anthropic Messages & Token Counting​

Native Anthropic ingress bridges allowing Anthropic SDKs to route directly through GoRoute.

Anthropic Messages (POST /v1/messages)​

curl -X POST http://localhost:8080/v1/messages \
-H "Authorization: Bearer sk-gw-prod-7f9a2b8c4d1e" \
-H "Content-Type: application/json" \
-d '{
"model": "claude-3-5-sonnet-20241022",
"max_tokens": 1024,
"system": "You are a concise assistant.",
"messages": [
{"role": "user", "content": "Explain goroutines in 2 sentences."}
]
}'

Anthropic Token Counting (POST /v1/messages/count_tokens)​

Preflight token calculation conforming to Anthropic's count tokens schema:

curl -X POST http://localhost:8080/v1/messages/count_tokens \
-H "Authorization: Bearer sk-gw-prod-7f9a2b8c4d1e" \
-H "Content-Type: application/json" \
-d '{
"model": "claude-3-5-sonnet-20241022",
"messages": [
{"role": "user", "content": "Explain goroutines in 2 sentences."}
]
}'

Response (HTTP 200 OK)​

{
"input_tokens": 16
}

8. Fast Heuristic Tokenization (POST /v1/tokenize)​

Zero-latency, in-process token counter approximating BPE without network calls. Aliased at /tokenize.

cURL Request​

curl -X POST http://localhost:8080/v1/tokenize \
-H "Authorization: Bearer sk-gw-prod-7f9a2b8c4d1e" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-4o",
"prompt": "GoRoute AI Gateway provides ultra-fast routing and FinOps accounting."
}'

Response (HTTP 200 OK)​

{
"count": 13,
"model": "gpt-4o",
"profile": "cl100k_base",
"total_tokens": 13
}

9. Audio STT & TTS (/v1/audio/*)​

Supports multipart audio uploads spooled to disk for zero-memory retryability, with micro-dollar FinOps billing based on integer duration seconds and character count.

Audio Transcription (POST /v1/audio/transcriptions)​

curl -X POST http://localhost:8080/v1/audio/transcriptions \
-H "Authorization: Bearer sk-gw-prod-7f9a2b8c4d1e" \
-F "file=@meeting_audio.mp3" \
-F "model=whisper-1" \
-F "language=en"

Audio Translation (POST /v1/audio/translations)​

Translates foreign audio files into English text:

curl -X POST http://localhost:8080/v1/audio/translations \
-H "Authorization: Bearer sk-gw-prod-7f9a2b8c4d1e" \
-F "file=@french_speech.mp3" \
-F "model=whisper-1"

Audio Speech Synthesis (POST /v1/audio/speech)​

Synthesizes text into streaming audio (formats: mp3, opus, aac, flac, wav):

curl -X POST http://localhost:8080/v1/audio/speech \
-H "Authorization: Bearer sk-gw-prod-7f9a2b8c4d1e" \
-H "Content-Type: application/json" \
-d '{
"model": "tts-1",
"input": "GoRoute AI Gateway deployed in production.",
"voice": "alloy",
"response_format": "mp3"
}' --output speech.mp3

10. Image Generation, Edits & Variations (/v1/images/*)​

Supports DALL-E 2, DALL-E 3, and Imagen models, returning url or b64_json payloads with exact micro-dollar pricing matrices keyed to quality (standard vs hd) and dimensions (1024x1024, 1024x1792, 1792x1024).

Image Generation (POST /v1/images/generations)​

curl -X POST http://localhost:8080/v1/images/generations \
-H "Authorization: Bearer sk-gw-prod-7f9a2b8c4d1e" \
-H "Content-Type: application/json" \
-d '{
"model": "dall-e-3",
"prompt": "Futuristic cloud architecture diagram rendered in neon blue",
"size": "1024x1024",
"quality": "standard",
"n": 1
}'

Image Edits / Inpainting (POST /v1/images/edits)​

curl -X POST http://localhost:8080/v1/images/edits \
-H "Authorization: Bearer sk-gw-prod-7f9a2b8c4d1e" \
-F "image=@base_image.png" \
-F "mask=@mask.png" \
-F "prompt": "Add a glowing golden trophy on the table" \
-F "model=dall-e-2"

Image Variations (POST /v1/images/variations)​

curl -X POST http://localhost:8080/v1/images/variations \
-H "Authorization: Bearer sk-gw-prod-7f9a2b8c4d1e" \
-F "image=@logo.png" \
-F "model=dall-e-2" \
-F "n=2"

11. Stateful Tenant Files (/v1/files/*)​

Multi-tenant secure file storage with 0700 directory and 0600 file modes, automatic streaming SHA-256 deduplication indexing, and PostgreSQL metadata.

Upload File (POST /v1/files)​

curl -X POST http://localhost:8080/v1/files \
-H "Authorization: Bearer sk-gw-prod-7f9a2b8c4d1e" \
-F "file=@batch_requests.jsonl" \
-F "purpose=batch"

Response (HTTP 201 Created):

{
"id": "file-8f91a2bc-3d4e",
"object": "file",
"bytes": 1048576,
"created_at": 1731081600,
"filename": "batch_requests.jsonl",
"purpose": "batch",
"status": "uploaded"
}

List Files (GET /v1/files)​

curl http://localhost:8080/v1/files?purpose=batch \
-H "Authorization: Bearer sk-gw-prod-7f9a2b8c4d1e"

Retrieve File Metadata (GET /v1/files/{file_id})​

curl http://localhost:8080/v1/files/file-8f91a2bc-3d4e \
-H "Authorization: Bearer sk-gw-prod-7f9a2b8c4d1e"

Download File Stream (GET /v1/files/{file_id}/content)​

curl http://localhost:8080/v1/files/file-8f91a2bc-3d4e/content \
-H "Authorization: Bearer sk-gw-prod-7f9a2b8c4d1e" \
--output downloaded_batch.jsonl

Delete File (DELETE /v1/files/{file_id})​

curl -X DELETE http://localhost:8080/v1/files/file-8f91a2bc-3d4e \
-H "Authorization: Bearer sk-gw-prod-7f9a2b8c4d1e"

12. Asynchronous Batches (/v1/batches/*)​

Processes JSONL line items asynchronously within a 24-hour completion window for a 50% discount. State transitions: validating → in_progress → completed | failed | cancelled.

Create Batch (POST /v1/batches)​

curl -X POST http://localhost:8080/v1/batches \
-H "Authorization: Bearer sk-gw-prod-7f9a2b8c4d1e" \
-H "Content-Type: application/json" \
-d '{
"input_file_id": "file-8f91a2bc-3d4e",
"endpoint": "/v1/chat/completions",
"completion_window": "24h",
"metadata": { "job": "nightly_summaries" }
}'

Response (HTTP 201 Created):

{
"id": "batch-3c91a0b1-4f2e",
"object": "batch",
"endpoint": "/v1/chat/completions",
"input_file_id": "file-8f91a2bc-3d4e",
"completion_window": "24h",
"status": "validating",
"created_at": 1731081600,
"request_counts": {
"total": 0,
"completed": 0,
"failed": 0
}
}

Inspect Batch (GET /v1/batches/{batch_id})​

curl http://localhost:8080/v1/batches/batch-3c91a0b1-4f2e \
-H "Authorization: Bearer sk-gw-prod-7f9a2b8c4d1e"

Cancel Batch (POST /v1/batches/{batch_id}/cancel)​

curl -X POST http://localhost:8080/v1/batches/batch-3c91a0b1-4f2e/cancel \
-H "Authorization: Bearer sk-gw-prod-7f9a2b8c4d1e"

13. OpenAI Responses API (POST /v1/responses)​

Stateless translation bridge conforming to the OpenAI Responses API schema (input, instructions), routing through GoRoute's 7-stage reliability pipeline.

cURL Request​

curl -X POST http://localhost:8080/v1/responses \
-H "Authorization: Bearer sk-gw-prod-7f9a2b8c4d1e" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-4o",
"input": "Summarize quantum computing principles",
"instructions": "Be concise, use bullet points",
"stream": true
}'

SSE Stream Format​

Emits Responses events (response.created, response.output_item.added, response.output_text.delta, response.output_text.done, response.output_item.done, response.completed, response.done):

event: response.created
data: {"response":{"id":"resp_0192a","status":"in_progress","model":"gpt-4o"}}

event: response.output_text.delta
data: {"output_index":0,"delta":"Quantum computing utilizes superposition and entanglement."}

event: response.completed
data: {"response":{"id":"resp_0192a","status":"completed"}}

event: response.done
data: {}

14. OpenAI Realtime WebSocket (GET /v1/realtime)​

Full-duplex bidirectional WebSocket session handler supporting real-time speech and text streaming. Backed by nhooyr.io/websocket connection hijacking.

Connecting via WebSocket​

Clients connect to ws://localhost:8080/v1/realtime?model=gpt-4o-realtime-preview passing authentication via Bearer header or subprotocol:

const ws = new WebSocket(
'ws://localhost:8080/v1/realtime?model=gpt-4o-realtime-preview',
['realtime', 'openai-insecure-api-key.sk-gw-prod-7f9a2b8c4d1e']
);

ws.onopen = () => {
ws.send(JSON.stringify({
type: 'response.create',
response: {
modalities: ['text'],
instructions: 'You are a real-time speech assistant.'
}
}));
};

Next Steps​