Skip to main content

OpenAI-Compatible Dialects

The modern AI ecosystem standardizes heavily on the OpenAI REST protocol (POST /v1/chat/completions, POST /v1/embeddings, GET /v1/models). However, individual vendors exhibit subtle dialect variations: differences in parameter naming (max_completion_tokens vs max_tokens), reasoning stream extraction (delta.reasoning_content), strict temperature limits, and custom HTTP authentication headers.

GoRoute includes a dedicated, certified OpenAI-Compatible Dialect Engine (internal/adapter/openaicompat/) containing 60+ pre-certified built-in and extended dialect profiles (builtin_profiles.go and extended_profiles.go) as part of the 73 ProviderType profiles registered in AllProviders() (internal/domain/inference.go), plus CustomOpenAICompatible for custom endpoints.


Dialect Engine Architecture​

The dialect engine is built around the DialectProfile and ProfileRegistry structs (internal/adapter/openaicompat/profile.go and registry.go):

Dialect Profile Pipeline Standard OpenAI Chat Request Client / SDK GoRoute Routing Engine Dialect Profile Engine(internal/adapter/openaicompat) Certified Upstream Provider(Groq, Cerebras, Together, vLLM, etc.) Parameter Translation(max_completion_tokens → max_tokens) Constraint Enforcement(Temperature range, TopP rejection) Authentication Strategy(Bearer / Header) Streaming Adapter(ExtractReasoningDelta)

Core Dialect Capabilities​

  1. Parameter Translation: Maps non-standard parameters dynamically (e.g. converting max_completion_tokens to max_tokens for providers that reject the newer OpenAI property).
  2. Reasoning Content Extraction (ExtractReasoningDelta): Intercepts streaming chunks containing extended thinking tokens (e.g., DeepSeek R1, Grok, SambaNova) and exposes them uniformly via delta.reasoning_content.
  3. Dialect Constraints: Automatically clamps or removes invalid parameters based on model family (e.g., stripping temperature and top_p for reasoning models like DeepSeek-R1 or Groq distill, or stripping stream_options when SupportsStreamOptions is false).
  4. Operation Enforcement (SupportedOperations): Profiles can declare explicit ProviderOperations (Chat, Completion, Responses, Embeddings, Rerank, etc.) enforced by OperationSupportChecker before dispatch.
  5. Zero-Latency Model Discovery: Handles GET /models probes dynamically when SupportsListModels: true.

Certified Provider Catalog​

The 73 ProviderType profiles supported across GoRoute (spanning native adapters and 60+ built-in/extended OpenAI-compatible profiles in builtin_profiles.go and extended_profiles.go, plus CustomOpenAICompatible) include the following core dialect categories:

1. Ultra-Fast Wafer-Scale & Hardware Acceleration​

Engineered for extreme token generation speeds (hundreds to thousands of tokens per second):

ProviderIDDefault Base URLReasoning DeltaToolsKey Models
Cerebrascerebrashttps://api.cerebras.ai/v1NoYesllama3.3-70b, llama3.1-8b
Groqgroqhttps://api.groq.com/openai/v1DistillYesllama-3.3-70b-versatile, mixtral-8x7b-32768
SambaNovasambanovahttps://api.sambanova.ai/v1YesYesMeta-Llama-3.3-70B-Instruct, DeepSeek-R1

2. High-Performance Serverless Inference​

Low-latency, cost-effective serverless inference providers:

ProviderIDDefault Base URLReasoning DeltaEmbeddingsKey Models
Fireworks AIfireworkshttps://api.fireworks.ai/inference/v1NoYesllama-v3p3-70b-instruct, qwen2p5-coder-32b-instruct
Together AItogetherhttps://api.together.xyz/v1NoYesmeta-llama/Llama-3.3-70B-Instruct-Turbo
DeepInfradeepinfrahttps://api.deepinfra.com/v1/openaiNoYesmeta-llama/Meta-Llama-3.1-70B-Instruct
Hyperbolichyperbolichttps://api.hyperbolic.xyz/v1YesNodeepseek-ai/DeepSeek-V3, DeepSeek-R1
Novita AInovitahttps://api.novita.ai/v3/openaiYesYesdeepseek/deepseek-r1, llama-3.3-70b
Friendli AIfriendlihttps://inference.friendli.ai/v1NoNometa-llama-3.1-70b-instruct
Nebius AI Studionebiushttps://api.studio.nebius.ai/v1YesNodeepseek-ai/DeepSeek-R1, meta-llama/Meta-Llama-3.1-70B
Scalewayscalewayhttps://api.scaleway.ai/v1NoNollama-3.3-70b-instruct
Lambda Labslambdahttps://api.lambdalabs.com/v1NoNohermes-3-llama-3.1-405b

3. Search & Reasoning​

Engines featuring integrated web search or deep multi-step reasoning capabilities:

ProviderIDDefault Base URLReasoning DeltaSpecial Capabilities
xAI (Grok)xaihttps://api.x.ai/v1YesGrok 2, Grok 2 Vision, reasoning output
Perplexity AIperplexityhttps://api.perplexity.aiNoLive citation generation (sonar-pro, sonar)
SiliconFlowsiliconflowhttps://api.siliconflow.cn/v1YesDeepSeek R1/V3, Qwen 2.5, BGE reranker & embeddings
DeepSeekdeepseekhttps://api.deepseek.com/v1YesDeepSeek-V3, DeepSeek-R1 reasoning models

4. Multi-Model Aggregators​

Dynamic routing across hundreds of upstream endpoints:

ProviderIDDefault Base URLReasoning DeltaDescription
OpenRouteropenrouterhttps://openrouter.ai/api/v1YesUniversal aggregator with auto-failover and model routing
Anyscaleanyscalehttps://api.endpoints.anyscale.com/v1NoScalable foundation model hosting powered by Ray

5. Self-Hosted & Local Inference Engines​

Run models on internal GPU clusters or local developer workstations:

ProviderIDDefault Base URLSupported Frameworks
vLLMvllmhttp://localhost:8000/v1PagedAttention, Tensor Parallelism, Guided Decoding
SGLangvllm (compatible)Custom URLRadixAttention for fast prefix caching
TensorRT-LLMvllm (compatible)Custom URLNVIDIA optimized execution engine
Basetenbasetenhttps://bridge.baseten.co/v1Truss-packaged custom model deployments
LMStudio / Jan / LocalAIollama or customLocal portDesktop developer sandboxes

6. Specialized Embeddings & Reranking​

Optimized exclusively for dense retrieval and ranking:

ProviderIDDefault Base URLModalityTop Models
Voyage AIvoyagehttps://api.voyageai.com/v1Embeddingsvoyage-3, voyage-3-large, voyage-code-2
Jina AIjinahttps://api.jina.ai/v1Embeddings & Rerankjina-embeddings-v3, jina-reranker-v2-base-multilingual

7. Enterprise Foundation Model Platforms​

Enterprise AI platforms with custom SLAs and hardware governance:

ProviderIDDefault Base URLModalityTop Models
NVIDIA NIMnvidiahttps://integrate.api.nvidia.com/v1Chat, Visionmeta/llama-3.3-70b-instruct, mistralai/mistral-large-2
Replicatereplicatehttps://api.replicate.com/v1Chat, MultiHosted open-source model deployments
AI21 Labsai21https://api.ai21.com/studio/v1Chatjamba-1.5-large, jamba-1.5-mini

8. Global Enterprise & Regional Providers​

Tier-1 cloud platforms across Asia, Europe, and North America:

ProviderIDDefault Base URLFeatures
Cloudflare Workers AIcloudflarehttps://api.cloudflare.com/client/v4/ai/v1Serverless edge inference across 300+ PoPs
Databricks Model ServingdatabricksConfigured per workspaceUnity Catalog governance and Mosaic AI
Snowflake CortexsnowflakeConfigured per accountIn-database LLM inference and search
Alibaba DashScopedashscopehttps://dashscope.aliyuncs.com/compatible-mode/v1Qwen 2.5 series (qwen-max, qwen-plus)
ByteDance Volcenginevolcenginehttps://ark.cn-beijing.volces.com/api/v3Doubao models, temperature clamped [0.0, 1.0]
Zhipu AI BigModelbigmodelhttps://open.bigmodel.cn/api/paas/v4GLM-4 series with web search
Moonshot AI (Kimi)moonshothttps://api.moonshot.cn/v1Long-context Kimi models (up to 2M tokens)
MiniMaxminimaxhttps://api.minimax.chat/v1MiniMax-Text-01 with reasoning extraction

Configuration Examples​

Dialect Configuration in configs/config.yaml​

model_list:
# 1. Cerebras Ultra-Fast Wafer-Scale
- model_name: ultra-fast
model_params:
model: cerebras/llama3.3-70b
api_key: os.environ/CEREBRAS_API_KEY
weight: 100

# 2. xAI Grok with Reasoning
- model_name: grok-chat
model_params:
model: xai/grok-2-1212
api_key: os.environ/XAI_API_KEY

# 3. Together AI with Temperature Clamping
- model_name: together-llama
model_params:
model: together/meta-llama/Llama-3.3-70B-Instruct-Turbo
api_key: os.environ/TOGETHER_API_KEY

# 4. DeepSeek R1 with Reasoning Delta Extraction
- model_name: deepseek-reasoner
model_params:
model: deepseek/deepseek-reasoner
api_key: os.environ/DEEPSEEK_API_KEY

# 5. NVIDIA NIM API
- model_name: nim-llama
model_params:
model: nvidia/meta/llama-3.3-70b-instruct
api_key: os.environ/NVIDIA_API_KEY

# 6. Voyage AI Dedicated Embeddings
- model_name: voyage-embed
model_params:
model: voyage/voyage-3
api_key: os.environ/VOYAGE_API_KEY

Reasoning Delta Extraction (ExtractReasoningDelta)​

When calling reasoning-oriented models (such as DeepSeek-R1, Grok-2, or SambaNova's R1 endpoint) via streaming, the provider returns reasoning thoughts in a non-standard JSON delta field:

{"choices": [{"delta": {"reasoning_content": "Let's first decompose the algorithm into three phases..."}}]}

GoRoute's dialect adapter automatically detects ExtractReasoningDelta: true, preserves the field, and streams it to the client:

curl -N -X POST http://localhost:8080/v1/chat/completions \
-H "Authorization: Bearer your-gateway-virtual-key" \
-H "Content-Type: application/json" \
-d '{
"model": "deepseek-reasoner",
"messages": [{"role": "user", "content": "Solve the 8-queens problem with backtracking."}],
"stream": true
}'
data: {"id":"chatcmpl-912","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":"Phase 1: Board representation..."}}]}
...
data: {"id":"chatcmpl-912","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"Here is the Python implementation:"}}]}

Dialect Parameter Clamping & Overrides​

Certain providers enforce strict constraints on parameters:

  • Together AI: Restricts temperature strictly between 0.0 and 1.0. Any request with temperature: 1.5 is clamped to 1.0 before transmission.
  • DeepSeek R1 / Grok Distill: Rejects requests that pass custom temperature or top_p. The dialect adapter strips these parameters cleanly to prevent upstream HTTP 400 errors.
  • Standard Param Map: Translates max_completion_tokens to max_tokens for providers that have not adopted OpenAI's August 2024 spec changes.

[!TIP] If you run a custom internal model that exhibits idiosyncratic parameter requirements, configure a profile in providers.custom_openai_compatible in configs/config.yaml. See the Custom Provider Guide for syntax details.