Skip to main content

Legacy Text Completions API

While modern generative LLM architectures predominantly utilize the conversational /v1/chat/completions schema, GoRoute AI Gateway maintains first-class support for the classic Text Completions API (POST /v1/completions). This endpoint is essential for base foundation models, instruction-following base models (such as gpt-3.5-turbo-instruct), code-infilling engines, and legacy pipelines.


Endpoint Specification​

  • HTTP Method: POST
  • Supported Paths:
    • /v1/completions
    • /completions (Unversioned alias)
  • Authentication: Bearer sk-gw-... (Virtual Key or Master Admin Key)
  • Content-Type: application/json

Request & Response Schemas​

Request Schema (CanonicalCompletionRequest)​

FieldTypeRequiredDescription
modelstringYesTarget model (e.g. gpt-3.5-turbo-instruct, mistral-tiny).
promptstring | string[]YesThe prompt(s) to generate completions for. Can be a single string or an array of strings.
max_tokensintegerNoMaximum number of tokens to generate.
temperaturefloatNoSampling temperature between 0.0 and 2.0.
top_pfloatNoNucleus sampling probability cutoff (alternative to temperature).
streambooleanNoWhether to stream back partial progress via Server-Sent Events (SSE).
stopstring[]NoUp to 4 sequences where the model will cease generating further tokens.
userstringNoA unique identifier representing your end-user for abuse monitoring.

[!NOTE] GoRoute's JSON parser uses an internal polymorphic unmarshaler (CompletionPrompt), transparently accepting either "prompt": "Write a poem" or "prompt": ["First prompt", "Second prompt"].

Response Schema (CanonicalCompletionResponse)​

{
"id": "cmpl-8a19b3d0-994c-4e89-9fa2-5813f02e6a12",
"model": "gpt-3.5-turbo-instruct",
"provider": "openai",
"choices": [
{
"index": 0,
"text": "\n\nConcurrency in Go is built on goroutines and channels.",
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 8,
"completion_tokens": 12,
"total_tokens": 20
}
}

Code Examples​

# Single string prompt
curl -X POST http://localhost:8080/v1/completions \
-H "Authorization: Bearer sk-gw-prod-7f9a2b8c4d1e" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-3.5-turbo-instruct",
"prompt": "Complete the sentence: In computer science, an idempotent operation is",
"max_tokens": 64,
"temperature": 0.2
}'

# Array prompt batching
curl -X POST http://localhost:8080/v1/completions \
-H "Authorization: Bearer sk-gw-prod-7f9a2b8c4d1e" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-3.5-turbo-instruct",
"prompt": [
"def fibonacci(n):",
"def quicksort(arr):"
],
"max_tokens": 128,
"temperature": 0.0
}'

Token Quota & FinOps Ledger Integration​

Requests entering /v1/completions follow the identical multi-layer FinOps lifecycle as chat completions:

  1. Pre-Flight Estimation: Tokens are estimated as PromptChars / 4 (minimum 10 tokens) and reserved via the sliding-window rate limiter.
  2. Post-Flight Reconciliation: When upstream execution finishes, exact prompt and completion token counts from Usage are reconciled with the rate limiter.
  3. Micro-Dollar Ledger: An immutable SpendEvent is committed asynchronously with cost calculated using 64-bit ceiling integer arithmetic:
Cost (µ$) = ⌈ (PromptTokens * InputRate + CompletionTokens * OutputRate) / 1,000,000 ⌉

[!TIP] If your application is migrating from legacy base models to instruction-tuned chat models, consider switching to /v1/chat/completions for broader multi-provider routing and structured tool calling.