Skip to main content

Document Reranking API

The GoRoute Data Plane provides a unified semantic reranking API compatible with Cohere v2 (/v2/rerank) and Jina AI. Reranking takes an initial set of candidate documents retrieved via lexical or vector search and reorders them using cross-encoder deep neural models for optimal relevance against a search query.


Endpoint Specification​

  • HTTP Methods: POST
  • Supported Paths:
    • /v1/rerank (Standardized gateway endpoint)
    • /rerank (Unversioned compatibility alias)
    • /v2/rerank (Cohere v2 wire specification alias)
  • Authentication: Bearer sk-gw-... (Virtual Key or Master Admin Key)
  • Content-Type: application/json

Request & Response Schemas​

Request Schema (CanonicalRerankRequest)​

FieldTypeRequiredDescription
modelstringYesTarget reranker model or provider/model (e.g. cohere/rerank-v3.5, jina/jina-reranker-v2-base-multilingual).
querystringYesThe search query against which candidate documents are scored.
documentsarrayYesCandidate documents. Accepts either an array of plain strings (string[]) or an array of arbitrary JSON objects (object[]).
top_nintegerNoMaximum number of top-ranked results to return. If omitted, all documents are returned in ranked order.
rank_fieldsstring[]NoWhen documents contains structured objects, specifies which JSON keys to consider for relevance ranking (e.g. ["title", "content"]).
return_documentsbooleanNoWhether to return the document payload inside each result item. Default: false.
max_tokens_per_docintegerNoMaximum tokens per document to evaluate. Documents exceeding this length will be truncated by the upstream cross-encoder.

Response Schema (CanonicalRerankResponse)​

{
"id": "rerank_c7b5f90a-1123-4f9e-8c3b-280145c2ee01",
"model": "rerank-v3.5",
"provider": "cohere",
"results": [
{
"index": 1,
"relevance_score": 0.9842,
"document": {
"text": "GoRoute AI Gateway is built with pure Go 1.24 and avoids Cgo overhead."
}
},
{
"index": 0,
"relevance_score": 0.3129,
"document": {
"text": "The solar system contains eight major planets orbiting the Sun."
}
}
],
"meta": {
"api_version": {
"version": "2"
},
"billed_units": {
"search_units": 1
}
},
"usage": {
"search_units": 1,
"total_tokens": 84
}
}

[!NOTE] The index field in each result item corresponds to the original 0-based position of the document in the input documents array, allowing zero-copy mapping back to your local candidate cache.


Code Examples​

# Plain string documents
curl -X POST http://localhost:8080/v1/rerank \
-H "Authorization: Bearer sk-gw-prod-7f9a2b8c4d1e" \
-H "Content-Type: application/json" \
-d '{
"model": "cohere/rerank-v3.5",
"query": "high performance zero allocation proxy",
"documents": [
"Traditional Python proxies incur GIL locking and GC latency pauses.",
"GoRoute AI Gateway achieves sub-millisecond p99 latency with pure Go zero-allocation buffers.",
"PostgreSQL stores virtual key credentials with AES-256-GCM envelope encryption."
],
"top_n": 2,
"return_documents": true
}'

# Structured object documents with rank_fields via /v2/rerank alias
curl -X POST http://localhost:8080/v2/rerank \
-H "Authorization: Bearer sk-gw-prod-7f9a2b8c4d1e" \
-H "Content-Type: application/json" \
-d '{
"model": "jina/jina-reranker-v2-base-multilingual",
"query": "how to configure upstream fallbacks",
"documents": [
{
"title": "Reliability & Fallbacks",
"section": "configuration",
"content": "Configure fallback_group_ids in configs/config.yaml for seamless provider failover."
},
{
"title": "Audio Speech Synthesis",
"section": "audio",
"content": "Speech synthesis maps characters using integer ceiling division to micro-dollars."
}
],
"rank_fields": ["title", "content"],
"top_n": 1,
"return_documents": true
}'

FinOps Accounting & Micro-Dollar Ledger​

Rerank workloads do not consume standard prompt/completion tokens; upstream providers (like Cohere) bill based on Search Units (where 1 Search Unit typically represents up to 100 candidate documents processed in a query).

GoRoute AI Gateway enforces micro-dollar integer accounting:

Cost (µ$) = search_units * RerankCostMicrosPerSearchUnit

[!IMPORTANT] If a provider catalog record does not have a verified RerankCostMicrosPerSearchUnit, the gateway tags the request with X-Gateway-Pricing-Status: unknown and falls back defensively, preventing ledger divergence.

Telemetry Headers​

Each rerank response carries telemetry headers for observability:

  • X-Gateway-Request-Id: Unique tracking ID (req_...)
  • X-Gateway-Provider: Winning upstream provider (cohere or jina)
  • X-Gateway-Cost-Micros: Exact micro-dollar charge deducted from the virtual key's budget
  • X-Gateway-Pricing-Status: verified, custom, or unknown