Skip to content
zidane
SDK reference

zidane/providers/openai-compat

A factory for any OpenAI-compatible Chat Completions endpoint, with the message and stream helpers it uses.

import { … } from 'zidane/providers/openai-compat'src/providers/openai-compat.ts

Use openaiCompat() when your model sits behind an OpenAI-style /chat/completions route that has no dedicated provider here, such as a gateway or a self-hosted server. You pass the base URL and key. You can also set the auth header, extra headers and capability flags. promptCacheKey sends a cache-affinity key: 'session' uses the agent's session id, and true uses it when there is one, else a hash of the static system prompt and the first user message. The OpenRouter, Cerebras, xAI, Baseten, Arcee, Neuralwatt, Z.ai, Mistral and local providers wrap it. The entry also exports the converters and stream readers, for custom providers.

TypeScript
import { createAgent } from 'zidane'
import { openaiCompat } from 'zidane/providers/openai-compat'

const provider = openaiCompat({
  name: 'my-gateway',
  apiKey: process.env.GATEWAY_API_KEY ?? '',
  baseURL: 'https://gateway.example.com/v1',
  defaultModel: 'my-model',
})

const agent = createAgent({ provider })
await agent.run({ prompt: 'Say hello.' })

Functions#

applyOAICacheBreakpointsFunction#

Add cache_control: { type: 'ephemeral' } breakpoints to the system message's last text part and the last message's final content part.

Mutates messages in place. Converts plain-string content into a single-element content array so the cache marker can attach — this shape is accepted verbatim by OpenRouter's Anthropic and Gemini routes and ignored by routes with automatic caching (OpenAI, DeepSeek, Grok, Groq, Moonshot).

Skip conditions (safe no-ops):

  • Empty messages array.
  • Assistant messages with no text (tool-call-only) — attaching a cache marker to a tool_calls block has no defined semantics, so we skip and let the prior system/tools breakpoints carry caching.

System-prompt boundary handling: when originalSystem (the un-rendered system text passed to toOAIMessages) contains SYSTEM_PROMPT_BOUNDARY, the leading system message is rewritten as a two-part array — cache_control lands on the static prefix only, so per-turn churn in the dynamic suffix doesn't invalidate the cached doctrine above. originalSystem is optional: omit it (or pass a marker-free string) for the single-block behavior.

TypeScript
function applyOAICacheBreakpoints(
  messages: OAIMessage[],
  originalSystem?: string,
  ttl?: CacheTtl,
): void

Parameters

Parameters
NameTypeDescription
messagesOAIMessage[]

Wire messages to mutate. The first system message and the last cacheable message get a marker.

originalSystem?string

System prompt before boundary rendering, used to split the static and dynamic parts.

ttl?CacheTtl

Cache lifetime for the markers. Uses the host default when omitted.

src/providers/openai-compat.ts:559

applyOAIToolCacheBreakpointFunction#

Return a copy of tools with cache_control on the last entry.

OpenRouter accepts the marker alongside the standard type + function fields and forwards it when routing to Anthropic/Gemini. Leaves the non-cached tools unchanged so the caller's reference is not mutated.

TypeScript
function applyOAIToolCacheBreakpoint(tools: OAITool[], ttl?: CacheTtl): OAITool[]

Parameters

Parameters
NameTypeDescription
toolsOAITool[]

Wire tools. Not mutated.

ttl?CacheTtl

Cache lifetime for the marker. Uses the host default when omitted.

Returns

OAITool[]

src/providers/openai-compat.ts:659

assistantMessageFunction#

Creates a text-only assistant message, for priming.

TypeScript
function assistantMessage(content: string): SessionMessage

Parameters

Parameters
NameTypeDescription
contentstring

Message text.

src/providers/openai-compat.ts:744

buildAssistantContentFunction#

Builds an assistant message from streamed text, tool calls and reasoning. Falls back to a [no content] text block when everything is empty.

TypeScript
function buildAssistantContent(
  text: string,
  toolCalls: { id: string; input: Record<string, unknown>; name: string }[],
  thinking?: string,
  reasoning?: { details: unknown[]; model?: string; producer: 'openrouter' } | { model?: string; producer: 'vllm'; reasoningText: string },
): SessionMessage

Parameters

Parameters
NameTypeDescription
textstring

Streamed text.

toolCalls{ id: string; input: Record<string, unknown>; name: string }[]

Tool calls requested by the model.

thinking?string

Streamed thinking text.

reasoning?{ details: unknown[]; model?: string; producer: 'openrouter' } | { model?: string; producer: 'vllm'; reasoningText: string }

Reasoning state to persist, from OpenRouter details or vLLM-style text.

src/providers/openai-compat.ts:795

classifyOpenAICompatErrorFunction#

Classify an OpenAI-compatible error into ClassifiedError.

Recognizes:

  • AbortError (from fetch) → aborted.
  • OpenAICompatHttpError with a context-exceeded code or message → context_exceeded.
  • A rate limit, spent credit or overload (429, 402, 529) → provider_error worded by describeProviderLimit.
  • Any other OpenAICompatHttpError → provider_error.

Returns null for unrecognized error shapes (the loop falls back to AgentProviderError).

TypeScript
function classifyOpenAICompatError(err: unknown, label?: string): ClassifiedError | null

Parameters

Parameters
NameTypeDefaultDescription
errunknown

Error thrown by fetch or the SSE consumer.

label?string'openai-compat'

Product a limit's message names (OpenRouter). Defaults to 'openai-compat', the factory's default name.

Returns

ClassifiedError | null

src/providers/openai-compat.ts:894

consumeSSEFunction#

Reads an SSE chat-completions response to the end, forwarding text and thinking deltas to callbacks. Returns the accumulated text, tool calls, usage, finish reason and, when the server sent them, energy in joules (see readEnergyJoules) and the request's own USD price as costUsd (see readRequestCostUsd). Throws OpenAICompatStreamError on an empty or truncated stream and an AbortError when signal aborts.

TypeScript
function consumeSSE(
  response: Response,
  callbacks: StreamCallbacks,
  signal?: AbortSignal,
): Promise<{ costUsd: number | undefined; energyJoules: number | undefined; finishReason: string; reasoningDetails: Record<string, unknown>[]; text: string; thinking: string; toolCalls: { id: string; input: Record<string, unknown>; malformedArgs?: MalformedToolArguments; name: string }[]; usage: OpenAICompatUsage }>

Parameters

Parameters
NameTypeDescription
responseResponse

Streaming HTTP response with a body.

callbacksStreamCallbacks

Receives text, thinking and tool-call activity as chunks arrive.

signal?AbortSignal

Aborts reading when signaled.

Returns

Promise<{ costUsd: number | undefined; energyJoules: number | undefined; finishReason: string; reasoningDetails: Record<string, unknown>[]; text: string; thinking: string; toolCalls: { id: string; input: Record<string, unknown>; malformedArgs?: MalformedToolArguments; name: string }[]; usage: OpenAICompatUsage }>

src/providers/openai-compat-stream.ts:225

formatToolsFunction#

Converts tool specs to OpenAI function tools, after schema sanitization for the openai profile.

TypeScript
function formatTools(tools: ToolSpec[]): OAITool[]

Parameters

Parameters
NameTypeDescription
toolsToolSpec[]

Tool specs to convert.

Returns

OAITool[]

src/providers/openai-compat.ts:678

mapOAIFinishReasonFunction#

Map an OpenAI-compatible finish_reason string to the zidane TurnFinishReason union.

TypeScript
function mapOAIFinishReason(
  reason: string | null | undefined,
): TurnFinishReason | undefined

Parameters

Parameters
NameTypeDescription
reasonstring | null | undefined

Raw finish_reason from the response.

Returns

TurnFinishReason | undefined

src/providers/openai-compat.ts:957

openaiCompatFunction#

Factory for any OpenAI-compatible HTTP endpoint.

Speaks the standard POST /chat/completions + stream: true + SSE dialect. Thin wrappers such as openrouter, cerebras and baseten call this with pinned defaults.

TypeScript
function openaiCompat(params: OpenAICompatParams): Provider

Parameters

Parameters
NameTypeDescription
paramsOpenAICompatParams

Endpoint, API key, model and capability settings for the OpenAI-compatible host.

Returns

Provider

Examples

Example
openaiCompat({
  name: 'baseten',
  apiKey: process.env.BASETEN_API_KEY!,
  baseURL: process.env.BASETEN_PROXY_URL!,
  authHeader: { name: 'Authorization', scheme: 'Api-Key' },
})
Example
openaiCompat({
  name: 'openai',
  apiKey: process.env.OPENAI_API_KEY!,
  baseURL: 'https://api.openai.com/v1',
  capabilities: { vision: true, documents: true },
})

src/providers/openai-compat.ts:1247

openAICompatPromptMessageFunction#

Build a user SessionMessage from multimodal prompt parts.

Passes every kind (text/image/audio/video/document) through as a canonical block; the actual wire emission, capability gating, and unsupported-media throwing all happen later in toOAIMessages. Implementing this (rather than letting the agent fall back to defaultPromptMessage) is what lets openai-compat routes accept base64 documents/PDFs at all — the default builder throws on base64 documents.

TypeScript
function openAICompatPromptMessage(parts: PromptPart[]): SessionMessage

Parameters

Parameters
NameTypeDescription
partsPromptPart[]

Prompt parts to pack into one user message.

src/providers/openai-compat.ts:715

toOAIMessagesFunction#

Converts a system prompt and session messages to OpenAI Chat Completions messages. Starts with a system message and applies the capability and reasoning settings in options.

TypeScript
function toOAIMessages(
  system: string,
  messages: SessionMessage[],
  options?: ToOAIOptions,
): OAIMessage[]

Parameters

Parameters
NameTypeDefaultDescription
systemstring

System prompt. The prompt-boundary marker is removed before sending.

messagesSessionMessage[]

Session messages to convert.

options?ToOAIOptions{}

Capability, reasoning and model settings that shape the wire messages.

Returns

OAIMessage[]

src/providers/openai-compat.ts:271

toolResultsMessageFunction#

Creates a user message carrying tool results back to the model.

TypeScript
function toolResultsMessage(results: ToolResult[]): SessionMessage

Parameters

Parameters
NameTypeDescription
resultsToolResult[]

Results of the tool calls of the previous turn.

src/providers/openai-compat.ts:753

userMessageFunction#

Creates a text-only user message.

TypeScript
function userMessage(content: string): SessionMessage

Parameters

Parameters
NameTypeDescription
contentstring

Message text.

src/providers/openai-compat.ts:699

Classes#

OpenAICompatHttpErrorClass#

HTTP error thrown when an OpenAI-compatible endpoint returns a non-OK response.

The body is best-effort JSON-parsed; error.message / error.code / error.type are extracted for clean downstream classification.

TypeScript
class OpenAICompatHttpError extends Error

Constructor

TypeScript
new OpenAICompatHttpError(status: number, bodyText: string)

Parses bodyText as JSON to build the error message when possible.

Parameters

Parameters
NameTypeDescription
statusnumber

HTTP status code.

bodyTextstring

Raw response body.

Properties

OpenAICompatHttpError properties
NameTypeDescription
bodyTextreadonlystring

Raw response body.

providerCode?readonlystring

Error code or type from the response body, when present.

statusreadonlynumber

HTTP status code of the response.

src/providers/openai-compat.ts:844

OpenAICompatStreamErrorClass#

Error thrown when an OpenAI-compatible stream is empty, truncated or malformed. Classified as retryable.

TypeScript
class OpenAICompatStreamError extends Error

Constructor

TypeScript
new OpenAICompatStreamError(message: string)

Creates the error.

Parameters

Parameters
NameTypeDescription
messagestring

Description of the stream failure.

src/providers/openai-compat-stream.ts:27

Constants#

ASSISTANT_TOOL_CALLS_TAGConstant#

Sentinel tag that marks a message carrying round-tripped assistant tool calls.

TypeScript
const ASSISTANT_TOOL_CALLS_TAG = '__zidane_assistant_tc__'

src/providers/openai-compat.ts:75

TOOL_RESULTS_TAGConstant#

Sentinel tag that marks a message carrying round-tripped tool results.

TypeScript
const TOOL_RESULTS_TAG = '__zidane_tool_results__'

src/providers/openai-compat.ts:73

Interfaces#

OAIMessageInterface#

A chat message in the OpenAI Chat Completions wire format.

TypeScript
interface OAIMessage

Properties

OAIMessage properties
NameTypeDescription
content?unknown

Message content: a string or an array of content parts. null or absent on tool-call turns.

reasoning?string

Plain reasoning string, used when OpenAICompatParams.reasoningContentField is reasoning.

reasoning_content?string

Plain reasoning string echoed back on assistant messages for vLLM-style reasoning models (Arcee Trinity-Thinking, DeepSeek-R1). Arcee's API accepts reasoning_content; some self-hosted vLLM versions read reasoning.

reasoning_details?unknown[]

OpenRouter normalized reasoning envelope. Echoed back unmodified on assistant messages so the gateway can resume an extended-reasoning chain on the upstream route. Ignored by hosts that don't speak the envelope.

role'system' | 'tool' | 'user' | 'assistant'

Author of the message.

tool_call_id?string

Id of the tool call a tool message answers.

tool_calls?{ function: { arguments: string; name: string }; id: string; type: 'function' }[]

Tool calls requested by an assistant message.

src/providers/openai-compat.ts:35

OAIToolInterface#

A function tool in the OpenAI Chat Completions wire format.

TypeScript
interface OAITool

Properties

OAITool properties
NameTypeDescription
function{ description: string; name: string; parameters: Record<string, unknown> }

Function name, description and JSON Schema parameters.

type'function'

Tool kind. Always function.

src/providers/openai-compat.ts:61

OpenAICompatAuthHeaderInterface#

Auth header config. scheme is prepended to the api key with a space, e.g. { name: 'Authorization', scheme: 'Bearer' } → Authorization: Bearer <key>. Omit scheme for raw header values (e.g. { name: 'X-Api-Key' } → X-Api-Key: <key>).

Real-world examples:

  • Default OpenAI / OpenRouter / Cerebras: { name: 'Authorization', scheme: 'Bearer' }.
  • Baseten dedicated deployments: { name: 'Authorization', scheme: 'Api-Key' }.
  • Some gateways: { name: 'X-Api-Key' }.
TypeScript
interface OpenAICompatAuthHeader

Properties

OpenAICompatAuthHeader properties
NameTypeDescription
namestring

Header name.

scheme?string

Scheme prepended to the key with a space, such as Bearer. Omit to send the key raw.

src/providers/openai-compat.ts:989

OpenAICompatParamsInterface#

Options for openaiCompat.

TypeScript
interface OpenAICompatParams

Properties

OpenAICompatParams properties
NameTypeDescription
apiKeystring

Bearer-style API key.

authHeader?OpenAICompatAuthHeader

Auth header shape. Defaults to { name: 'Authorization', scheme: 'Bearer' }.

baseURLstring

Base URL — /chat/completions is appended.

cacheBreakpoints?boolean

Whether this endpoint honors cache_control: { type: 'ephemeral' } markers on message content parts and tool definitions.

  • true — inject markers when the caller asks for caching. OpenRouter routes to Anthropic/Gemini forward the markers; routes to OpenAI/DeepSeek/ Grok/Groq/Moonshot ignore them (those backends cache automatically).
  • false — never inject markers. Safe default for endpoints that strictly validate the request schema (OpenAI direct, most OSS inference servers) and would reject unknown fields.

Default: false. The openrouter wrapper sets this to true.

capabilities?ProviderCapabilities

Provider-level capability flags. Routed into the message shaper and the agent loop so images in tool results + user messages are handled correctly for the underlying model.

Defaults when omitted: vision: false, imageInToolResult: false, audio: false, video: false — a conservative assumption matching most OSS text-only OpenAI-compat endpoints. Override when routing to a known capable endpoint: set audio: true for audio-input models (input_audio), video: true for video-input models (video_url, e.g. Gemini via OpenRouter), documents: true for PDF-capable models (OpenAI file content part with base64 file_data), or vision: true for vision models.

defaultModel?string

Default model id when run({ model }) is omitted.

extraBodyParams?Record<string, unknown>

Generic pass-through for fields on the Chat Completions request body that zidane does not yet type. Spread into the request before the typed core (model / messages / tools / max_tokens / stream / tool_choice), so explicit options always win on collision.

Forward-compat escape hatch for endpoints that ship one-off fields ahead of zidane (e.g. OpenAI reasoning_effort, OpenRouter provider routing, vendor-specific safety_level knobs).

extraHeaders?Record<string, string>

Extra headers sent with every request (e.g. referer, user-agent).

label?string

Product a rate-limit, quota or overload message names (OpenRouter). Defaults to name.

name?string

Provider name exposed as Provider.name. Defaults to 'openai-compat'.

normalizeNullContent?boolean

Normalize content: null → '' on assistant tool-call turns. Some reasoning models (Arcee/vLLM) degrade or emit malformed follow-ups when a tool-call assistant message carries a null content. Off by default to keep the wire byte-identical for existing providers (OpenRouter, Cerebras, …).

pdfEngine?'native' | 'pdf-text' | 'mistral-ocr'

PDF-parsing engine for OpenRouter's file-parser plugin. Only takes effect on endpoints that honor the plugins request field (OpenRouter) AND only when a request actually carries a forwarded file content part — otherwise it's omitted, so it's a safe no-op for plain-text turns and non-OpenRouter endpoints (which ignore the unknown field).

Without an explicit engine, OpenRouter's per-model native PDF handling is inconsistent (some models silently fail to receive the parsed document). Pinning the engine makes it deterministic:

  • 'native' — use the model's own file support (free; for gpt-4o / Gemini / Claude routes). Opt-in: the openrouter wrapper defaults to 'pdf-text'.
  • 'pdf-text' — free text-only extraction (no OCR / images).
  • 'mistral-ocr'— OCR for scans / image PDFs (paid).

When unset, no plugins field is added.

planReasoningRequest?(ctx: { model: string; thinking?: 'off' | 'minimal' | 'low' | 'medium' | 'high' | 'xhigh' | 'max' | 'adaptive'; thinkingBudget?: number }) => Record<string, unknown> | undefined

Per-request reasoning body planner for hosts whose thinking knob is a vendor-specific request field rather than OpenRouter's reasoning object (e.g. Baseten's reasoning_effort / chat_template_args.enable_thinking). Called on every stream() with the resolved model + thinking options; the returned fields are merged into the request body (after extraBodyParams, before the typed core, so model/messages/etc. still win on collision). Return undefined to send nothing.

promptCacheKey?PromptCacheKeyOption

Prompt-cache affinity key — an opt-in routing hint that keeps a session's requests landing on a replica whose KV cache is already warm. See PromptCacheKeyOption for the shapes, and ./prompt-cache-key.ts for which routes actually honor it (real on OpenAI/Azure and OpenRouter, effectively a no-op on Baseten, real on any proxy/gateway in front).

Default: unset — the field is absent from requests.

promptCacheKeyField?false | 'user' | (string & {}) | 'prompt_cache_key'

Request-body field carrying OpenAICompatParams.promptCacheKey.

  • 'prompt_cache_key' (default) — OpenAI's own hint, combined server-side with the prefix hash; also what OpenRouter reads as its sticky-routing key.
  • 'user' — the legacy spelling of the same idea, for older routes that bucket on it but reject prompt_cache_key.
  • false — no body field; the key travels only in OpenAICompatParams.promptCacheKeyHeader.

Any other field name is accepted for vendor-specific spellings.

promptCacheKeyHeader?string

Additionally send the affinity key as this HTTP header, for gateways that route on a header rather than a body field. Defers to any same-named entry in OpenAICompatParams.extraHeaders.

Default: unset — no header is added.

reasoningContentField?'reasoning_content' | 'reasoning'

Field name to use when echoing plain reasoning back upstream in 'reasoning_content' mode. The OpenAI SDK returns reasoning_content, but some self-hosted vLLM versions only read reasoning on input (vllm#38488). Default: 'reasoning'.

reasoningMode?'openrouter' | 'reasoning_content'

Shape used to round-trip reasoning state across turns when supportsReasoning is true. Ignored when supportsReasoning is false.

  • 'openrouter' (default) — capture/echo OpenRouter's structured reasoning_details[] envelope.
  • 'reasoning_content' — capture the plain reasoning_content string and echo it back on assistant messages (DeepSeek-R1 reasoning parser / Arcee Trinity-Thinking served via vLLM). See https://docs.arcee.ai/capabilities/reasoning-traces.

Default: 'openrouter'.

supportsReasoning?boolean

Whether this endpoint speaks a reasoning round-trip shape selected by reasoningMode — either OpenRouter's normalized reasoning request field plus structured reasoning_details[], or vLLM/Arcee's plain reasoning_content assistant-message field.

  • true — capture compatible reasoning fields from streaming responses into provider_reasoning blocks and echo them back on subsequent assistant messages. In 'openrouter' mode, also map zidane's behavior.thinking / behavior.thinkingBudget to the request's reasoning field.
  • false — never set the field; drop any stored provider_reasoning blocks before sending. Safe default for hosts that strict- validate the request schema.

Default: false. The openrouter wrapper sets this to true.

src/providers/openai-compat.ts:997