Skip to content
zidane
SDK reference

zidane/providers/local

A provider for a local OpenAI-compatible server such as Ollama, llama.cpp or LM Studio.

import { … } from 'zidane/providers/local'src/providers/local.ts

Use local() when the model runs on your machine or network. Pass baseURL and defaultModel, or set LOCAL_LLM_BASE_URL and LOCAL_LLM_DEFAULT_MODEL. A key is optional. Set LOCAL_LLM_API_KEY for a server that checks one. For a model that makes tool-call mistakes, spread localReliabilityBehavior() from zidane into behavior. It caps tool output and steers the model when it writes a tool call as text.

TypeScript
import { createAgent } from 'zidane'
import { local } from 'zidane/providers/local'

const provider = local({ baseURL: 'http://localhost:11434/v1', defaultModel: 'llama3.1:8b' })
const agent = createAgent({ provider })
const stats = await agent.run({ prompt: 'Say hello.' })
console.log(stats.turns)

Functions#

localFunction#

Local OpenAI-compatible LLM provider.

Thin wrapper around openaiCompat for self-hosted runtimes that speak the standard POST /chat/completions + SSE dialect: Ollama, vLLM, LM Studio, Lemonade, llama.cpp's server, text-generation-webui, etc.

Caching and reasoning are left off — local runtimes typically strict- validate the request schema and would 400 on the extra fields. Override capabilities when pointing at a vision-capable deployment.

TypeScript
function local(params?: LocalParams): Provider

Parameters

Parameters
NameTypeDescription
params?LocalParams

Base URL, API key, default model, capabilities and extra headers.

Returns

Provider

Example

Example
import { createAgent } from 'zidane'
import { local } from 'zidane/providers/local'

const agent = createAgent({
  provider: local({ baseURL: 'http://localhost:11434/v1', defaultModel: 'llama3.1:8b' }),
})
await agent.run({ prompt: 'Say hello.' })

src/providers/local.ts:99

Interfaces#

LocalParamsInterface#

Options for local.

TypeScript
interface LocalParams

Properties

LocalParams properties
NameTypeDescription
apiKey?string

Optional bearer key. Most local servers don't authenticate; some (vLLM with --api-key, gateway proxies, llama.cpp with --api-key) do. Falls back to LOCAL_LLM_API_KEY, then to a placeholder string accepted by unauthenticated endpoints.

baseURL?string

Base URL of the local OpenAI-compatible server. /chat/completions is appended. Examples:

  • Ollama: http://localhost:11434/v1
  • vLLM: http://localhost:8000/v1
  • LM Studio: http://localhost:1234/v1
  • Lemonade: http://localhost:8000/api/v1
  • llama.cpp: http://localhost:8080/v1

Resolution order: explicit params.baseURL > LOCAL_LLM_BASE_URL env > throw.

capabilities?ProviderCapabilities

Provider capability flags. Defaults to text-only — typical OSS local deployment. Override when pointing at a vision-capable runtime (e.g. Ollama serving a multimodal model, vLLM with a VLM).

defaultModel?string

Default model id. Local runtimes have no fixed catalogue — whatever the user has loaded (e.g. llama3.1:8b, qwen2.5-coder, a HF repo path). Falls back to LOCAL_LLM_DEFAULT_MODEL.

extraHeaders?Record<string, string>

Extra HTTP headers sent on every request. Forwarded to the underlying openaiCompat client — use for auth/routing headers a self-hosted gateway in front of baseURL consumes.

src/providers/local.ts:5