Extensions

Testing extensions

Load an extension in a test host without an app, drive its commands, hooks, RPC, dialogs and settings, and run the tests with zidane extension test.

createTestHost loads extensions the way an app does, against in-memory storage, settings, dialogs and UI. A test then runs commands, fires hooks, calls RPC, answers dialogs and reads the UI items, with no window and no model. zidane extension test runs an extension's tests with the right imports.

Test every extension you publish. The test host runs the same runtime as the apps: the same setup, guards, scopes and registries. What passes there behaves the same in the terminal, desktop and web apps, up to how each app draws the items (placements).

A minimal test#

index.test.ts

import type { TestHost } from 'zidane/extensions/testing'
import { afterEach, expect, it } from 'bun:test'
import { defineExtension, Slot } from 'zidane'
import { createTestHost } from 'zidane/extensions/testing'

const hello = defineExtension({
  apiVersion: 1,
  id: 'hello',
  settings: { items: { name: { type: 'string', label: 'Name', default: 'world' } } },
  setup(ctx) {
    ctx.add(Slot, { at: 'app.statusbar', text: 'hello' })
    ctx.commands.register({
      id: 'greet',
      title: 'Say hello',
      run: async () => {
        if (await ctx.ui.confirm({ title: 'Greet?' }))
          await ctx.ui.toast({ text: `Hello, ${ctx.settings.get('name') ?? 'world'}` })
      },
    })
  },
})

let host: TestHost | undefined
afterEach(async () => {
  await host?.dispose()
  host = undefined
})

it('greets after a yes', async () => {
  host = await createTestHost({ extensions: [hello], settings: { hello: { name: 'Ada' } } })
  expect(host.extensions()).toMatchObject([{ id: 'hello', status: 'active' }])
  expect(host.ui.items('slot').map(e => e.item.text)).toEqual(['hello'])

  host.dialogs.answer('confirm', true)
  await host.commands.run('hello.greet')
  expect(host.dialogs.shown.map(d => d.request.kind)).toEqual(['confirm', 'toast'])
})

createTestHost resolves once every setup has settled. host.dispose() unloads everything: call it after each test, or timers and listeners leak into the next.

`createTestHost(options)`#

Import it from zidane/extensions/testing.

Option	Type	Does
`extensions`	`(ExtensionDefinition \| { definition, source?, dir?, enabled? })[]`	Required. The extensions to load, in dependency order. `dir` sets `ctx.dir`. `enabled: false` loads one turned off
`settings`	`{ [extensionId]: { [key]: value } }`	Starting values, in the `user` layer
`ports`	`RuntimePorts`	Replace the in-memory ports, or add the host ports a test needs: sessions, generate, workspace, artifacts, catalogs
`host`	`ExtensionHostKind`	What `ctx.app.host` reports. Default `'sdk'`
`paths`	`Partial<ExtensionPaths>`	`cwd`, `dataDir`, `cacheDir` and the rest. Default: under the OS temp folder
`budget`	`Partial<ErrorBudget> \| false`	The guard's error budget. `false` turns off auto-disable
`setupTimeoutMs`	`number`	The bound on `setup`
`onLog`	`(record) => void`	Gets every log record: warnings, errors attributed to an extension

The test host#

Member	Type	Does
`ctx(id)`	`ExtensionContext`	The context the extension's `setup` got. Pass the settings type for typed reads: `ctx<typeof settings>('hello')`
`extensions()`	`ExtensionInfo[]`	Every extension with its `status` (`active`, `disabled`, `failed`) and `error`
`commands.run(id, { args?, sessionId? })`	`Promise<boolean>`	Runs a command by its published id, `<extension>.<id>`
`commands.list()`	`{ id, title, slash?, bind? }[]`	Every registered command
`fire(event, payload)`	`Promise<void>`	Sends an agent hook or a host event (`run:settled`, `session:event`) to the observers, and waits for them
`intercept(name, input, initial, { matchKey? })`	`Promise<result>`	Runs the interceptor chain of `name` and returns the result
`rpc(contract)`	`RpcClient`	A client that calls as another extension, `__test__`
`ui.items(token)`	`UiEntry[]`	The items of one token as the apps see them: `'slot'`, `'panel'`, `'screen'`, `'renderer'`, `'settings-page'`, `'webview'`, `'menu'`, `'theme'`, `'reference'`, `'badge-action'`, `'context-section'`
`ui.snapshot()`	`UiSnapshot`	Every token's items
`ui.client.calls`	`{ method, clientId, args }[]`	Panel, navigation, composer and notification calls, oldest first
`dialogs.answer(kind, value)`	`void`	Queues the answer to the next dialog of `kind`. `value` may be a function of the request. A dialog with no queued answer resolves `undefined`
`dialogs.shown`	`{ request, extensionId, sessionId? }[]`	Every dialog and toast shown
`schedules`		The test clock for `ctx.schedules`: `advance`, `setTime`, `now`, `prompts`, `setEnabled` (below)
`settings.put(id, layer, values)`	`void`	Replaces a settings layer, as an edit in Settings would. `onChange` listeners fire
`storage.snapshot(location)`, `storage.remote(location, change)`		What is stored, and a write from another window
`registry`	`ExtensionRuntimeOutputs`	The shared registries: catalogs, commands, interceptors, UI, RPC, integrations, instructions
`runtime`	`ExtensionRuntime`	Load, unload and reload one extension
`settled()`	`Promise<void>`	Waits until every async callback started so far has finished
`dispose()`	`Promise<void>`	Unloads everything

Common tests#

Each one is a complete test file: a small extension and the test that drives it. The kitchen sink's tests (examples/extension-kitchen-sink/index.test.ts) cover every domain.

A setting change that withdraws an item. settings.put fires onChange, then settled() waits for the extension to react:

status.test.ts

import { expect, test } from 'bun:test'
import { defineExtension, Slot } from 'zidane'
import { createTestHost } from 'zidane/extensions/testing'

const status = defineExtension({
  apiVersion: 1,
  id: 'status',
  settings: { items: { show: { type: 'boolean', label: 'Show the status', default: true } } },
  setup(ctx) {
    const slot = ctx.add(Slot, { at: 'app.statusbar', text: 'status' })
    ctx.settings.onChange('show', (show) => {
      if (show === false)
        slot.dispose()
    })
  },
})

test('hiding the status withdraws its slot', async () => {
  const host = await createTestHost({ extensions: [status] })
  host.settings.put('status', 'user', { show: false })
  await host.settled()
  expect(host.ui.items('slot')).toEqual([])
  await host.dispose()
})

A command behind a confirm dialog. Each answer feeds the next dialog of that kind:

reset.test.ts

import { expect, test } from 'bun:test'
import { defineExtension } from 'zidane'
import { createTestHost } from 'zidane/extensions/testing'

const counter = defineExtension({
  apiVersion: 1,
  id: 'counter',
  setup(ctx) {
    ctx.commands.register({
      id: 'reset',
      title: 'Reset the counter',
      run: async () => {
        if (await ctx.ui.confirm({ title: 'Reset the counter?' }))
          await ctx.storage.kv('user').set('count', 0)
      },
    })
  },
})

test('reset asks first', async () => {
  const host = await createTestHost({ extensions: [counter] })
  const kv = host.ctx('counter').storage.kv('user')
  await kv.set('count', 3)
  host.dialogs.answer('confirm', false)
  await host.commands.run('counter.reset')
  expect(kv.get('count')).toBe(3)
  host.dialogs.answer('confirm', true)
  await host.commands.run('counter.reset')
  expect(kv.get('count')).toBe(0)
  await host.dispose()
})

An interceptor. intercept runs the whole chain from a starting result, and returns the final one. matchKey is the tool name a match list is checked against:

guard.test.ts

import { expect, test } from 'bun:test'
import { defineExtension } from 'zidane'
import { createTestHost } from 'zidane/extensions/testing'

const guard = defineExtension({
  apiVersion: 1,
  id: 'guard',
  setup(ctx) {
    ctx.hooks.intercept('tool.call', call => (String(call.input.command).includes('rm -rf') ? { block: 'No rm -rf.' } : undefined), { match: ['shell'] })
  },
})

test('blocks rm -rf in shell', async () => {
  const host = await createTestHost({ extensions: [guard] })
  const call = (command: string) => ({ tool: 'shell', displayName: 'shell', input: { command }, callId: 'c1', turnId: 't1', runToolCounts: {} })
  expect(await host.intercept('tool.call', call('rm -rf /'), 'allow', { matchKey: 'shell' })).toEqual({ block: 'No rm -rf.' })
  expect(await host.intercept('tool.call', call('ls'), 'allow', { matchKey: 'shell' })).toBe('allow')
  await host.dispose()
})

RPC. An input the schema refuses rejects with an RpcCallError:

rpc.test.ts

import { expect, test } from 'bun:test'
import { defineExtension, Rpc } from 'zidane'
import { createTestHost } from 'zidane/extensions/testing'
import { z } from 'zod'

const CounterRpc = Rpc.define({
  id: 'counter',
  methods: { bump: { input: z.object({ by: z.number().max(100) }), output: z.object({ count: z.number() }) } },
})

const counter = defineExtension({
  apiVersion: 1,
  id: 'counter',
  setup(ctx) {
    let count = 0
    ctx.rpc.register(CounterRpc, { bump: ({ by }) => ({ count: (count += by) }) })
  },
})

test('bump checks its input', async () => {
  const host = await createTestHost({ extensions: [counter] })
  const rpc = host.rpc(CounterRpc)
  expect(await rpc.bump({ by: 3 })).toEqual({ count: 3 })
  await expect(rpc.bump({ by: 5000 })).rejects.toMatchObject({ type: 'rpc.invalid-input' })
  await host.dispose()
})

A host event. fire waits for every observer:

runs.test.ts

import { expect, test } from 'bun:test'
import { defineExtension } from 'zidane'
import { createTestHost } from 'zidane/extensions/testing'

const runs = defineExtension({
  apiVersion: 1,
  id: 'runs',
  setup(ctx) {
    ctx.hooks.on('run:settled', async ({ sessionId }) => {
      const kv = ctx.storage.kv('session', sessionId)
      await kv.ready()
      await kv.set('runs', (kv.get<number>('runs') ?? 0) + 1)
    })
  },
})

test('counts settled runs', async () => {
  const host = await createTestHost({ extensions: [runs] })
  await host.fire('run:settled', { sessionId: 's1', runId: 'r1', ok: true, aborted: false, session: null })
  expect(host.ctx('runs').storage.kv('session', 's1').get('runs')).toBe(1)
  await host.dispose()
})

Schedules#

ctx.schedules runs on an in-memory scheduler with a clock the test moves. Nothing fires on a real timer.

Member	Type	Does
`schedules.advance(ms)`	`Promise<void>`	Moves the clock ahead and runs every fire that came due, `onFire` handlers included, then resolves
`schedules.setTime(epochMs)`	`Promise<void>`	Sets the clock, then runs what came due the same way
`schedules.now()`	`number`	The test clock, epoch ms. It starts at the real time.
`schedules.prompts`	`{ sessionId, text, scheduleId }[]`	The prompts that fires sent, oldest first. No session runs.
`schedules.setEnabled(on)`	`void`	Turns Reminders on or off. It starts on. While off, every call but `enabled()` rejects.

A command that sets a reminder, and the prompt it sends about 20 minutes later. The extension reads the real clock, a few milliseconds past the test clock, so the test moves past the due time:

remind.test.ts

import { expect, test } from 'bun:test'
import { defineExtension } from 'zidane'
import { createTestHost } from 'zidane/extensions/testing'

const remind = defineExtension({
  apiVersion: 1,
  id: 'remind',
  setup(ctx) {
    ctx.commands.register({
      id: 'deploy',
      title: 'Check the deploy in 20 minutes',
      run: async ({ sessionId }) => {
        if (sessionId)
          await ctx.schedules.create({ trigger: { kind: 'at', at: Date.now() + 20 * 60_000 }, action: { kind: 'prompt', sessionId, text: 'Check the deploy' } })
      },
    })
  },
})

test('the reminder arrives after 20 minutes', async () => {
  const host = await createTestHost({ extensions: [remind] })
  await host.commands.run('remind.deploy', { sessionId: 's1' })
  await host.schedules.advance(19 * 60_000)
  expect(host.schedules.prompts).toEqual([])
  await host.schedules.advance(2 * 60_000)
  expect(host.schedules.prompts).toMatchObject([{ sessionId: 's1', text: 'Check the deploy' }])

  host.schedules.setEnabled(false)
  await expect(host.ctx('remind').schedules.list()).rejects.toThrow()
  await host.dispose()
})

Host ports#

On the default 'sdk' host, ctx.sessions, ctx.generate, ctx.workspace and ctx.artifacts reject: no app is there to run sessions or models. Pass fakes in ports.host for the calls your extension makes. The kitchen sink fakes each one and records the calls (fakeHostPorts in its test):

TypeScript

const calls: { verb: string, args: unknown[] }[] = []
const recorded = <T>(verb: string, answer: T) => async (...args: unknown[]): Promise<T> => {
  calls.push({ verb, args })
  return answer
}
const verbs = {
  get: async (id: string) => ({ id, title: 'Soup night', cwd: '/w', status: 'idle', modelId: 'fake-1' }),
  note: recorded('note', true),
  submit: recorded('submit', true),
}
const ports = { host: { sessions: { sessionControl: () => ({ verbs: () => verbs }) } } }

Pass them as createTestHost({ extensions: [...], ports: ports as never }). The real port types are wider than what a test needs, hence the cast.

A folder for the extension#

An extension that reads its own files (ctx.dir, a webview, an Agent Plugin it loads) needs a folder named after its id. Link the folder under that name in a temporary directory, as the kitchen sink does:

folder.test.ts

import { mkdtempSync, symlinkSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { expect, test } from 'bun:test'
import { defineExtension } from 'zidane'
import { createTestHost } from 'zidane/extensions/testing'

const notes = defineExtension({ apiVersion: 1, id: 'notes', setup: () => {} })

test('loads from a folder named after its id', async () => {
  const dir = join(mkdtempSync(join(tmpdir(), 'notes-test-')), 'notes')
  symlinkSync(import.meta.dir, dir, 'dir')
  const host = await createTestHost({ extensions: [{ definition: notes, dir }] })
  expect(host.ctx('notes').dir).toBe(dir)
  await host.dispose()
})

`zidane extension test`#

Terminal

zidane extension test                 # in the extension's folder
zidane extension test ./my-extension -- --watch

It runs bun test in the folder. Arguments after -- go to bun test. When the folder has no zidane in its own node_modules, the command maps zidane, zidane/extensions and zidane/extensions/testing to the zidane that runs it, so tests need no install. The compiled zidane binary carries no package to map: there, add zidane as a dev dependency (bun add -d zidane).

The exit code is bun test's. A folder with no index.ts, index.tsx, index.mjs or index.js fails with exit code 1.

zidane extension create writes a first test next to the extension: it loads it and runs its sample command (publishing).

Pitfalls#

Call host.dispose() after each test. A host left running keeps the extension's timers and listeners alive.
After a change that an extension reacts to asynchronously (a settings edit, a store write, a fired event), await host.settled() before you assert.
There is no fake clock in the test host. For timeoutMs and timers, use jest.useFakeTimers() in bun, or assert on what the extension did, not when.
A dialog with no queued answer resolves undefined, and a confirm reads as false. Queue answers before the command runs.
Command ids in tests are the published ones: kitchen-sink.reset, not reset.
A failed status means setup threw or timed out. host.extensions()[0].error says why.