zidaneServer operations
Logs, health checks, the storage warning and the errors zidane-server and its clients print, with their causes and fixes.
Logs#
The server writes its log to stderr. At start, it prints where it listens, then any notes and warnings:
✓ zidane server listening on ws://0.0.0.0:4280 and /data/.zidane/server.sock (data /data/.zidane, pid 7)
· Stored the bootstrap token from ZIDANE_SERVER_BOOTSTRAP_TOKEN as tok_3f2a… "bootstrap". Revoke it once device tokens exist: zidane server token revoke bootstrap.
! warning: behind a proxy without trusted proxies: X-Forwarded-* headers from any peer are believed …| Source | Shows |
|---|---|
| stderr, always | The listening line, warnings, a broken token file, shutdown (zidane server: SIGTERM, shutting down) |
stderr, with --verbose (or ZIDANE_SERVER_VERBOSE=1) | Every connection, authentication, refusal and disconnect, as [server] and [ipc-ws] lines |
server-audit.jsonl | Connections, tokens, sync, installs: the audit log |
server-audit.failures.jsonl | Refused upgrades, aggregated per address and minute |
Under systemd: journalctl -u zidane-server -f. In Docker: docker logs -f zidane.
Health#
| Command | Prints on success | Exit |
|---|---|---|
zidane-server health | ok server "<name>" v<version> protocol <n> pid <pid> up <s>s | 0, else 1 |
zidane-server health --http | ok http://127.0.0.1:4280/healthz | 0 on a 200, else 1 |
zidane-server wait-idle | idle, or idle after <s>s | 0 once idle (or when no server answers), 1 when work is still going at --timeout (10 minutes by default) |
wait-idle asks the host over the unix socket until no run, compaction, / command or background task is in flight. Open terminals do not count. Run it before a restart (an upgrade, a renewed certificate), because a restart cuts off running tool calls. daemon.info reports the same as activeWork.
health asks the host over the unix socket, without a token. --timeout <ms> bounds it (default 5000).
health --http requests GET /healthz on the configured port. When the server listens on every interface, the request goes to loopback. It does not check the TLS certificate. From another machine, run:
curl -fsS https://zidane.example.com/healthzChannels#
zidane-server channels check checks the channels config offline and exits 1 on errors. channels status lists the running server's channel integrations. channels reload makes the running server read its config again. It sends SIGHUP to the pid the server reports over its unix socket, so run it where the server runs (in a container, through docker exec). reload sends nothing when no channel runs. A server that started without channels exits on SIGHUP and needs a restart. If its config did not load, fix it and send SIGHUP yourself. See Channels.
Storage warning#
In a Linux container, the server looks at its mounts at start. If its data dir is not on a persistent mount, it prints this warning:
! warning: the data dir is not on a persistent volume; it will be lost when the container is recreated (/data/.zidane is on an anonymous volume, a new one for each container). Mount a named volume or a host directory on it …The cause in parentheses is one of three: the container's own filesystem, a tmpfs, or an anonymous volume. Docker makes an anonymous volume for the image's VOLUME when nothing is mounted, and a new one each time it creates the container. Mount a named volume or a host directory on /data. On Coolify, use Persistent Storage › Volume Mount.
Troubleshooting#
At start#
| Message | Cause | Fix |
|---|---|---|
Refusing to listen on 0.0.0.0 without TLS: device tokens and transcripts would cross the network in the clear. | A non-loopback --host without TLS or a proxy | --tls-cert and --tls-key, or --behind-proxy, or --host 127.0.0.1 |
TLS on the server and behind-proxy are exclusive: … | A certificate with behind-proxy on, often the image's default | ZIDANE_SERVER_BEHIND_PROXY=0 and an empty ZIDANE_SERVER_TRUSTED_PROXIES, or drop the certificate |
TLS needs both a certificate and a key: … | Only one of the two is set | Set both |
… only applies behind a proxy: also set --behind-proxy … | Trusted proxies without --behind-proxy | Add --behind-proxy, or clear the list |
Invalid ZIDANE_SERVER_BOOTSTRAP_TOKEN: expected a token from `zidane server token generate` … | Not a zds_ token | Paste the output of token generate |
ZIDANE_SERVER_BOOTSTRAP_TOKEN and ZIDANE_SERVER_BOOTSTRAP_TOKEN_FILE are both set: set one, not both. | Both variables | Keep one |
warning: … holds a revoked token …: ignored. | The bootstrap value was revoked | Remove the variable, or set a fresh token |
Invalid <file> key "x": expected …, unknown key "x" | A bad server.json | Fix the key. The message lists the known keys |
Cannot read the TLS certificate <file>: ENOENT. | A wrong path, or a file the server's user cannot read | Fix the path or its permissions |
EADDRINUSE | Another process holds the port | Stop it, or change --port |
warning: behind a proxy without trusted proxies … | --behind-proxy with an empty list | Set --trusted-proxies to your proxy |
warning: the TLS certificate <file> expires on <date> …, … expired on … | The certificate the server serves is within 14 days of its end, or past it | Renew it, then restart once idle |
warning: the TLS certificate or key changed on disk … | A renewal: the server keeps serving the pair it read at start | Restart once idle (zidane-server wait-idle, then restart) |
When a client connects#
| Message | Cause | Fix |
|---|---|---|
token refused (401) | A missing, unknown or revoked token | Create a token for the device |
forbidden (403): the server only accepts connections through its HTTPS proxy | Behind a proxy, a request that skipped it, or a proxy that sends no X-Forwarded-Proto: https | Connect through the proxy's wss:// URL, or fix the proxy's headers |
forbidden (403): the server only accepts connections from its trusted proxies | The proxy's address is outside --trusted-proxies | Add it |
forbidden (403): this client's origin is not allowed by the server (allowedOrigins) | A browser origin missing from the allowlist | Add it to ZIDANE_SERVER_ALLOWED_ORIGINS |
the server redirects (301); not following it with the token | An http to https redirect, or a wrong path | Use the final wss:// URL |
Refusing ws://host: the token would travel unencrypted. | ws:// to a remote host | Use wss://, or an SSH tunnel to ws://127.0.0.1 |
Protocol version mismatch with <host>: the server speaks v13, this client v12. Upgrade the client. (status version mismatch) | Different protocol versions | Upgrade the side it names. The client checks again every 5 minutes, or at once with Retry |
A different server answers at <host>'s address | The server was reinstalled, or the name points elsewhere | Remove the host and add it again |
too many connections or connection attempts (429) | Too many upgrades from one address, or too many connections for one token | Close stale windows. If the connections are yours, raise the caps |
TLS failed: … | A private or self-signed certificate this computer does not trust | Trust it in the system's certificate store, or list it in NODE_EXTRA_CA_CERTS |
… can be read by other users (mode 644) … | A --token-file other users can read | chmod 600 the file |
While connected#
| Symptom | Cause | Fix |
|---|---|---|
| The connection closes with 1009 | A message over ZIDANE_SERVER_MAX_MESSAGE_BYTES (32 MiB) | Smaller attachments, or raise the cap |
Rate limit exceeded: slow down and retry. (-32012) | Over 50 requests per second (burst 200) on one connection | Batch requests, or raise ZIDANE_SERVER_RATE_LIMIT |
The reply is over … bytes (-32013) | A reply too large for one message | Ask for less, for example a smaller limit on sessions.loadEvents |
| Long sessions open slowly over a slow link | Transcript pages go out uncompressed | Set ZIDANE_SERVER_COMPRESS_LARGE_MESSAGES=1 (compressLargeMessages): messages of 64 KiB or more go out compressed to clients that accept it, at some CPU cost per message |
| A window drops and reloads its session | Missed heartbeats, a slow consumer, or a server restart | Make sure that the proxy's idle timeout is longer than 20 seconds. The window resumes without help |
| Tool approvals never come | Tool approvals set to Ask or Model decision, and only a browser client connected | Answer from the TUI or the desktop app (why) |
| A new session lands in an odd folder | The session got no folder, so it uses the first workspace root. A session started from a git project on your computer runs in a clone of it, <first root>/<repository name> | Pick a folder in the new-session flow |
| A send from a local project fails with a clone error | The server could not clone the project's origin: no git credentials there, no network, or another folder of that name in the first root | Set up git credentials on the server, or pick a folder in the new-session flow |