A canary result is “checks passed” on “canary runs”. It is never a score for accuracy or quality, and it is never a percentage. A check Zespan could not judge is counted as not checked, never as passed.
Set up a canary
1
Pick the golden tasks
Build a dataset of up to 10 tasks. A task that came from a real run carries its source run, which Zespan uses to check that the run did not call a tool that can move money or delete data. To use real runs, create the dataset from traces (Datasets, then New dataset, then From traces), so each task keeps its source run. On a run of an agent that has a canary, the button beside the run reads Open the canary’s dataset and opens that dataset page. It does not add the run for you: Zespan has no way yet to add a single run to an existing dataset, so a task you paste in by hand has no source run and waits for the safety box.
2
Pick what Zespan calls
Most agents are called at their own endpoint: register it once under HTTP targets and pick it here. Your agent then uses its own model keys and tools, so it pays for its own calls. An agent that is only a managed prompt can instead run on your own model connection at your own price.
3
Choose how often
Every N minutes, from 10 to 1,440. The default is 60.
4
Add checks
Under each task, add the checks that say what a good answer holds. Every task is always checked for an answer first. Saving replaces the checks of this canary.
5
Let it start
Owners and admins can save a canary. It starts on its own once it has a place to call and at least one task that can run on its own. There is no estimate to click first, but the page always shows what to expect: “I will send 3 tasks to
refund_agent every 60 minutes: up to 72 calls a day.”The checks
Checks are plain code. No model judges an answer, and no model writes a task.
A result reads “6 of 7 checks passed, 1 not checked”. A run with failures reads, in separate parts, “4 of 8 checks passed”, “1 of 3 tasks failed” and “1 not checked”.
What a canary reads and what it never touches
A canary only asks your agent questions you wrote. It does not read your real runs to build tasks, and it does not pool anything across customers.- Safe tasks only. Before a canary runs, I check the tools the source run of each task called. A task whose source run called a tool that can move money or delete data is never run: “I won’t run this task on my own: its source run called
issue_refund, which can move money or delete data.” A task that did not come from a run, or whose tools I could not read, waits for you to tick the safety box: “These tasks are safe to run again and again: no money moves and no records change.” - Paused for a risky tool. If your agent’s own spans show that a canary call used such a tool, I pause the canary and ask you in Needs you to change the task or point it at a test endpoint.
- Apart from your real numbers. Canary calls are kept out of your run counts, costs and issues. They are stored on their own, and the page shows them under each run’s events.
- No retries of a call. A failed call is not retried. A run that fails is confirmed once, 10 minutes later.
What the canary costs
I make no model call of my own for a canary. When your agent is called at its own endpoint, your agent pays for its model and tool calls, and the page says “I can’t see what each call costs you yet” until your agent’s own spans come back with a cost. Owners and admins then see the measured cost per run and per day. When the canary runs on your model connection, owners and admins see the estimated cost per day at your price, and I refuse to run a model whose price I do not know. Caps always apply, and Zespan never raises its own cap.
A run can fail and also be cut short by a limit; the page and Home say both. The canary has a shadow mode in which I count what I would run and make no calls: then Home shows no canary heartbeat and no next canary, and nothing promises one.
Limits reset at 00:00 UTC, shown in your own time zone. When a limit is reached the page says “Paused: today’s canary limit is reached. Resets at 02:00.” and Home says the same. A canary is never silently skipped.
What Home shows
Home says what the canary found, in first person, and each fact once.
Home never shows a dollar figure for the canary.
Next steps
- Home — where the canary’s results appear
- Datasets — build the golden tasks
- HTTP targets — register the endpoint a canary calls
- Agents — open an agent and choose Canary

