# AGENTS
Source: https://docs.zespan.com/AGENTS
> **First-time setup**: Customize this file for your project. Prompt the user to customize this file for their project.
> For Mintlify product knowledge (components, configuration, writing standards),
> install the Mintlify skill: `npx skills add https://mintlify.com/docs`
# Documentation project instructions
## About this project
* This is a documentation site built on [Mintlify](https://mintlify.com)
* Pages are MDX files with YAML frontmatter
* Configuration lives in `docs.json`
* Run `mint dev` to preview locally
* Run `mint broken-links` to check links
## Terminology
## Style preferences
* Use active voice and second person ("you")
* Keep sentences concise — one idea per sentence
* Use sentence case for headings
* Bold for UI elements: Click **Settings**
* Code formatting for file names, commands, paths, and code references
## Content boundaries
# API keys
Source: https://docs.zespan.com/account/api-keys
Understand project API keys and personal API keys in Zespan, where to create and revoke them, and best practices for managing them safely.
Zespan uses two distinct kinds of API keys, scoped differently and used for different things. **Project API keys** authenticate the SDK when it sends trace data to Zespan. **Personal API keys** authenticate you when an AI assistant connects to Zespan's hosted MCP server. This page covers how each is scoped, where to manage it, and how to keep it safe.
## Key types at a glance
| | Project API key | Personal API key |
| ---------------- | --------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- |
| **Prefix** | `zsp_` | `lqtp_` |
| **Scope** | One project — can only write trace data to the project it was created for | The user who created it — inherits that user's role and permissions |
| **Used for** | SDK tracing/ingest (`zespan.init(...)`), direct API requests via the `x-api-key` header | Connecting an AI assistant (Claude Desktop, Cursor, etc.) to Zespan's MCP server as a Bearer token |
| **Managed from** | The project's **Settings → API Keys** page | Your account settings |
## Project API keys
A project API key is what the Zespan SDK uses to send trace events for a single project. It starts with `zsp_` followed by 64 hex characters, and it is scoped to exactly one project — it can only write trace data to the project it was created for, and it cannot read or write anything in any other project.
Use it when initializing the SDK:
```typescript TypeScript theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
zespan.init({
apiKey: "zsp_your_key_here",
environment: "production",
});
```
```python Python theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
zespan.init(
api_key="zsp_your_key_here",
environment="production",
)
```
The SDK sends this key as an `x-api-key` header on requests to the ingest API. If you're calling Zespan's API directly rather than through the SDK, set the same header yourself.
### Creating and managing project keys
Project API keys are created and managed from that project's **Settings → API Keys** page. The key is shown in full only once, at creation — copy it somewhere safe immediately, since Zespan cannot show it to you again.
You can revoke or rotate a project key at any time from the same page. A rotated-out key stays valid for **24 hours**, so you can roll a new key into your deployed services without downtime. See [Security](/platform/security) for the full authentication model.
## Personal API keys
A personal API key authenticates you — not a project — and is used specifically to connect an AI assistant to Zespan's hosted MCP server. It starts with `lqtp_` and is sent as a Bearer token:
```json theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
{
"mcpServers": {
"zespan": {
"url": "https://api.zespan.com/v1/mcp",
"headers": {
"Authorization": "Bearer lqtp_YOUR_PERSONAL_API_KEY"
}
}
}
}
```
Unlike a project key, a personal key isn't scoped to a single project — it inherits whatever role and permissions the user who created it has across the organization. That means what the MCP tools can see and do (for example, whether they're allowed to promote a prompt to `production`) depends on your role, exactly as it would if you performed the same action from the dashboard.
### Creating and managing personal keys
Generate and revoke personal API keys from your account settings. Give each one a descriptive name so you can tell them apart later, and treat it with the same care as a password — anyone with your personal key can act as you through any tool that supports MCP.
See [Zespan MCP server](/guides/zespan-mcp) for the full client setup for Claude Desktop, Cursor, and other MCP clients.
## Best practices
* **Never commit keys to source control.** Load them from environment variables (`ZESPAN_API_KEY`, or similar) instead of hardcoding them in your codebase.
* **Use a distinct project key per environment.** Separate keys for development, staging, and production mean you can revoke or rotate one without affecting the others.
* **Rotate keys periodically**, and immediately if you suspect a key has leaked.
* **Scope access deliberately.** Only generate a personal API key for MCP if you actually need an AI assistant querying your data, and revoke it when you no longer need it.
For the full security model — encryption, tenant isolation, and how keys are stored — see [Security](/platform/security).
# Billing
Source: https://docs.zespan.com/account/billing
Compare Zespan pricing plans, understand overage billing, view usage, and manage your subscription.
Billing is visible to **Owner**, **Admin**, and **Billing** roles. Only **Owner** and **Billing** can make changes — upgrade, downgrade, cancel, or update payment details. Admin can view the plan and usage but cannot modify billing. **Editor** and **Viewer** have no billing access at all. See [Organizations](/account/organizations) for the full role breakdown.
Zespan bills per organization through [Polar](https://polar.sh). Manage your subscription from **Settings → Billing**.
## Plans
| | Free | Solo | Pro | Team | Scale |
| ------------------ | ------- | ------- | --------- | --------- | --------- |
| **Price** | \$0/mo | \$29/mo | \$149/mo | \$299/mo | Custom |
| **Traces/month** | 50K | 100K | 500K | 2M | 10M+ |
| **Data retention** | 14 days | 30 days | 90 days | 180 days | 365 days |
| **Projects** | 2 | 10 | Unlimited | Unlimited | Unlimited |
| **Members** | 1 | 2 | Up to 10 | Up to 15 | Unlimited |
| **SSO (OIDC)** | — | — | — | Included | Included |
| **Self-hosted** | — | — | — | — | Available |
## Overage billing
When your organization exceeds its monthly trace quota, additional traces are billed at the end of the period:
| Plan | Overage rate |
| ----- | ------------------- |
| Solo | \$0.50 / 10K traces |
| Pro | \$0.50 / 10K traces |
| Team | \$0.50 / 10K traces |
| Scale | Custom |
On the **Free** plan there is no overage — traces are dropped once the monthly limit is reached.
The usage bar in **Settings → Billing** shows your current month-to-date count against your quota and updates in real time.
## Extra seats
Each plan includes a set number of members before you pay per seat: Free and Solo don't allow extra seats at all (you must upgrade to add members), Pro is **$10/seat/month** beyond its included 10, and Team is **$8/seat/month** beyond its included 15. **Settings → Billing** shows a seat-cost preview before you confirm an invite that would push you over your included count.
## What's gated by plan
Beyond trace quota, retention, and seats, several features require a specific plan or higher:
| Feature | Minimum plan |
| ------------------------------------ | ------------ |
| ZespanPilot / AI features | Solo |
| Alerts | Pro |
| Audit log | Pro |
| Cost attribution / Cost Optimizer | Pro |
| Prompt Enhance | Pro |
| Playground BYOK (bring your own key) | Pro |
| Evaluations (create/manage) | Team |
| Simulations | Team |
| Incidents | Team |
| SSO (OIDC) | Team |
| Auto-evaluators | Team |
| Guardrails | Scale |
| Self-hosted deployment | Scale |
This table lists plan-level feature gates as implemented in the product. If a page elsewhere in these docs states a different minimum plan for a specific feature, trust that page's own gating note first — some features (like Audit log) are gated by plan rather than by role, so every role can see them once the organization is on a qualifying plan.
## Opening billing settings
1. Sign in as an **Owner**, **Admin**, or **Billing**-role member of your organization.
2. Click your organization name in the top navigation.
3. Go to **Settings → Billing**.
You'll see your current plan and month-to-date usage. **Owner** and **Billing** additionally see options to upgrade, downgrade, or manage payment — **Admin** sees this page read-only.
## Upgrading your plan
1. In **Settings → Billing**, click **Upgrade** or **Change plan**.
2. Select the plan you want.
3. If you don't have an active subscription yet, you're redirected to a Polar checkout page — enter or confirm your payment details. Zespan never stores card information. If you already have an active subscription, the change applies in place (no redirect) and is prorated automatically.
4. Once payment is confirmed, your new limits take effect **immediately**, and Zespan emails the organization's owners a confirmation.
Mid-month upgrades are prorated for the remaining days in the billing period.
## Downgrading your plan
1. In **Settings → Billing**, click **Change plan**.
2. Select a lower plan (Solo, Pro, or Team — downgrading to Free is done by [cancelling](#cancelling) instead).
3. Zespan applies the change to your existing subscription via Polar with prorated billing — no separate checkout is needed.
4. The new (lower) limits take effect **immediately**, the same as an upgrade.
## Payment, invoices, and billing details
1. In **Settings → Billing**, click **Manage billing**.
2. You're taken to the Polar customer portal where you can:
* Update your credit card or payment method
* Download past invoices as PDFs
* Update billing address or tax ID
## Cancelling
Cancellation takes effect at the end of the current billing period. Your organization moves to the Free plan — data retention drops to 14 days and any data older than that becomes inaccessible.
1. In **Settings → Billing**, click **Manage billing**.
2. In the Polar customer portal, cancel your subscription.
3. Your current plan stays active until the end of the billing period.
After downgrade to Free:
* Traces capped at 10K/month
* Data retention drops to 14 days
* Max 2 projects and 1 member
# Organizations: workspaces, members, and settings
Source: https://docs.zespan.com/account/organizations
Learn how to create and manage organizations in Zespan, invite team members, assign roles, and control access to your projects and billing.
An organization is your top-level workspace in Zespan. It contains your projects, holds your billing subscription, and groups the team members who can access your data. Every project you create belongs to an organization, and all usage — events ingested, data retained, and members invited — counts against that organization's plan. You can belong to multiple organizations and switch between them at any time.
## Creating an organization
When you sign up for Zespan, the onboarding wizard walks you through creating your first organization automatically. If you need additional organizations later, you can create up to **3 organizations per account**.
Click the organization name in the top of the left sidebar. At the bottom of the dropdown, select **Create new workspace**.
Enter a display name for the organization. This name appears in the sidebar and in email notifications to your team.
Give your first project a name. A project is the ingest scope that your SDK will target — each project gets its own API key.
Copy the SDK snippet shown on screen and add it to your application. The snippet includes your project's API key pre-filled.
Zespan waits for your first event to arrive. Once it does, you're redirected to the project dashboard automatically.
## Switching organizations
The **org switcher** is the dropdown in the top of the left sidebar. Click it to see all organizations you belong to, then select one to switch your active context. Everything in the dashboard — projects, usage, settings — reflects the selected organization.
## Inviting team members
You can invite teammates to your organization from **Settings > Team**. Invitations are role-scoped, so you assign a role at the time of invite.
The Free and Solo plans are limited to 1 member (the owner). You must be on the Pro plan or higher to invite additional members.
Navigate to **Settings > Team** within your organization.
Enter the teammate's email address and select a role — **Admin**, **Editor**, **Viewer**, or **Billing**.
Click **Send invite**. Zespan sends an email to that address with a link to accept. Invitations expire after **7 days** if not accepted.
You can view pending invitations in the same Team settings page and resend or cancel them at any time.
## Roles and permissions
Zespan has five roles: **Owner**, **Admin**, **Editor**, **Viewer**, and **Billing**. There is no role literally named "Member" — every teammate you invite gets one of these five.
| Permission | Owner | Admin | Editor | Viewer | Billing |
| ----------------------------------- | ----- | ----- | -------------- | -------------- | -------------- |
| View dashboards and traces | Yes | Yes | Yes | Yes | Yes |
| Export traces | Yes | Yes | Yes | No | No |
| Manage projects | Yes | Yes | No | No | No |
| Manage API keys | Yes | Yes | No | No | No |
| Manage evaluations and simulations | Yes | Yes | Yes | No | No |
| Manage guardrails and prompts | Yes | Yes | No | No | No |
| Manage alerts and incidents | Yes | Yes | Yes | No | No |
| Manage agents | Yes | Yes | Yes | No | No |
| Use AI features (ZespanPilot, etc.) | Yes | Yes | Yes | No | No |
| Manage team members | Yes | Yes | No (view only) | No (view only) | No (view only) |
| View audit log | Yes | Yes | Yes | Yes | Yes |
| View billing | Yes | Yes | No | No | Yes |
| Manage billing and payment | Yes | No | No | No | Yes |
| Manage organization settings | Yes | Yes | No | No | No |
| Delete the organization | Yes | No | No | No | No |
**Editor** cannot manage guardrails or prompts — those are treated as high-risk operations reserved for Admin and Owner. **Billing** is a narrow role: it can view traces and dashboards like Viewer, but its only *management* permission is billing — it cannot touch projects, alerts, or team settings.
Every organization has exactly one owner — the person who created it. Ownership cannot be transferred through the dashboard.
## Changing a member's role
In **Settings > Team**, each member row has a role selector. Owners and Admins can change any member's role between **Admin**, **Editor**, **Viewer**, and **Billing**. You cannot change the owner's role.
## Removing a member
Click **Remove** next to any member in **Settings > Team** to revoke their access immediately. Removed members lose access to all projects in the organization. The owner cannot be removed.
## Renaming your organization
Go to **Settings > General** and update the **Organization name** field. The new name takes effect immediately across the dashboard and in any email notifications.
## Deleting an organization
Deleting an organization is permanent and cannot be undone. All projects, trace data, and API keys are destroyed immediately, and your billing subscription is cancelled at the end of the current period.
Only the **owner** can delete an organization. To do so, go to **Settings > General**, scroll to the **Danger zone** section, and click **Delete organization**. You will be asked to confirm by typing the organization's name.
## Plan limits for members
| Plan | Members included |
| --------------- | ---------------- |
| Free | 1 (owner only) |
| Solo — \$39/mo | 1 (owner only) |
| Pro — \$149/mo | Up to 10 |
| Team — \$299/mo | Up to 15 |
| Scale — Custom | Unlimited |
To add more members than your plan allows, upgrade from **Settings > Billing**. See [Billing](/account/billing) for a full plan comparison.
# The raw dependency graph for a project
Source: https://docs.zespan.com/api-reference/blast-radius/the-raw-dependency-graph-for-a-project
/api-reference/openapi.yaml get /v1/projects/{id}/graph
Returns every node and edge in the project's dependency graph, optionally filtered to a subset of node kinds — the same graph `GET /v1/projects/{id}/blast-radius` traverses from a single root, but unfiltered by root. Sized for a future graph visualization; nothing in the dashboard renders it today.
Authenticated the same way as `GET /v1/projects/{id}/blast-radius` (dashboard session, `dashboard:read`).
# What breaks if this resource changes
Source: https://docs.zespan.com/api-reference/blast-radius/what-breaks-if-this-resource-changes
/api-reference/openapi.yaml get /v1/projects/{id}/blast-radius
Walks the dependency graph from one root node and returns every dependent found (breadth-first, bounded by `maxDepth`/`maxNodes`), summarized into an impact rollup. Backs the pre-release impact card on the Prompts page and the delete-blocking check on Evaluations.
Dependency direction: an edge's `to` depends on its `from` — walking dependents from a node answers "what would be affected if I changed this," which is the reverse of the call direction for prompt/model edges (an agent *calls* a prompt or model, but the *prompt or model change* is what affects the agent, so the edge points prompt→agent / model→agent).
**Authenticated with a dashboard session (browser cookie), not `x-api-key`**, and requires the `dashboard:read` permission. There is no server-side cache in front of this endpoint today — every request rebuilds the graph, so `computedAt` in the response is effectively "now."
# Coverage preview for a framework and period, without generating
Source: https://docs.zespan.com/api-reference/compliance/coverage-preview-for-a-framework-and-period-without-generating
/api-reference/openapi.yaml get /v1/projects/{id}/compliance/coverage
Runs the same evidence queries a control-evidence document would for the given framework and period, and reports per-control record counts — without rendering or storing anything. This is what the Compliance page calls before you commit to generating a pack, so a gap in the underlying data is visible before a document leaves the building, not after.
**Authenticated with a dashboard session (browser cookie), not `x-api-key`**, and requires `compliance:read` plus the Pro plan or above.
# Generate an evidence pack
Source: https://docs.zespan.com/api-reference/compliance/generate-an-evidence-pack
/api-reference/openapi.yaml post /v1/projects/{id}/evidence-packs
Creates an `EvidencePack` row in `pending` status and enqueues generation on a background worker — this endpoint returns immediately, before the document exists. Poll `GET /v1/projects/{id}/evidence-packs/{packId}` (or list packs) to watch `status` move through `processing` to `completed` (or `failed`).
`kind: "control_evidence"` requires `framework`; `kind: "agent_card"` ignores it. `periodStart` must be before `periodEnd`, and the period may not exceed 400 days. There is no `"pdf"` value for `format` — this deployment has no headless-browser rendering path, so a PDF request is rejected here rather than silently downgraded to HTML.
**Authenticated with a dashboard session (browser cookie), not `x-api-key`**, and requires both the `compliance:generate` permission (owner/admin only — editor, viewer, and billing roles can read and download but not generate) and the Pro plan or above.
# Get one evidence pack, optionally its content
Source: https://docs.zespan.com/api-reference/compliance/get-one-evidence-pack-optionally-its-content
/api-reference/openapi.yaml get /v1/projects/{id}/evidence-packs/{packId}
Returns the pack row plus, once `status` is `completed`, either a presigned `downloadUrl` (object-storage backend) or the raw `content` string (local-disk backend, when `download=true` is passed) — never both. A pack belonging to a different project than `id` returns `404`, the same non-enumerable behavior used elsewhere in the API for cross-project access.
**Authenticated with a dashboard session (browser cookie), not `x-api-key`**, and requires `compliance:read` plus the Pro plan or above.
# List available compliance frameworks
Source: https://docs.zespan.com/api-reference/compliance/list-available-compliance-frameworks
/api-reference/openapi.yaml get /v1/compliance/frameworks
A static capability listing — the frameworks Zespan can map evidence to today, each with its controls. `soc2` is the only registered framework as of this release; EU AI Act and ISO/IEC 42001 are not yet available. Not project-scoped and not plan-gated: a customer deciding whether to upgrade needs to see what they'd get before they commit.
**Authenticated with a dashboard session (browser cookie), not `x-api-key`**, and requires the `compliance:read` permission.
# List evidence packs for a project
Source: https://docs.zespan.com/api-reference/compliance/list-evidence-packs-for-a-project
/api-reference/openapi.yaml get /v1/projects/{id}/evidence-packs
Paginated list, newest first. `evidenceIndex` (the citation list) is deliberately excluded from every row here — it can hold hundreds of entries per pack and this view never renders them; fetch a single pack to read it.
**Authenticated with a dashboard session (browser cookie), not `x-api-key`**, and requires `compliance:read` plus the Pro plan or above.
# Re-verify a generated evidence pack
Source: https://docs.zespan.com/api-reference/compliance/re-verify-a-generated-evidence-pack
/api-reference/openapi.yaml get /v1/projects/{id}/evidence-packs/{packId}/verify
Re-hashes the stored document and re-resolves every citation it made against live data, scoped to the pack's project or organization. Never errors on a data condition — a deleted record, a deleted storage object, or a pack that never finished generating are all reported as *results*, not exceptions. See [Verification](https://docs.zespan.com/compliance/verification) for the full field reference, in particular what `unverifiableRefs` means and why it must be read alongside `evidenceStillPresent` rather than ignored.
**Authenticated with a dashboard session (browser cookie), not `x-api-key`**, and requires `compliance:read` plus the Pro plan or above.
# Compare two dataset runs
Source: https://docs.zespan.com/api-reference/datasets/compare-two-dataset-runs
/api-reference/openapi.yaml get /v1/datasets/{datasetId}/runs/compare
Compare two runs of a dataset. Returns each dataset item shared by both runs with each run's trace id and score side by side, plus each run's average score over the shared items.
# Create or fetch a dataset run
Source: https://docs.zespan.com/api-reference/datasets/create-or-fetch-a-dataset-run
/api-reference/openapi.yaml post /v1/datasets/{datasetId}/runs
Create a named run for a dataset, or return the existing run with that name. Idempotent: the SDK calls this every time a job starts.
# Get dataset run detail
Source: https://docs.zespan.com/api-reference/datasets/get-dataset-run-detail
/api-reference/openapi.yaml get /v1/datasets/{datasetId}/runs/{runId}
Fetch a run with every linked item joined to its dataset item content and, if the run has been scored, its evaluation result.
# Link an item to a dataset run
Source: https://docs.zespan.com/api-reference/datasets/link-an-item-to-a-dataset-run
/api-reference/openapi.yaml post /v1/datasets/{datasetId}/runs/{runId}/items
Link a dataset item to a run by recording the trace produced for it. Idempotent: re-linking updates the stored trace pointer.
# List dataset items
Source: https://docs.zespan.com/api-reference/datasets/list-dataset-items
/api-reference/openapi.yaml get /v1/datasets/{datasetId}/items
List the items in a dataset (up to 1000), oldest first.
# List dataset runs
Source: https://docs.zespan.com/api-reference/datasets/list-dataset-runs
/api-reference/openapi.yaml get /v1/datasets/{datasetId}/runs
List runs for a dataset (newest first), each enriched with its latest scoring status and average score if it has been scored.
# Resolve a dataset by name
Source: https://docs.zespan.com/api-reference/datasets/resolve-a-dataset-by-name
/api-reference/openapi.yaml get /v1/datasets/by-name/{name}
Resolve a dataset id from its name within the authenticated project. This is the SDK's entry point, since customer code refers to datasets by name.
# Run a dataset against a registered HTTP Target
Source: https://docs.zespan.com/api-reference/datasets/run-a-dataset-against-a-registered-http-target
/api-reference/openapi.yaml post /v1/datasets/{datasetId}/runs/http-target
Start a Zespan-executed run: for every item in the dataset, Zespan hydrates the target's request template with the item's `input` and POSTs it directly to the registered HTTP Target's endpoint, capturing the raw response as a trace tagged `sdk_name: "zespan-http-endpoint"`. Unlike every other dataset-run endpoint on this page, Zespan itself makes the call — no SDK integration on the target's side is required.
The target itself is created, updated, and deleted from the dashboard (**Project Settings → HTTP Targets**), not through this public API — see [HTTP Targets](/dashboard/http-targets).
The run is created (or re-attached to, if `runName` matches an existing run) and its execution is enqueued asynchronously; poll `GET /v1/datasets/{datasetId}/runs/{runId}` for progress and to see each item link as it completes.
# Score a dataset run
Source: https://docs.zespan.com/api-reference/datasets/score-a-dataset-run
/api-reference/openapi.yaml post /v1/datasets/{datasetId}/runs/{runId}/score
Trigger scoring of a run's linked traces with an evaluator. Creates a fresh evaluation run over the run's trace ids and enqueues it for asynchronous scoring.
# Run a guardrails check
Source: https://docs.zespan.com/api-reference/guardrails/run-a-guardrails-check
/api-reference/openapi.yaml post /v1/guardrails/check
Evaluate text against the authenticated project's enabled guardrails at runtime. Returns whether the text is allowed, the per-guardrail results, and any modified (e.g. redacted) text. Requires `x-api-key`.
# Ingest trace events
Source: https://docs.zespan.com/api-reference/ingestion/ingest-trace-events
/api-reference/openapi.yaml post /v1/ingest
Ingest a batch of trace/span events. The request body is newline-delimited JSON (`application/x-ndjson`): one JSON event object per line. A maximum of 100 events per request and a 1 MB body size limit apply. Events are validated and queued asynchronously; invalid lines are skipped rather than failing the whole batch.
Authenticated with `x-api-key`. Rate limited to 300 requests/minute per API key.
# API reference
Source: https://docs.zespan.com/api-reference/introduction
Authenticate, choose a base URL, and call the Zespan Public API directly or through the SDKs.
The Zespan Public API is the surface your application and the Zespan SDKs call
at runtime: send traces, manage prompts, drive dataset experiments, and check
guardrails. Every endpoint is versioned under `/v1` and returns JSON.
## Base URL
```
https://api.zespan.com
```
All paths in this reference are shown with the `/v1` prefix, for example
`POST https://api.zespan.com/v1/ingest`.
## Authentication
Every endpoint in this reference authenticates with a project API key sent in the
`x-api-key` header. A key scopes the request to exactly one project, so
project-scoped endpoints infer the project from the key and the `projectId`
parameter is optional.
```bash theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
curl https://api.zespan.com/v1/prompts \
-H "x-api-key: $ZESPAN_API_KEY"
```
Treat your API key like a password. Set it from an environment variable or
secret store, never commit it to source control, and rotate it from the
dashboard if it is exposed.
Create and rotate keys from your project settings in the Zespan dashboard. See
[API keys](/account/api-keys) for details.
### Guardrail policy changes are not API-key accessible
Reading and applying guardrail [policy-as-code](/policies/as-code) checks the
acting user's **role in the organization**, which a project key cannot supply — a
key identifies a project, not a person. Those actions are therefore outside this
API and refuse a project API key rather than partially honouring it. Perform them
from the dashboard, or with [`zespan policy`](/cli/policy) after
[`zespan auth login`](/cli/auth).
**Ingest and the rest of this reference are unaffected.** `/v1/ingest`,
`/v1/traces`, `/v1/guardrails/check` and every other endpoint documented here
continue to authenticate with `x-api-key`, exactly as before.
## Rate limits
Ingestion endpoints (`/v1/ingest`, `/v1/traces`) share the same limits: 300 requests
per minute per API key. When you exceed the limit the API responds with `429`
and a `Retry-After` header telling you how many seconds to wait.
Both ingestion endpoints also cap request size at 1 MB, and both count against
your organization's monthly event quota — but their per-request event caps
differ: `/v1/ingest` caps at 100 events per request, while `/v1/traces` (OTLP)
caps at 512 spans per request, matching the OpenTelemetry SDK/Collector's own
default `max_export_batch_size`. A batch over 512 spans sent to `/v1/traces`
returns `202` with the overflow reported in `partialSuccess.rejectedSpans` —
lower your exporter's max batch size rather than relying on the cap.
The OTLP metrics (`/v1/metrics`) and logs (`/v1/logs`) endpoints are not
implemented and return `501`. Send traces to `/v1/traces`.
## What you can do
Send trace and span events with the native NDJSON endpoint, or via OpenTelemetry.
Fetch and manage versioned prompts, labels, tags, and folders.
Read datasets and drive dataset runs and scoring.
Evaluate text against your project's guardrails at runtime.
See the endpoint list in the sidebar for full request/response schemas.
# Dismiss a model deprecation finding
Source: https://docs.zespan.com/api-reference/model-lifecycle/dismiss-a-model-deprecation-finding
/api-reference/openapi.yaml post /v1/projects/{id}/model-lifecycle/{findingId}/dismiss
Suppresses a finding at its current urgency band. It reopens automatically — exactly once, and notifies again — the next time the daily scan finds the band has tightened (90 → 30 → 7 days, or into retired). A band that widens, which only happens when a feed correction pushes the retirement date further out, never reopens a dismissal.
**Authenticated with a dashboard session (browser cookie), not `x-api-key`**, and requires the `alerts:manage` permission — dismissing a finding is the same shape of decision as editing an alert rule.
# List model deprecation findings for a project
Source: https://docs.zespan.com/api-reference/model-lifecycle/list-model-deprecation-findings-for-a-project
/api-reference/openapi.yaml get /v1/projects/{id}/model-lifecycle
Returns findings from the daily deprecation scan for this project, soonest deadline first (an already-retired model, with a negative `daysRemaining`, sorts to the very top). Backs the Model Lifecycle dashboard page, the Overview widget, and the Models page banner.
**Authenticated with a dashboard session (browser cookie), not `x-api-key`**, and requires the `dashboard:read` permission.
# The full bundled model lifecycle feed
Source: https://docs.zespan.com/api-reference/model-lifecycle/the-full-bundled-model-lifecycle-feed
/api-reference/openapi.yaml get /v1/model-catalogue
Returns Zespan's curated, bundled catalogue of provider-announced deprecation and retirement dates in full — not scoped to a project, since the catalogue is the same for every tenant. This is the raw data [Model Lifecycle](/dashboard/model-lifecycle) findings and the Models page's Lifecycle column are matched against. See the [feed reference](/reference/model-lifecycle-feed) for the field semantics.
**Authenticated with a dashboard session (browser cookie), not `x-api-key`**, and requires the `dashboard:read` permission — not project-scoped, but still member-gated, since the curation is part of what a paying customer gets.
# Per-model drill-down — usage, reliability, quality, and cost-vs-quality frontier
Source: https://docs.zespan.com/api-reference/models/per-model-drill-down-—-usage-reliability-quality-and-cost-vs-quality-frontier
/api-reference/openapi.yaml get /v1/projects/{id}/models/{model}
The follow-up question to the Models table: everything about ONE model over the selected range. Backs the Model detail dashboard page reached by clicking a model row.
Quality figures are computed directly from `evaluation_scores` grouped by `model`, not through the `llm_events`↔`evaluation_scores` span join the Cost vs Quality page uses — there is no fan-out to correct for when both sides are grouped by model directly.
`frontier.points` lists every OTHER model called in the project over the same range with its own cost and quality, each flagged `cheaper` against this model and carrying a `qualityDelta` (`null` when either side has no quality score yet) — the data behind "should I switch this workload?".
Returns `found: false` with empty panels (200, not 404) for a model with zero calls in range, rather than erroring — a model can be renamed, retired, or simply unused in the chosen window.
**Authenticated with a dashboard session (browser cookie), not `x-api-key`**, and requires the `dashboard:read` permission.
# Per-model usage, cost, latency, and error rate
Source: https://docs.zespan.com/api-reference/models/per-model-usage-cost-latency-and-error-rate
/api-reference/openapi.yaml get /v1/projects/{id}/models
Aggregates every model called in the project over the selected range, ranked by the chosen sort. Backs the Models dashboard page's table.
Each row also carries a `lifecycle` overlay: `null` when the model has neither a catalogue entry nor an open finding, otherwise the same countdown/urgency fields as a [Model Lifecycle](#tag/Model-Lifecycle) finding, joined from the project's `ModelLifecycleFinding` rows (open findings) and the global `ModelLifecycle` catalogue (dates for models that haven't produced a finding yet, e.g. because the retirement is further out than the detection horizon).
**Authenticated with a dashboard session (browser cookie), not `x-api-key`**, and requires the `dashboard:read` permission.
# Export OTLP logs (not implemented)
Source: https://docs.zespan.com/api-reference/opentelemetry/export-otlp-logs-not-implemented
/api-reference/openapi.yaml post /v1/logs
Not implemented. This endpoint authenticates the API key and returns `501`; no logs are stored. Send traces to `/v1/traces` instead.
# Export OTLP metrics (not implemented)
Source: https://docs.zespan.com/api-reference/opentelemetry/export-otlp-metrics-not-implemented
/api-reference/openapi.yaml post /v1/metrics
Not implemented. This endpoint authenticates the API key and returns `501`; no metrics are stored. Send traces to `/v1/traces` instead.
# Export OTLP traces
Source: https://docs.zespan.com/api-reference/opentelemetry/export-otlp-traces
/api-reference/openapi.yaml post /v1/traces
OTLP-compatible trace export endpoint. Accepts an `ExportTraceServiceRequest` payload as JSON (`application/json`) or protobuf (`application/x-protobuf`). Spans are converted to Zespan events and queued asynchronously.
Authenticated with `x-api-key`. Rate limited to 300 requests/minute per API key, capped at 1 MB and 100 spans per request, and counted against the organization's monthly event quota. Spans beyond the 100-span cap are reported in `partialSuccess.rejectedSpans`.
# Distinct outcome kinds reported for a project
Source: https://docs.zespan.com/api-reference/outcomes/distinct-outcome-kinds-reported-for-a-project
/api-reference/openapi.yaml get /v1/projects/{id}/outcomes/kinds
Returns the distinct `kind` values reported for this project in `[now - range, now]`, sorted alphabetically. Used to populate kind filters in the dashboard.
**Authenticated with a dashboard session (browser cookie), not `x-api-key`**, and requires the `dashboard:read` permission — the same auth model as Blast Radius.
# Outcome summary by agent or model, joined to cost
Source: https://docs.zespan.com/api-reference/outcomes/outcome-summary-by-agent-or-model-joined-to-cost
/api-reference/openapi.yaml get /v1/projects/{id}/outcomes/summary
Returns one row per distinct value of `dimension` (agent name or model name), summarizing every outcome reported in `[now - range, now]`: total outcomes, successes, total value, and the real LLM cost of the traces those outcomes are attributed to, plus two derived ratios (`costPerSuccess`, `valuePerDollar`). Backs the Value dashboard page's stat bar, chart, and breakdown table.
Cost is joined at read time from `llm_events` and is **not** windowed by `range` — a trace's cost is fixed at ingestion and never corrected, unlike an outcome, so only which outcomes count toward a row is affected by the time window; the cost attributed to those outcomes' traces is not.
A trace touching more than one model has its outcome attributed to an arbitrary one of them when `dimension=model` — outcomes are recorded at the trace level, not the span level, so a genuinely multi-model trace's outcome is not fanned out across every model it used.
**Authenticated with a dashboard session (browser cookie), not `x-api-key`**, and requires the `dashboard:read` permission — the same auth model as Blast Radius.
# Report business outcomes attributed to traces
Source: https://docs.zespan.com/api-reference/outcomes/report-business-outcomes-attributed-to-traces
/api-reference/openapi.yaml post /v1/ingest/outcomes
Report a batch of business outcomes (a deflected ticket, an avoided refund, an SLA met) attributed to traces your SDK already emitted. This is a direct, synchronous write to ClickHouse — not routed through the same async queue as `POST /v1/ingest` — since it's a single low-volume out-of-band call rather than a batched stream of trace events.
The trace an outcome names does not need to exist in Zespan yet. Outcomes and traces are joined at query time (`GET /v1/projects/{id}/outcomes/summary`), not at write time, so an outcome that arrives before its trace has finished ingesting is stored and joined correctly once the trace shows up. This is the expected shape, not an edge case: an outcome is usually known minutes or hours after the trace that produced it has already finished, from a customer's own backend (a support-desk webhook, a billing reconciliation job) — not from the same process that ran the agent.
Re-reporting the same `(kind, traceId)` pair is how you correct an earlier outcome — for example, a ticket that reopens after you'd already reported `ticket_deflected: true`. It is not an update to the earlier row; it's a new one, and the most recently ingested value is what reads (including the summary endpoint below) return.
**Authenticated with `x-api-key`**, the same as the rest of ingestion — this route is never reachable with a dashboard session.
# Org-wide policy inventory, grouped by policy id
Source: https://docs.zespan.com/api-reference/policies/org-wide-policy-inventory-grouped-by-policy-id
/api-reference/openapi.yaml get /v1/orgs/{orgId}/policies
Every policy id enforced (or once enforced) anywhere in the organisation, one group per id, with every project/environment occurrence underneath it. Backs the rollup cards and table on [Organisation view](/dashboard/policies#organisation-view).
**Default scope is every environment**, not production — `environment` and `productionOnly` are both opt-in narrowings, never the other way around. They narrow the same axis and are mutually exclusive: passing both is a 400.
`presentIn` / `absentFrom` on each group describe **coverage, not compliance** — the numerator and denominator are both projects "measurable" in the current scope. Under a narrowed scope, a project with no matching environment was never measurable and is excluded from the denominator entirely, rather than counted as absent. There is deliberately no field anywhere in this response naming a project non-compliant, failing, or in violation — no standard exists yet for a project to be measured against.
`lastAppliedAt` on an occurrence is bounded to the trailing 90 days and the most recent 2,000 applies across the organisation (whichever limit is hit first); `applyRecencyTruncated` reports whether the row cap was hit, and `applyRecencyCap` echoes the cap that ran. A `null` `lastAppliedAt` means no apply landed inside that window — it does **not** mean the policy was never applied, which is what `status: "never_applied"` on the same occurrence means instead.
`orgId` in the path selects the scope: it accepts either the organisation's slug or its id, and the caller must be a member of it or the request is refused (403). The handler reads the resolved active organisation rather than re-parsing the path segment itself, but the path segment is exactly what determined it.
**Authenticated with a dashboard session (browser cookie), not `x-api-key`**, and requires the `policy:read` permission, which every role (including viewer) has.
# Recent policy applies across every project in the org
Source: https://docs.zespan.com/api-reference/policies/recent-policy-applies-across-every-project-in-the-org
/api-reference/openapi.yaml get /v1/orgs/{orgId}/policy-applies
Every policy apply across every project in the organisation, newest first, cursor-paginated. Backs the **Recent applies** section on [Organisation view](/dashboard/policies#organisation-view).
`origin` is the **entrypoint** that ran the apply — `cli` or `ui` — which is a different fact from a policy's **owner** (`git` or `zespan`, returned as `origin` on an occurrence in the policies response above). A dashboard-authored (`zespan`-owned) policy can be applied by either entrypoint, and a `git`-owned policy applied in CI still reports `origin: "cli"` here; never read one field as the other.
`policyIds` is extracted from the apply's manifest server-side — the manifest itself is the largest column on the row and is never returned.
`orgId` in the path selects the scope — see the note on the policies endpoint above; the same resolution applies here.
**Authenticated with a dashboard session (browser cookie), not `x-api-key`**, and requires the `policy:read` permission, which every role (including viewer) has.
# Create a prompt version
Source: https://docs.zespan.com/api-reference/prompts/create-a-prompt-version
/api-reference/openapi.yaml post /v1/prompts
Create a new version of a prompt. If the named prompt already exists a new incremented version is created; otherwise version 1 is created. The `latest` label is always applied automatically.
# Get a prompt
Source: https://docs.zespan.com/api-reference/prompts/get-a-prompt
/api-reference/openapi.yaml get /v1/prompts/{name}
Fetch a prompt by name. Without `version` or `label` the latest version is returned. The response includes `resolvedPrompt` with prompt dependencies inlined.
# List prompt folders
Source: https://docs.zespan.com/api-reference/prompts/list-prompt-folders
/api-reference/openapi.yaml get /v1/prompts/folders
List the distinct folder paths used across the project's prompts.
# List prompt versions
Source: https://docs.zespan.com/api-reference/prompts/list-prompt-versions
/api-reference/openapi.yaml get /v1/prompts/{name}/versions
List every version of a prompt family, newest first.
# List prompts
Source: https://docs.zespan.com/api-reference/prompts/list-prompts
/api-reference/openapi.yaml get /v1/prompts
List prompts for the authenticated project. When using an API key the project is inferred from the key, so `projectId` is optional.
# Move a prompt to a folder
Source: https://docs.zespan.com/api-reference/prompts/move-a-prompt-to-a-folder
/api-reference/openapi.yaml patch /v1/prompts/{name}/folder
Move a prompt family into a folder path, or to the root by passing `null`. The folder is applied to every version of the named prompt.
# Set labels on a prompt version
Source: https://docs.zespan.com/api-reference/prompts/set-labels-on-a-prompt-version
/api-reference/openapi.yaml patch /v1/prompts/{name}/versions/{version}/labels
Replace the set of labels on a specific prompt version. Assigning the `production` label promotes that version and triggers a background quality regression check.
# Set tags on a prompt family
Source: https://docs.zespan.com/api-reference/prompts/set-tags-on-a-prompt-family
/api-reference/openapi.yaml patch /v1/prompts/{name}/tags
Replace the set of tags across all versions of a prompt family.
# Check whether spans are actually arriving for a project
Source: https://docs.zespan.com/api-reference/sdk-cli-support/check-whether-spans-are-actually-arriving-for-a-project
/api-reference/openapi.yaml get /v1/projects/{id}/ingest-health
Reports the most recent span timestamp, a 24h span count, and any recorded ingest rejections for a project — the data `zespan doctor`'s Data flow check reads to tell "your integration is broken" apart from "this project hasn't sent its first trace yet."
`lastSpanAt` is bounded to a 90-day lookback (so the underlying ClickHouse query can prune partitions on large, long-lived projects). A project whose most recent span is older than that window reports `lastSpanAt: null` — identically to a project that has never sent anything. The two cases are not distinguishable from this response alone.
`rejections` is currently always an empty array: nothing in the ingest path persists rejection reasons yet, so the field is shipped honestly empty rather than backed by invented bookkeeping.
**Authenticated with a dashboard session (browser cookie), not `x-api-key`** — unlike every other endpoint on this page. It requires the `dashboard:read` permission on the project's organization. A request carrying only a project API key does not satisfy this and receives `403`.
# Resolve project identity from an API key
Source: https://docs.zespan.com/api-reference/sdk-cli-support/resolve-project-identity-from-an-api-key
/api-reference/openapi.yaml get /v1/sdk/whoami
Answers "which project does this API key belong to?" — the one thing `GET /v1/sdk/config` can't answer, since that route requires the caller to already know the `projectId` and rejects a mismatch. This is `zespan doctor`'s first call, and how it establishes project identity before anything else in the doctor run.
Authenticated with `x-api-key`, same as every other endpoint on this page. Returns identity and the project's resolved SDK config only — never a secret: no API key hash, no raw key, no provider credentials.
# zespan auth
Source: https://docs.zespan.com/cli/auth
Sign in to the CLI as yourself with an OAuth 2.0 device grant, so control-plane commands act as a person with a role — not as a project API key.
`zespan auth` gives the CLI a credential that belongs to **you** rather than to a
project. Control-plane commands — `zespan policy pull`, `test`, `plan`, `apply`,
`zespan link`, `zespan projects list` — check your role in the organization, so
they need a person's session. A project API key cannot supply one.
**Ingest and the data plane are unchanged.** `/v1/ingest`, guardrail checks,
the SDKs, and [`zespan doctor`](/cli/doctor) still authenticate with
`ZESPAN_API_KEY`. Signing in adds a second credential; it does not replace the
first. See [the two credentials](/cli/overview#two-credentials-two-planes).
## Quick start
```bash npx theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
npx @zespan/cli auth login
```
```bash Installed theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
zespan auth login
```
```text theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
── Sign in to Zespan ───────────────────────────────────────
ℹ Your code: 3KS23N6T
Open: https://app.zespan.com/device?user_code=3KS23N6T
Approving a terminal asks for your second factor, even though you are already signed in.
This machine has not signed in before — approving it registers it as a new device.
Approved.
✔ Signed in. Credential written to /home/you/.zespan/credentials.json (readable only by you).
You are Ada Lovelace .
Organizations: Acme Health.
Next: run `zespan link` to choose a project for this directory.
```
The CLI prints the second-factor line to everyone, because it cannot know which
account will approve. If your account has no second factor configured, the
approval screen simply asks you to confirm instead — see [Approving a
terminal](#approving-a-terminal).
## How the login works
`zespan auth login` is an [OAuth 2.0 device authorization grant
(RFC 8628)](https://datatracker.ietf.org/doc/html/rfc8628) — the same shape as
signing a TV into a streaming service. The terminal never sees your password.
It requests a short user code from the API, sending this machine's public key
and — when it can determine them — its operating system, OS version, and CLI
version.
The CLI prints the code and opens the dashboard's `/device` screen with the
code pre-filled. The code is valid for **10 minutes**.
Check what the screen says about the terminal that asked, then authorize it.
If your account has a second factor, you re-enter it here even though you are
already signed in. See [Approving a terminal](#approving-a-terminal).
It polls every 5 seconds until you approve, then writes the token to
`~/.zespan/credentials.json` and caches who you turned out to be so later
commands can name you without a network call.
## Approving a terminal
What the approval screen asks of you depends on whether your account has
two-factor authentication turned on.
| Your account | What approval takes |
| ------------------------ | ----------------------------------------------------------------------------------------------------------------------------------- |
| **Has a second factor** | You re-enter a TOTP code, an emailed one-time code, or a backup code, then authorize — even though the browser is already signed in |
| **Has no second factor** | You review the terminal that asked and authorize. There is no code to enter, and nothing is blocked |
The requirement on an enrolled account is enforced by the API as well as by the
page, so a request that skips the screen entirely is refused too. The proof is
bound to the exact browser session that supplied it and is good for **five
minutes**, so verifying in one browser cannot authorize a terminal from another.
**Turning two-factor authentication on is worth it here.** It adds the
confirmation step to this screen and to your sign-ins, so a terminal cannot be
authorized from a session someone walked up to on an unlocked machine — which
is the attack this step exists for, and the one a re-entered code actually
stops. Turn it on under **Settings → Profile → Two-factor authentication**. It
is not required to use the CLI: the same account can already apply enforcement
policy from the dashboard without one, and this path is not the place to
invent a stricter rule.
A device grant is phishable by construction: someone can start
`zespan auth login` on their own machine and read the eight-character code to you
over the phone. Two things on the approval screen exist to make that attempt
fail.
### The terminal that asked
On both approval paths, before you authorize, the screen shows what the server
observed about the machine that made the request:
| Row | Where it comes from |
| ----------------- | ----------------------------------------------- |
| Operating system | Reported by the CLI when it asked for the code |
| CLI version | Reported by the CLI when it asked for the code |
| Request came from | The IP address the API resolved for the request |
None of it comes from the link you followed, and none of it is editable. If it
does not describe the machine in front of you, cancel. An older CLI that reported
nothing about itself says exactly that, rather than showing a blank.
**Self-hosters: the IP row is only as trustworthy as your proxy.** Zespan
resolves the address from `X-Forwarded-For`, so an edge that appends to a
client-supplied value — or an API reachable without going through the edge —
lets a caller choose the address a human is shown here. On the row that exists
precisely to be corroborated by eye, that is worth an hour of your time. See
[Client IP behind a proxy](/platform/client-ip).
### The warning
The screen states, before you authorize, that the code should already be on
screen in a terminal in front of you — and that if someone read it to you, you
should cancel.
## Machines are remembered
The first login on a machine mints an Ed25519 keypair at `~/.zespan/device-key.pem`
(with the public half beside it) and sends the public key with the grant. The API
fingerprints it and records the machine, so signing in again from the same
machine keeps the **same** device identity rather than registering a new one
every time.
That identity is what makes the audit trail readable: a policy apply records the
user, the session, and the device it came from, not just a name. See
[Attribution](/cli/policy#every-apply-names-a-person).
`zespan auth logout` removes the credential but **keeps the device key**, so
signing back in reuses the same device.
## Abandoned logins are cleaned up
A login you start and never finish leaves a pending grant on the server, holding
the code, the machine's public key, and the address the request came from. Two
behaviours bound that, and both are worth knowing before you script anything
around `zespan auth login`.
**Starting a login is rate-limited.** Requesting a code is capped at **30 per 5
minutes per IP address**. That is generous enough for a whole team behind one
office address onboarding together, and it applies only to *starting* a login —
the CLI's polling for the result is deliberately not counted, since one login
polls roughly 120 times.
**Pending grants are swept.** A code is valid for 10 minutes; a grant that has
expired is deleted **15 minutes** later by a sweep that runs **every 15 minutes**,
so nothing survives beyond roughly 40 minutes from the moment it was requested.
The grace period is not an oversight — it is what lets a slow client be told its
code expired rather than getting a bare error.
**Self-hosters: both depend on infrastructure that is easy to leave out.** The
rate limit needs `REDIS_URL` configured on the API — without Redis it does not
apply at all. The sweep runs in the **worker** process, and only when
`WORKER_TYPE` is unset or `all`; if you split workers by type and never run a
general one, nothing consumes the sweep queue and expired grants accumulate
indefinitely. See [Environment
variables](/platform/environment-variables#workers).
## Files on disk
| Path | Contents | Commit it? |
| ---------------------------- | -------------------------------------------------------------------------- | -------------------------- |
| `~/.zespan/credentials.json` | Your session token, the host it belongs to, and your cached name and email | **Never** — it is a secret |
| `~/.zespan/device-key.pem` | This machine's private device key | **Never** |
| `~/.zespan/device-key.pub` | The matching public key, replayed on every login | **Never** |
| `/.zespan/config.json` | The linked project and org — written by [`zespan link`](/cli/link) | Yes — it holds no secret |
The credential and key files are written with owner-only permissions (`0600`)
and re-tightened on every write, so an older CLI that left the file world-readable
is fixed the next time you sign in. Windows has no POSIX mode to set; there the
files rely on your user profile's permissions.
**A token is tied to the host it was minted against.** If you sign in to
`https://api.zespan.com` and then run a command with `--api-url` pointing
somewhere else, the CLI refuses rather than sending your session token to a host
it was not issued for.
## Commands
```text theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
Usage: zespan auth [flags]
Commands:
login Sign in with your Zespan account and store a device credential
status Show who you are signed in as, and what this directory is linked to
logout Remove the stored credential from this machine
Flags:
--no-browser Print the URL instead of opening a browser (SSH, CI, headless)
--api-url API base URL (or ZESPAN_API_URL)
--app-url Dashboard base URL for the approval page (or ZESPAN_APP_URL)
--help Show this help
```
### `zespan auth login`
```bash theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
zespan auth login
```
On a machine with no browser — an SSH session, a container, a headless build
host — pass `--no-browser`. The CLI prints the URL and the code instead of trying
to open anything, and you approve them from any other machine:
```bash theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
zespan auth login --no-browser
```
Without `--no-browser`, a login started where there is no terminal to open a
browser from is refused immediately, before a code is minted — rather than
hanging for ten minutes on a code nobody can see.
### `zespan auth status`
```bash theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
zespan auth status
```
Prints the resolved identity: the account, the host, where the credential lives,
its expiry if the server reported one, the organizations you reach with your role
in each, and the project linked in this directory.
It still answers when the network is down — the account details fall back to the
copy cached at login, and the output says so rather than silently showing stale
data as if it were live.
### `zespan auth logout`
```bash theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
zespan auth logout
```
Removes `~/.zespan/credentials.json` from this machine. The device key stays, so
the next login is recognised as the same machine. Running it when nothing is
stored says so rather than pretending to have done something.
## Self-hosted deployments
The API tells the CLI where to send you by building the approval URL from
`NEXT_PUBLIC_APP_URL` — the same variable that already builds invitation and
two-factor links. Set it to your dashboard's public origin; see [Environment
variables](/platform/environment-variables). The approval screen is the web app's
`/device` route.
If you need to override it for a single login — a tunnel, a preview deployment —
pass `--app-url` (or set `ZESPAN_APP_URL`):
```bash theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
zespan auth login --api-url https://api.internal.example --app-url https://zespan.internal.example
```
One more thing to check before you hand this to a team: the approval screen shows
the IP the request came from, and that row means what it says only if your proxy
overwrites `X-Forwarded-For`. See [Client IP behind a
proxy](/platform/client-ip).
## When a login fails
Someone pressed **Cancel this request**. Nothing was granted. If it was not
you, that is the control working — do not approve a code you did not start.
The grant lives for 10 minutes. Run `zespan auth login` again for a fresh
code.
A revoked machine cannot re-register itself by signing in again. Remove the
revocation, or delete `~/.zespan/device-key.pem` to sign in as a new device —
the CLI prints the exact path.
The stored token was rejected. Run `zespan auth login` to sign in again from
this machine.
The token was minted and written — the login succeeded. Only the follow-up
read of your account details did not answer. Run `zespan auth status` once the
network is back.
## Next steps
Choose the project this directory belongs to, and list what you can reach.
The commands that need this sign-in, and what an apply records about you.
The two credentials, configuration precedence, and install options.
The project key the SDKs and `zespan doctor` still use.
What self-hosters must configure so the approval screen's IP row is real.
# zespan doctor
Source: https://docs.zespan.com/cli/doctor
Diagnose SDK setup issues — bad API key, unreachable API, no data arriving, or a provider client wrapped in the wrong order — before they turn into a churned trial.
`zespan doctor` runs a fixed set of checks against your project and prints a
pass/warn/fail/skip summary. It exists for one specific, common failure: a
customer installs the SDK, sees nothing in the dashboard, and has no way to
tell whether the problem is their API key, their network, their code, or
nothing at all (a project that just hasn't sent its first trace yet).
## Quick start
```bash npx theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
npx @zespan/cli doctor --api-key $ZESPAN_API_KEY
```
```bash Installed theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
zespan doctor
```
With no flags, `doctor` reads `ZESPAN_API_KEY` from the environment and
`.zespan.yaml` from the current directory — see [CLI overview](/cli/overview)
for the full precedence rules. Self-hosted deployments should also pass
`--api-url` (or set `ZESPAN_API_URL`).
## What it checks
Checks run in a fixed order, grouped under four headings. Every `fail` or
`warn` result comes with at least one actionable fix line, printed indented
under it (`→`).
### Configuration
Confirms the key authenticates and reports which project it belongs to.
This is the first check, and the only one whose *failure* short-circuits
every check below it (see [Short-circuit on a bad key](#short-circuit-on-a-bad-key)).
* **pass** — `Valid — project "" ()`
* **fail** — the key was flatly rejected (HTTP 401). Fix: double-check
`ZESPAN_API_KEY` / `--api-key` against **Settings → API Keys**, or
rotate if the key was revoked.
* **skip** — a network failure, not a rejected key (DNS, TLS, timeout,
connection refused). Fix: check connectivity and any corporate proxy.
Independent of the whoami call — this hits the API host's root
`/health` route, so it still runs and gives a useful second data point
even when the key check above failed on the network (not on the key).
* **pass** — ` responded in ms`
* **fail** — names the host it couldn't reach, and lists any
`HTTP_PROXY` / `HTTPS_PROXY` / `NO_PROXY` (or lowercase) environment
variables that were set at the time — or says none were set, since an
unconfigured egress proxy on a corporate network is a common cause.
Reports the resolved `--env` / `ZESPAN_ENVIRONMENT` / `.zespan.yaml`
`environment` value, or **skip** if none is configured. Purely
informational — there's nothing to pass or fail here.
### SDK
Compares the `@zespan/sdk` version against the latest version the API
reports. When `node_modules/@zespan/sdk/package.json` is readable,
compares the ACTUALLY RESOLVED installed version; otherwise falls back to
the range declared in your `package.json` (`dependencies` or
`devDependencies`, e.g. `^1.0.0`) and labels it "declared range" rather
than "installed", since a range's floor isn't necessarily what's really
installed.
* **skip** — `@zespan/sdk` isn't a dependency at all (a Python-only or
OTel-native project is expected to skip this), or the API didn't
report a latest version to compare against.
* **warn** — the compared version is behind. Fix: `pnpm add @zespan/sdk@latest`.
* **pass** — up to date.
See [Wrapper ordering: a heuristic, not a guarantee](#wrapper-ordering-a-heuristic-not-a-guarantee) below.
### Data flow
Checks whether spans have actually arrived recently, using the project id
you configured (or the one your API key resolves to). Reports a **fail**
once your integration has gone more than 15 minutes without a span, with
the last-seen timestamp and 24h span count.
A `lastSpanAt` of `null` — no span in the last 90 days — is reported as
**skip**, not fail: the API bounds this query to a 90-day lookback for
ClickHouse partition pruning, so a brand-new project that hasn't sent its
first trace yet and a project whose integration went quietly dead over 90
days ago are indistinguishable from this response alone. `doctor` says so
explicitly rather than guessing, and gives a fix path for both readings.
If `--project`/`ZESPAN_PROJECT_ID` names a project that doesn't match the
one your API key actually belongs to, this check **fails** with a plain
message naming both project ids — one of the five causes `doctor` exists to
diagnose — instead of the raw `HTTP 403: {"error":"API key does not match
the requested project"}` the API returns.
Any ingest rejections recorded for the project are folded into the detail
and fix lines by reason and count.
### Policy & redaction
Reads the project's resolved redaction policy from the same whoami call.
* **pass** — the server reports redaction as on (names the preset, if one
is set).
* **skip** — the server didn't report a redaction policy at all, OR
reported it as off. The server-side policy has no way to be changed
today, so `doctor` never `warn`s about it — a warning a user has no way
to clear is worse than no check at all. The skip detail still names the
real, working client-side fix (`zespan.init({ redactPii: true })`,
optionally scoped with a preset: `gdpr`, `hipaa`, `ccpa`, `pci-dss`,
`soc2`, `finance`, `education`, or `transportation`) in case your SDK is
already configured that way — client-side redaction happens before data
reaches the API, which this check can't see.
## Wrapper ordering: a heuristic, not a guarantee
A provider client (`OpenAI`, `Anthropic`, `Mistral`, `Groq`, `Cohere`,
`GoogleGenAI`) constructed **before** `zespan.init()` runs never gets patched
by the SDK — every call it makes is invisible to Zespan. This is the single
most common cause of "I installed the SDK and see nothing," so `doctor`
scans your project's source files (TypeScript, JavaScript, and Python; up to
500 files, 8 directories deep, skipping `node_modules`/`dist`/`.venv` and
similar) looking for it.
The rule it applies is deliberately narrow: it only flags a provider
constructor at **module scope** (zero leading indentation — not inside a
function, method, or class body) in a file where `zespan.init(` does not
appear on an earlier line of that same file.
**This is a heuristic and it can produce false positives.** The check reads
one file at a time — it has no way to know that `zespan.init()` already ran
in a *different* module that got imported first. If your actual entry point
calls `zespan.init()` before it imports the file `doctor` flagged, the
provider client in that file genuinely runs after init and the warning is a
false positive.
**How to tell:** trace your own import graph. If the module that calls
`zespan.init()` is imported (directly or transitively) before the flagged
file anywhere in your app, you're safe — the flagged construction really
does run after init, `doctor` just can't see across files to confirm it.
If you can't establish that ordering, treat the warning as real.
```
✓ No module-scope provider clients found before zespan.init() ← pass
⚠ 1 provider client constructed at module scope before ← warn
zespan.init() in your project.
→ src/openai.ts:3 — construct the OpenAI client after
zespan.init(), or lazily: move zespan.init() into a module
imported before src/openai.ts, or construct the OpenAI client
lazily inside the handler that uses it instead of at module scope.
(no zespan.init( call was found on an earlier line in this file.)
```
## Sample output
This is real output from `zespan doctor` run with no configured API key
(`ZESPAN_API_KEY` unset), showing the short-circuit behavior described below:
```
Configuration
✗ API key Key rejected: Request to api.zespan.com failed with HTTP 401: {"error":"Not authenticated"}
→ Double-check ZESPAN_API_KEY (or --api-key) against the key shown in Settings -> API Keys for this project.
→ If the key was recently rotated or revoked, generate a new one and update your environment.
− Reachable Skipped: "API key" failed above.
− Environment Skipped: "API key" failed above.
SDK
− Sdk version Skipped: "API key" failed above.
− Wrapper order Skipped: "API key" failed above.
Data flow
− Data flow Skipped: "API key" failed above.
Policy & redaction
− Redaction Skipped: "API key" failed above.
1 error, 0 warnings.
```
## Behaviors worth knowing
If the API key check itself reports `fail` (the key was flatly rejected —
a 401, not a network problem), every later check is marked `skip` without
ever running. Six confusing failures caused by one bad key is a worse
diagnostic experience than one clear failure, so `doctor` stops there.
This only triggers on `apiKeyValid` specifically failing — a network
failure (`skip`, not `fail`) does not short-circuit; `API reachability`
still runs independently and gives you a second data point.
If a check throws for any reason, `doctor` catches it and reports that
check as `fail` carrying the thrown message, instead of the whole command
crashing. `doctor` is the tool people reach for when something is already
broken — it has to stay usable when things go wrong in unexpected ways.
Only a `fail` anywhere in the results makes `doctor` exit `1`. A `warn` —
a stale SDK version, redaction disabled — is real, actionable advice, but
it is not a broken setup, and a CI job gating on `doctor`'s exit code must
not be blocked by advice it can't act on immediately.
| Exit code | Meaning |
| --------- | ------------------------------------------------------- |
| `0` | No check reported `fail` (there may still be warnings). |
| `1` | At least one check reported `fail`. |
| `2` | Usage error — an unrecognized command. |
## `--json` output
`zespan doctor --json` prints exactly one JSON line — `{ results, summary }`
— and suppresses the grouped-text renderer, for CI consumption.
```bash theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
zespan doctor --json
```
Real output from the same unauthenticated run shown above:
```json theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
{"results":[{"id":"apiKeyValid","group":"Configuration","status":"fail","title":"API key","detail":"Key rejected: Request to api.zespan.com failed with HTTP 401: {\"error\":\"Not authenticated\"}","fixes":["Double-check ZESPAN_API_KEY (or --api-key) against the key shown in Settings -> API Keys for this project.","If the key was recently rotated or revoked, generate a new one and update your environment."]},{"id":"reachable","group":"Configuration","status":"skip","title":"Reachable","detail":"Skipped: \"API key\" failed above."},{"id":"environment","group":"Configuration","status":"skip","title":"Environment","detail":"Skipped: \"API key\" failed above."},{"id":"sdk-version","group":"SDK","status":"skip","title":"Sdk version","detail":"Skipped: \"API key\" failed above."},{"id":"data-flow","group":"Data flow","status":"skip","title":"Data flow","detail":"Skipped: \"API key\" failed above."},{"id":"redaction","group":"Policy & redaction","status":"skip","title":"Redaction","detail":"Skipped: \"API key\" failed above."},{"id":"wrapper-order","group":"SDK","status":"skip","title":"Wrapper order","detail":"Skipped: \"API key\" failed above."}],"summary":{"errors":1,"warnings":0,"passed":0,"skipped":6}}
```
| Field | Type | Notes |
| ------------------ | ---------- | ---------------------------------------------------------------- |
| `results` | array | One entry per check, in the fixed run order. |
| `results[].id` | string | Stable check id, e.g. `apiKeyValid`, `data-flow`. |
| `results[].group` | string | `Configuration` \| `SDK` \| `Data flow` \| `Policy & redaction`. |
| `results[].status` | string | `pass` \| `warn` \| `fail` \| `skip`. |
| `results[].title` | string | Human-readable check name. |
| `results[].detail` | string? | Human-readable explanation. |
| `results[].fixes` | string\[]? | Actionable next steps. Always present on `fail`/`warn`. |
| `summary.errors` | number | Count of `fail` results. |
| `summary.warnings` | number | Count of `warn` results. |
| `summary.passed` | number | Count of `pass` results. |
| `summary.skipped` | number | Count of `skip` results. |
## Flags
Your Zespan API key. Falls back to `ZESPAN_API_KEY`. Never read from `.zespan.yaml`.
Project id, if it differs from the one your API key resolves to. Falls back to `ZESPAN_PROJECT_ID` or `.zespan.yaml`'s `project`.
Environment name for the Environment check. Falls back to `ZESPAN_ENVIRONMENT` or `.zespan.yaml`'s `environment`.
Override the API base URL. Falls back to `ZESPAN_API_URL`. Never read from `.zespan.yaml` (same treatment as `--api-key` — see [CLI overview](/cli/overview)). Needed for a self-hosted deployment.
Print `{ results, summary }` as a single JSON line instead of grouped text, and disable colour output.
Print this command's usage and exit — does not run any checks or make any network calls.
See [CLI overview](/cli/overview) for the full flag/env/file precedence rules.
## Next steps
Installation, `.zespan.yaml`, and environment variables.
Symptom-first fixes for missing traces, broken span trees, and \$0.00 cost.
Configure `redactPii` and choose a preset.
Create and rotate the key `doctor` authenticates with.
A separate credential, for the commands that act as a person rather than a project.
# zespan link
Source: https://docs.zespan.com/cli/link
Pick the Zespan project a directory belongs to and write it to a committable .zespan/config.json — and list every project your sign-in can reach, without prompting, for CI.
`zespan link` records which Zespan project the current directory belongs to, so
`zespan policy plan` and `apply` do not need a project id on the command line.
`zespan projects list` prints the same data without prompting, which is what CI
needs.
Both commands need a user session — run [`zespan auth login`](/cli/auth) first.
An API key authenticates a project, not a person, so it cannot answer "which
projects can *you* reach?".
## Quick start
```bash theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
zespan auth login
zespan link
```
```text theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
✔ Linked Acme Health / Intake Prod.
Project: Intake Prod (2845c0ef-1a3d-4f10-9b77-6c1f0a2d8e41)
Organization: Acme Health (acme-health, your role: admin)
Written to /home/you/checkout/.zespan/config.json — it holds no secret, so commit it.
```
The picker groups projects under the organization that owns them, and every row
carries the org name alongside the project name — because the ambiguity this
command exists to resolve, the same project name in two organizations, is
invisible from the project name alone.
## `.zespan/config.json`
```json .zespan/config.json theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
{
"project": {
"id": "2845c0ef-1a3d-4f10-9b77-6c1f0a2d8e41",
"name": "Intake Prod"
},
"org": {
"slug": "acme-health",
"name": "Acme Health"
},
"environment": "prod"
}
```
**Commit this file.** It carries no credential and cannot redirect one — there
is no `apiKey` field and no `apiUrl` field, so a teammate cloning the repo picks
up the project without picking up a secret, and a hostile edit cannot change
where your token is sent.
A Zespan project has an id and a name, but no slug — so a bare
`ZESPAN_PROJECT_ID=2845c0ef-…` tells a human nothing about which organization it
belongs to. The link config stores the readable names **beside** the id for
exactly that reason, and error messages print the names rather than the UUID.
Re-running `zespan link` to change project keeps the environment already recorded
in the file unless you pass a new `--env`.
## `zespan link`
```text theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
Usage: zespan link [flags]
Flags:
--project Link this project id without prompting (works in CI)
--env Record a default environment alongside the project
--api-url API base URL (or ZESPAN_API_URL)
--help Show this help
```
The picker needs a terminal. Over a pipe or in CI, `zespan link` refuses
**before** it makes any network call and names `--project` as the alternative,
rather than blocking forever on input nobody will type:
```bash theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
zespan link --project 2845c0ef-1a3d-4f10-9b77-6c1f0a2d8e41 --env prod
```
A `--project` id your sign-in cannot reach writes nothing, and says so by id
rather than leaving a config file naming a project that does not exist.
## `zespan projects list`
```text theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
Usage: zespan projects list [flags]
Flags:
--json Print machine-readable JSON
--api-url API base URL (or ZESPAN_API_URL)
--help Show this help
```
Never prompts, so it is safe in CI and over a pipe.
```text theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
ORGANIZATION PROJECT ID
* Acme Health (acme-health) Intake Prod 2845c0ef-1a3d-4f10-9b77-6c1f0a2d8e41
Acme Health (acme-health) Sandbox 9f14bb02-7c55-4e88-b0a2-3d5e7f1c4a90
* linked in this directory
```
`--json` prints the same data as `{ user, projects }`, with `id`, `name`,
`orgSlug`, `orgName` and `role` per project. It goes straight to stdout with no
indentation or colour, so it pipes cleanly into `jq`.
Only projects you can actually reach are listed — every organization you are a
member of, and every project in it that has not been deleted. It is the same list
the dashboard shows you.
## Where the project id comes from
For `zespan policy` and any other command that needs a project, the resolution
order is:
**`--project` flag → `ZESPAN_PROJECT_ID` → `.zespan/config.json` → `.zespan.yaml`**
`zespan link` outranks `.zespan.yaml` because it is the newer, explicit act
("link *this* directory to *that* project"). Nothing changes for existing
projects, which have no `.zespan/config.json` until someone runs `zespan link`.
The environment resolves the same way: `--env` → `ZESPAN_ENVIRONMENT` →
`.zespan/config.json` → `.zespan.yaml`.
The API key and the API base URL are never read from either file — see
[CLI overview](/cli/overview).
## Next steps
Sign in first — both commands need a user session.
Plan and apply guardrail policies against the project you just linked.
Configuration precedence and the two credentials.
What `--env` targets, and how environment slugs resolve.
# CLI overview
Source: https://docs.zespan.com/cli/overview
Install @zespan/cli, understand its two binaries (zespan and zespan-gate), and configure it with flags, environment variables, or a committed .zespan.yaml.
`@zespan/cli` is a small, dependency-free command-line tool. It ships two binaries:
| Binary | What it runs | Status |
| ------------- | ---------------------------------------------------------------------------------------------------- | --------- |
| `zespan` | The full command registry — `auth`, `link`, `projects`, `doctor`, `policy` and `gate` as subcommands | New |
| `zespan-gate` | The [CI quality gate](/sdk/cli) directly, with no subcommand | Unchanged |
`zespan-gate` is not deprecated and nothing about it changed. If you already
call `zespan-gate` from a CI pipeline, keep doing exactly that — `zespan gate`
is the same logic reachable through the new binary, not a replacement for it.
## Install
```bash npm theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
npm install --save-dev @zespan/cli
```
```bash pnpm theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
pnpm add -D @zespan/cli
```
```bash yarn theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
yarn add -D @zespan/cli
```
This installs both `zespan` and `zespan-gate` into `node_modules/.bin`. You can also run either without installing:
```bash theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
npx @zespan/cli doctor
npx @zespan/cli gate --name my-prompt --version 3 --dataset-run-id run_abc123 --evaluator-id eval_def456
```
## Commands
```
Usage: zespan [flags]
Commands:
auth Sign in to your Zespan account from this machine (login, status, logout)
link Choose the project this directory belongs to
projects List every project your sign-in can reach (projects list)
doctor Diagnose SDK setup issues (config, connectivity, data flow, redaction)
policy Manage policy-as-code files (init, validate, plan, apply)
gate CI quality gate for prompt versions (same binary as zespan-gate)
Flags:
--help Show this help
--version Print the CLI version
Run `zespan doctor --help`-style flags directly on the doctor command, e.g.:
zespan doctor --json
zespan doctor --project --api-key
```
That's the literal output of `zespan --help` (also shown by `zespan` with no
arguments). An unrecognized command prints the same text to stderr and exits `2`.
* **`zespan auth`** — signs this machine in as *you*, with an OAuth 2.0 device grant a human approves in the browser. See [zespan auth](/cli/auth).
* **`zespan link`** / **`zespan projects list`** — choose the project a directory belongs to, and list every project you can reach. See [zespan link](/cli/link).
* **`zespan doctor`** — diagnoses SDK setup problems: bad or missing API key, unreachable API, no data arriving, PII redaction posture, and the most common "I installed the SDK and see nothing" mistake. See [zespan doctor](/cli/doctor).
* **`zespan policy`** — authors, plans and applies guardrail [policy-as-code](/policies/as-code) files. See [zespan policy](/cli/policy).
* **`zespan gate`** — gates a prompt version's quality in CI. Identical to `zespan-gate`; see [CI quality gate](/sdk/cli) for the full flag reference and exit code contract.
## Two credentials, two planes
The CLI can hold two credentials, and they are not interchangeable. Which one a
command uses is decided by what the command does, not by which one you happen to
have set.
| | Data plane | Control plane |
| ----------------- | ------------------------------------------------------------------------ | --------------------------------------------------------------------------------------- |
| **Credential** | Project API key | Your user session |
| **Sent as** | `x-api-key` header | `Authorization: Bearer` header |
| **Obtained from** | **Settings → API Keys** | `zespan auth login` |
| **Identifies** | One project | One person, with a role in each organization |
| **Used by** | The SDKs, `/v1/ingest`, guardrail checks, `zespan doctor`, `zespan gate` | `zespan policy pull` / `test` / `plan` / `apply`, `zespan link`, `zespan projects list` |
**An API key cannot run a control-plane command.** Those routes check your role
in the organization, and a key authenticates a project — there is no role to
check. `zespan policy plan` with only `ZESPAN_API_KEY` set now refuses with an
explanation instead of sending a request that could only be rejected. Run
[`zespan auth login`](/cli/auth) first.
If both are available, the sign-in wins on the control plane and the API key is
left untouched for the data plane — the two identities never silently mix. A
stored sign-in that is expired, or that was minted against a different API host,
produces a refusal naming the fix rather than a quiet fallback to the API key.
## Configuration file — `.zespan.yaml`
Both `zespan` commands read an optional `.zespan.yaml` from the current working directory. It supports a flat `key: value` format only — no nested maps, lists, or multi-line scalars. A line the parser can't make sense of is silently ignored rather than rejected, so a `.zespan.yaml` written for a future CLI version degrades gracefully on an older one instead of breaking it.
```yaml .zespan.yaml theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
project: 3f9c2e10-...
environment: production
```
| Key | Aliases | Maps to |
| ------------- | ----------- | ---------------- |
| `project` | `projectId` | Project id |
| `environment` | `env` | Environment name |
**`.zespan.yaml` never carries your API key or your API URL**, even though
the parser would technically accept `apiKey:` or `apiUrl:` lines — neither
key is in the recognized set, and both are silently ignored. This is
deliberate, not an oversight: `.zespan.yaml` is meant to be committed to
your repository, and a secret in a committed file is a leaked secret. A
committed file that could choose *where* your key gets sent is the same
leak in one more hop — a malicious `apiUrl` could redirect your
`ZESPAN_API_KEY` to an attacker-controlled host via the `x-api-key` header.
Set the API key only via `--api-key` or `ZESPAN_API_KEY`, and the API URL
only via `--api-url` or `ZESPAN_API_URL`.
## Environment variables
| Variable | Equivalent flag | Notes |
| -------------------- | --------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `ZESPAN_API_KEY` | `--api-key` | Never read from `.zespan.yaml` — see above. |
| `ZESPAN_API_URL` | `--api-url` | Never read from `.zespan.yaml` — see above. Defaults to `https://api.zespan.com/v1`. |
| `ZESPAN_PROJECT_ID` | `--project` | Set this when it should differ from the project your API key resolves to. If it's set AND doesn't match, `doctor`'s Data flow check fails with a plain "wrong project" diagnosis instead of a raw HTTP 403. |
| `ZESPAN_ENVIRONMENT` | `--env` | Surfaced by `doctor`'s Environment check, and recorded by `zespan link` as this directory's default environment. |
| `ZESPAN_APP_URL` | `--app-url` | `zespan auth login` only. The dashboard origin to open the approval page on. Normally unnecessary — the API tells the CLI where to send you. |
## Precedence
For every value except the API key and the API URL: **flags > environment variable > `.zespan/config.json` > `.zespan.yaml` > built-in default**. The API key and the API URL both skip both files entirely — they only ever come from `--api-key`/`ZESPAN_API_KEY` and `--api-url`/`ZESPAN_API_URL` respectively.
`.zespan/config.json` is written by [`zespan link`](/cli/link) and carries the project and org. It sits above `.zespan.yaml` because it is the newer, explicit act; a project with no `.zespan/config.json` behaves exactly as before.
## Next steps
Sign in as yourself, and what the approval screen checks.
Link a project to a directory, and list what you can reach.
What each check verifies, sample output, and how to read a failure.
Author, plan and apply guardrail policies from your repository.
The `zespan gate` / `zespan-gate` flag reference and exit code contract.
Create and rotate the key `zespan doctor` and `zespan gate` authenticate with.
# zespan policy
Source: https://docs.zespan.com/cli/policy
Author, validate, plan and apply guardrail policies from your repository — with validate running fully offline so it is safe as a pre-commit hook.
`zespan policy` manages [policy-as-code](/policies/as-code) files: the YAML
policies in your repository that compile into guardrails.
## Quick start
```bash npx theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
npx @zespan/cli policy validate
```
```bash Installed theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
zespan policy validate
```
## You must be signed in
`pull`, `test`, `plan` and `apply` check your **role in the organization** that
owns the project. That is a question about a person, so they need a person's
session:
```bash theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
zespan auth login # once per machine
zespan link # once per repository
zespan policy plan --env prod
```
**An API key cannot run these commands, and the CLI now says so instead of
sending the request.** `ZESPAN_API_KEY` / `--api-key` authenticates a *project*,
not a person, and the server has no member role to check for one — so every
such request was already refused with a `403` that read like an org-membership
problem. The CLI refuses up front now:
```text theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
An API key cannot run this command.
`policy pull`, `test`, `plan` and `apply` check your role in the organization, and an API key authenticates a project, not a person.
Run `zespan auth login` to sign in from this machine, then re-run this command.
```
If you have a CI job that passes `ZESPAN_API_KEY` to `zespan policy plan`, it
was never working — see [In CI](#in-ci) for what to run instead.
If both a sign-in and an API key are present, the sign-in wins for these
commands; the API key is left alone for the data plane and
[`zespan doctor`](/cli/doctor). See [the two
credentials](/cli/overview#two-credentials-two-planes).
## Verbs
| Verb | What it does | Needs the network? | Needs sign-in? |
| ---------- | --------------------------------------------------------- | ------------------ | -------------- |
| `init` | Scaffolds `policies/` with a starter policy | **No** | **No** |
| `validate` | Checks every policy file | **No** | **No** |
| `pull` | Generates policy files from guardrails that already exist | Yes | Yes |
| `test` | Backtests policies against your recorded traffic | Yes | Yes |
| `plan` | Shows what applying would change | Yes | Yes |
| `diff` | Alias for `plan` | Yes | Yes |
| `apply` | Applies the current policy files | Yes | Yes |
`validate` and `init` make **no network call and need no credential of any
kind**. That is deliberate: `validate` is meant to run as a pre-commit or
pre-push hook, and a hook that needs the network is a hook people disable. The
parser and the full schema are compiled into the binary.
## Flags
| Flag | Applies to | Meaning |
| -------------------- | ------------------------------- | -------------------------------------------------------------------------------------------------- |
| `--env ` | `pull`, `plan`, `apply` | Target environment. Defaults to the project default |
| `--project ` | `pull`, `test`, `plan`, `apply` | Project id. Or `ZESPAN_PROJECT_ID`, or [`zespan link`](/cli/link), or `project:` in `.zespan.yaml` |
| `--api-url ` | `pull`, `test`, `plan`, `apply` | For self-hosted deployments. Or `ZESPAN_API_URL` |
| `--allow-remove` | `apply` | Permit removing policies that left the file set |
| `--adopt` | `apply` | Permit taking over dashboard-authored guardrails |
| `--untested` | `apply` | Promote to `deny` with no backtest behind it |
| `--against ` | `test` | `issues`, `last:7d` or `dataset:` (default `last:7d`) |
| `--all` | `pull` | Include policies already managed in code |
| `--force` | `apply` | Apply despite a stale plan or a detached-policy conflict |
| `--json` | `plan`, `apply` | Machine-readable output |
| `--help` | any | Prints usage, with no network call |
Configuration precedence is the same as every other `zespan` subcommand — see
[CLI overview](/cli/overview). As there, the API key and API URL are never read
from `.zespan.yaml`, so a committed config file cannot redirect where your key
is sent.
## `zespan policy init`
```bash theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
zespan policy init
```
Writes `policies/pii-egress.yaml` with a starter policy in `dryrun` mode. It
refuses rather than overwriting an existing file.
## `zespan policy pull`
```bash theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
zespan policy pull --env prod
```
Generates one policy file per guardrail that already exists, so adopting an
existing project does not mean transcribing it by hand. By default it covers
guardrails created in the dashboard; `--all` also re-emits ones already managed
in code, which is useful if a file was lost.
Each generated file pins its rule to the existing guardrail with a `slug:`, so
the plan straight afterwards reports no changes to make — only pending
adoptions. Files you have already edited are skipped unless you pass `--force`.
Generation is not lossless and prints a warning for every case it could not
express exactly; see [what `pull` cannot express](/policies/as-code#what-pull-cannot-express).
## `zespan policy validate`
```bash theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
zespan policy validate
```
Reads every `.yaml`/`.yml` file under `policies/` and validates all of them,
reporting every problem at once with file, line and a message that says what to
write instead. Exits `0` when everything is valid, `1` otherwise.
As a pre-commit hook:
```bash .git/hooks/pre-commit theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
#!/bin/sh
npx @zespan/cli policy validate || exit 1
```
## `zespan policy test`
```bash theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
zespan policy test --against issues
```
Runs your policies over the project's own recorded traffic and reports what they
would have caught versus what they would have broken, per rule, with the actual
conversation behind each false positive. See
[Policy testing](/policies/testing).
Promoting a policy to `deny` requires a test result or `--untested`.
## `zespan policy plan`
```bash theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
zespan policy plan --env prod
```
Prints what an apply would change, and changes nothing:
```text theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
+ pii-egress (pii-egress--0) Block PII in model output
~ phi-egress (phi-egress--0) PHI egress control
- stale-policy (stale-policy--0) Retired control
Plan: 1 to add, 1 to change, 1 to remove.
```
The plan is computed on the server against live state, so what the CLI shows and
what an apply would do cannot disagree. Re-running `plan` against unchanged
state prints `No changes.` — reformatting a file, reordering its keys, or adding
comments is not a change, because the comparison runs over a canonical form.
## `zespan policy apply`
```bash theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
zespan policy apply --env prod
```
`apply` computes a plan first and sends its hash with the apply, so you cannot
apply a plan you never saw. It refuses in the cases below, each naming the flag
that resolves it:
```text theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
Apply failed (409): 1 policy/policies do not list environment "prod" in appliesTo.environments: staging-only.
```
`appliesTo.environments` is a guard, not a router — it never sends a policy
somewhere you did not name on the command line, it only refuses to write it
where the policy says it does not belong. Either point `--env` at an
environment the policy lists, or add this one to the file. Every offending
policy is named, not just the first. Checked **before** every other refusal
below, so a misdirected apply is reported before anything else — see
[scoping to environments](/policies/as-code#scoping-a-policy-to-environments).
```text theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
Apply failed (409): The state changed since this plan was computed.
The state changed since this plan was computed. Re-run `zespan policy plan` and review the new diff.
```
Someone else applied, or changed a guardrail in the dashboard, between your
plan and your apply. Re-plan and look at the new diff. `--force` overrides.
```text theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
Apply failed (409): 1 policy/policies conflict with changes made in the UI.
Those policies were detached in the UI. Re-run with --force to take ownership back in code.
```
Someone took ownership of that policy from the dashboard — often during an
incident. Applying over it would undo their change. `--force` takes
ownership back into code.
```text theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
Apply failed (409): 1 policy/policies would enforce at "deny" without a backtest: hipaa-phi-egress.
Run `zespan policy test` to see what they would have blocked, or re-run with --untested.
```
Run `zespan policy test --against issues` and look at the false positives.
`--untested` exists because a platform engineer mid-incident has to be able
to ship — see [the deny gate](/policies/testing#the-deny-gate).
```text theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
Apply failed (409): This apply would take over 2 guardrail(s) created in the dashboard: block-ssn, pii-out.
Re-run with --adopt to take those dashboard-authored guardrails over in code.
```
Normal straight after `zespan policy pull`. Adopting makes those guardrails
read-only for whoever built them, so it is never implicit. The row is
updated in place and keeps its execution history.
```text theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
Apply failed (409): This apply would remove 1 policy/policies: phi-egress.
Removing a policy removes a control.
Re-run with --allow-remove if that is intended.
```
A policy that used to be in your file set is gone. Removing a guardrail
removes a control, so it is never implicit. `--allow-remove` makes it
deliberate.
Nothing is written when any of these fires.
## Every apply names a person
Because these commands run as you rather than as a project key, an apply is
attributed to a real identity. The `policy.applied` audit entry records the
**user**, their **organization**, the **session** the request came from, the
**device** — the machine registered by [`zespan auth login`](/cli/auth#machines-are-remembered) —
plus the IP address and user agent the server observed. The recorded apply itself
names the acting user alongside each file's SHA-256 content hash.
An apply made from the dashboard records no device, because there is no CLI
machine behind it. That is an honest blank rather than a guess.
A control-plane mutation the server cannot attribute to a real user is
**refused with `401`**, not recorded against a placeholder. There is no
unattributed apply to find in the audit log, because one can no longer be made.
See [Audit log](/platform/audit).
## In CI
```yaml .github/workflows/policies.yml theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
name: Policies
on: [pull_request, push]
jobs:
validate:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
# Offline: no credential, no network call.
- run: npx @zespan/cli policy validate
```
**`plan` and `apply` cannot run unattended today.** They need a user session,
and the only way to obtain one is `zespan auth login`, which requires a human to
approve in a browser. There is no machine credential for the control plane — an
API key is refused, by design.
If you previously ran `zespan policy plan` in CI with `ZESPAN_API_KEY`, it was
returning `403` rather than planning anything. Run `plan` and `apply` from a
developer machine that is signed in, and keep `validate` — which needs no
credential — as the CI gate.
Applying needs `policy:apply`, which is granted to **owner** and **admin** only —
`editor` deliberately does not have it, matching how guardrails are already
treated as a high-risk resource. A project you cannot reach is reported by name:
```text theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
You cannot reach project "Intake Prod" (2845c0ef-…) in "Acme Health" (acme-health).
Either your account is not a member of the organization that owns it, or your role there cannot do this.
Run `zespan projects list` to see what you can reach, and `zespan link` to choose a different project.
```
## Next steps
The same plan/apply pipeline and the same refusals, from the dashboard — a second entrypoint, not a second implementation.
The authoring loop and the ownership model.
Every field and the supported YAML subset.
Sign in — `pull`, `test`, `plan` and `apply` all need it.
Link a project so you never pass `--project`.
Configuration precedence and install options.
Diagnose SDK setup problems.
# Evidence Packs — audit-ready compliance documents
Source: https://docs.zespan.com/compliance/evidence-packs
Generate hash-addressed, re-verifiable audit documents — a per-agent Compliance Card or a SOC 2 control evidence report — from your existing Zespan data.
An evidence pack is a generated document that cites the Zespan records backing either a single agent's operational history or a compliance framework's controls, for a stated period. Every fact in the document links back to the record it came from — a guardrail config, an evaluation run, an approval request, an audit log entry — so an auditor (or you) can click through to the source, or later re-check that the source still exists and the document hasn't been altered. See [Verification](/compliance/verification) for how that re-check works.
Every pack is:
* **Scoped** to a project and a `periodStart`/`periodEnd` window (400 days maximum)
* **Hash-addressed** — content-addressed by its own SHA-256, computed over the exact bytes stored, not over a re-serialized copy
* **Immutable once generated** — nothing about a completed pack is edited in place; generate a new one for a new period
* **Never a compliance certification** — see the honesty constraint below
## The honesty constraint
Every generated document — Compliance Card or SOC 2 control evidence, in every format — carries this exact disclaimer, printed at both the top and the bottom of the document:
> "This document reports controls and evidence observed by Zespan for the stated scope and period. It is not a certification of compliance and does not constitute legal advice."
This isn't boilerplate — it's the feature's design premise. A section with no matching records for a period renders the literal text "No evidence available for this period." — never a suppressed section, never a silently-passing control. See [Frameworks and controls](/compliance/frameworks#coverage) for how an uncovered control is surfaced before you even generate a document.
## The two document kinds
Everything Zespan recorded about **one agent** (or all agents, if you leave the agent unset) for the period: profile, models used, guardrails in force, guardrail change history and outcomes, evaluation results with sample sizes, human approvals, known limitations, and change history.
The same underlying facts, reorganized and namespaced under the three SOC 2 controls Zespan maps today — CC6.1, CC7.2, CC8.1. See [Frameworks and controls](/compliance/frameworks) for what each control draws on.
## Generating a pack
From the project sidebar, go to **Monitor → Compliance**. Requires the Pro plan.
**Compliance Card** for one agent (or all agents), or **SOC 2 control evidence** for the framework mapping. Control evidence requires picking a framework — SOC 2 is the only one available today; see [Not yet available](/compliance/frameworks#not-yet-available).
Pick `periodStart` and `periodEnd`. The form defaults to the last full calendar month. Maximum period length is 400 days.
**HTML** (default) or **JSON** — see [Formats](#formats) below.
Click **Generate evidence pack**. This returns immediately with the pack in `pending` status — generation runs on a background worker, and the row updates to `processing` then `completed` (or `failed`) as it runs. The Compliance page polls automatically while any pack is in flight.
Once `completed`, use **Download** to get the document, or **Verify** to re-check it — see [Verification](/compliance/verification).
Before you generate, the **Coverage for this period** panel shows which controls have evidence for your chosen period and framework — so you find a gap before you generate (and possibly hand off) a document, not after. It runs the same evidence-collection logic the document itself would use, without rendering or storing anything.
## Formats
| Format | What it is | Use it for |
| ------ | ------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------- |
| `html` | A single self-contained HTML file — inline CSS, no external requests, a real `@media print` stylesheet | Printable to PDF from your browser; human review |
| `json` | The structured evidence document — every section, record, and source citation as data | Machine ingestion, archival, feeding into your own tooling |
PDF is not yet an available format. There's no headless-browser rendering path in Zespan today, and adding one is genuine new infrastructure — a browser runtime, a memory cap, a timeout, a fallback — rather than a config change, so a request for `format: "pdf"` is rejected at the API rather than silently downgraded to HTML. The HTML document is designed to print cleanly straight from your browser's own print-to-PDF, which is the interim path.
## Downloading
`GET /v1/projects/:id/evidence-packs/:packId?download=true` returns a presigned download URL when packs are stored in object storage (Cloudflare R2), or the raw document content directly when running on local-disk storage (typically self-hosted or local dev). The dashboard's **Download** button handles either case for you.
## Permissions
| Role | List / download / verify / coverage (`compliance:read`) | Generate (`compliance:generate`) |
| ----------------------- | ------------------------------------------------------- | -------------------------------- |
| Owner, Admin | Yes | Yes |
| Editor, Viewer, Billing | Yes | No |
A role without `compliance:generate` can still list, download, and verify existing packs — it just can't start a new generation.
## Next steps
* [Frameworks and controls](/compliance/frameworks) — the SOC 2 CC6.1/CC7.2/CC8.1 mapping and what evidence each control draws on
* [Verification](/compliance/verification) — what `/verify` re-checks, and what it can't
* [Compliance evidence limitations](/reference/compliance-limitations) — the permanent data gaps this feature reports about itself
* [Audit log](/platform/audit) — the record source behind access-control and change-history evidence
# Frameworks and controls
Source: https://docs.zespan.com/compliance/frameworks
The SOC 2 control mapping evidence packs draw on today — CC6.1, CC7.2, CC8.1 — what each control's evidence comes from, and what's not yet available.
A "SOC 2 control evidence" pack organizes Zespan's recorded facts under a framework's named controls, instead of under the agent-centric sections a Compliance Card uses. One framework is registered today.
## SOC 2
Selecting **SOC 2 control evidence** as your document type maps evidence to three Trust Services Criteria controls:
| Control | Title | Evidence sources |
| --------- | ----------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **CC6.1** | Logical access controls | Standing organization role assignments, plus every recorded membership or role change during the period |
| **CC7.2** | Monitoring of system components and anomaly detection | Configured alert rules and incidents detected during the period, with time-to-resolve where a resolution was recorded |
| **CC8.1** | Change management | Recorded changes to prompts, guardrail policies, and agent lifecycle state, plus externally-reported pipeline deploys and human approval requests raised for privileged actions |
Each control's narrative is explicit about scope. CC6.1, for example, states plainly that the section "reports what Zespan recorded; it does not assess whether those assignments were appropriate." A control document reports observed facts — never a judgment about whether those facts satisfy the control.
## Coverage
Before you generate, the Compliance page's **Coverage for this period** panel runs the same evidence queries each control would use and shows, per control, whether at least one of its sections produced records for the period you've chosen — without rendering or storing a document. A control with zero records across every one of its sections shows as uncovered, along with which sections came back empty, so a gap in the underlying data is visible before you generate — and possibly hand to an auditor — the document itself.
This mirrors what the generated document itself does at render time: a section with no records for the period prints the literal text "No evidence available for this period." rather than being silently omitted.
## `reviewedOn` / `reviewedBy`
Every SOC 2 control evidence document — in every format — carries a review-attribution line:
> Reviewed: 2026-08-08 by Zespan engineering — not reviewed by a licensed auditor
This is deliberately printed on the document itself rather than buried in a settings page. The SOC 2 mapping shipped in this release was authored and reviewed internally, by Zespan engineering — it has **not** been reviewed by a licensed external auditor. Treat a generated control-evidence document as a well-structured starting point for your own SOC 2 evidence collection, not as a substitute for your auditor's own judgment about what satisfies your specific control environment.
## Not yet available
**EU AI Act** and **ISO/IEC 42001** framework mappings are not available yet. Building either responsibly means qualified review before it ships — the same bar the SOC 2 mapping was held to — so this release ships one well-grounded mapping rather than several rushed ones. Adding a framework is additive to the current design (one new mapping file plus one registry entry), so it won't require a redesign when it happens.
## Next steps
* [Evidence packs](/compliance/evidence-packs) — the two document kinds and how to generate one
* [Verification](/compliance/verification) — re-checking a generated document's citations
* [Compliance evidence limitations](/reference/compliance-limitations) — permanent data gaps stated in the documents themselves
# Verification
Source: https://docs.zespan.com/compliance/verification
What GET /v1/projects/:id/evidence-packs/:packId/verify re-checks against a generated evidence pack — and, importantly, what it can't check at all.
A generated document is only useful to an auditor if it can be checked, not just trusted. `/verify` re-runs the checks a generated pack's own citations imply against your live data, and reports the result honestly — including admitting when it couldn't check something.
## What it checks
Click **Verify** next to any completed pack on the Compliance page, or call `GET /v1/projects/:id/evidence-packs/:packId/verify` directly. Two independent checks run:
1. **Content integrity** — the stored document is read back and re-hashed with SHA-256. If the recomputed hash doesn't match the hash recorded at generation time, `sha256Matches` is `false`: the stored bytes have changed since the document was generated (or the object couldn't be read at all).
2. **Citation survival** — every source record the document cited is re-resolved against your live data, scoped to the pack's project (or organization, for org-level sources like membership). A citation whose record is gone is reported as a **divergence**.
## Result fields
| Field | Meaning |
| ----------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ |
| `sha256Matches` | `true` when the recomputed hash of the stored document equals the hash recorded at generation time |
| `storedSha256` / `recomputedSha256` | The two hashes being compared |
| `evidenceStillPresent` | `true` only when zero divergences were found among **checked** citations — see `unverifiableRefs` below for what "checked" deliberately excludes |
| `checkedRefs` | Count of citations that were actually re-resolved against a live record |
| `unverifiableRefs` | Count of citations that structurally cannot be re-checked at record level |
| `divergences` | One entry per citation whose record is gone, each naming the source, id, label, and whether it's `missing` or `wrong_project` |
| `objectMissing` | `true` when the stored document itself couldn't be read back |
## `unverifiableRefs` — read this before you trust a clean result
Not every citation in a document points at a single database row that can be looked up again later. Two kinds of citation are structurally unverifiable:
* **ClickHouse aggregates.** A "models used" citation, for example, doesn't name one row — it names a synthetic key over an aggregate (a model, summed across a whole period). There's no single row to re-fetch, and re-running the aggregate would just compare a number the document already printed against itself, proving nothing about whether the underlying data changed.
* **Certain change-timeline events.** Prompt-deploy, agent-lifecycle, and externally-reported change events carry composite ids from their own source systems rather than an audit-log row. Audit-sourced change events *are* checkable — they resolve as ordinary audit-log citations for free.
These citations are **excluded from `evidenceStillPresent` entirely** — not counted as verified, not counted as failed. They're reported separately, in `unverifiableRefs`, specifically so a document with a hundred citations, ninety of them structurally uncheckable, doesn't come back reading like "the whole document checks out" just because the ten checkable ones passed. You — or your auditor — need to see that ninety citations were never re-checked at all, not just that the ten that could be were fine.
`evidenceStillPresent: true` means every **checkable** citation still resolves. It does not mean every citation in the document was checked. Always read `unverifiableRefs` alongside it: a pack reporting `checkedRefs: 3` and `unverifiableRefs: 340` passed a much smaller check than one reporting `checkedRefs: 340` and `unverifiableRefs: 3` — even though both can legitimately report `evidenceStillPresent: true`.
## `missing` vs. `wrong_project`
When a cited record doesn't resolve inside the pack's own project (or org), `/verify` does one more lookup — unscoped, across every tenant — before deciding how to report the divergence:
* **`missing`** — the record doesn't exist anywhere. It was deleted.
* **`wrong_project`** — the record still exists, just not under this project anymore.
Reporting these as two distinct reasons means "the guardrail was deleted" and "the guardrail moved to another project" don't look like the same failure.
## Failure modes that don't error
`/verify` never throws for a data condition — a deleted record, a deleted storage object, or a pack that never finished generating are all *results*, not exceptions:
* A pack whose status isn't `completed` (or that has no stored object or hash yet) returns a result with everything `false`, `0`, or empty, rather than a partial or misleading pass.
* A resolver failure — a transient database error while re-checking one source — is not treated as proof those records are gone. Those citations move into `unverifiableRefs` for that check instead of being reported as false divergences.
## Next steps
* [Evidence packs](/compliance/evidence-packs) — generating the document `/verify` checks
* [Frameworks and controls](/compliance/frameworks) — the SOC 2 mapping behind control evidence documents
* [Compliance evidence limitations](/reference/compliance-limitations) — permanent gaps in what Zespan can report, distinct from what `/verify` can re-check
# Core concepts: the Zespan data model
Source: https://docs.zespan.com/concepts
Learn how Zespan structures agent telemetry data — events, spans, span kinds, traces, projects, and organizations — and how cost is calculated from token counts.
Before diving into the dashboard, it helps to understand how Zespan structures the data it collects from your agents. Everything starts with a single operation — an LLM call, a tool invocation, an agent turn — and builds upward through a clear hierarchy.
## The data hierarchy
```
Organization
└── Project (one API key per project)
└── Trace (one end-to-end agent run)
└── Span (one step: LLM call, tool call, agent turn…)
```
Every piece of data in Zespan belongs to a project, and every project belongs to an organization. Within a project, individual operations are captured as spans, and related spans are grouped into traces. A trace represents one complete agent run from start to finish.
***
## Event
An event is the most fundamental unit of data in Zespan. Every time your instrumented code executes an operation — an LLM call, a tool invocation, an agent turn — the SDK captures a single event and sends it to the ingest endpoint. Events are immutable once stored.
`event_id` — unique UUIDv4 for this event
`trace_id` — links this event to others in the same agent run
`span_id` — unique ID for this specific operation
`parent_span_id` — set when this step is nested inside another span
`provider` — `"openai"`, `"anthropic"`, `"google"`, or `"custom"`
`model` — the exact model string, e.g. `"gpt-4o"`
`span_kind` — what type of operation this is (see below)
`operation` — `"chat"`, `"embed"`, `"tool"`, `"agent"`, or `"custom"`
`latency_ms` — total time from start to finish
`ttft_ms` — time to first token (streaming LLM calls only)
`input_tokens` — tokens consumed in the prompt
`output_tokens` — tokens generated in the completion
`cached_tokens` — prompt tokens served from provider cache
`cost_usd` — computed by the SDK from per-model token pricing
`status` — `"success"`, `"error"`, `"timeout"`, `"rate_limited"`, or `"cancelled"`
`error_code` — provider error code when status is not `"success"`
`error_message` — human-readable error from the provider
Events also carry context fields for filtering: `environment`, `user_id`, `session_id`, and a `tags` map of arbitrary string key-value pairs.
Prompt and completion text is **stored by default** with PII redaction applied before transmission. Set `storePrompts: false` in `zespan.init()` to disable prompt storage entirely.
***
## Span
A span is an event that carries trace context. Every event is technically a span — the term emphasizes that the event participates in a parent-child relationship with other events in the same agent run.
Two fields link spans together:
* **`span_id`** — a unique identifier for this specific operation
* **`parent_span_id`** — the `span_id` of the operation that triggered this one
When you look at a trace in the dashboard, the flame graph renders the span tree: each bar represents one span, its width is proportional to its duration, and indentation shows nesting depth.
In multi-agent systems where one agent delegates to another, or a retriever is called inside an LLM prompt construction step, the SDK automatically propagates trace context. Spans link correctly without any manual effort.
***
## Span kinds
Every span has a `span_kind` field that describes the type of operation it represents. This controls how the span is rendered in the flame graph and how it appears in the agent registry.
| Span kind | Emitted by | What it represents |
| ----------- | ----------------------------------------------- | ---------------------------------------- |
| `llm` | Provider wrappers | A direct call to an LLM API |
| `agent` | `withAgent()`, ADK/LangChain integrations | The execution scope of a single agent |
| `tool` | `agent.traceTool()`, LangChain/ADK integrations | A tool or function call by an agent |
| `planning` | `agent.logPlan()` | Steps the agent planned before executing |
| `handoff` | `agent.delegateTo()`, multi-agent frameworks | An agent delegating to another agent |
| `retriever` | Manual spans, LangChain retriever handler | A document retrieval operation (RAG) |
| `guardrail` | SDK guardrail client | A pre- or post-LLM content check |
| `general` | Manual `startSpan()` | Any other custom operation |
The SDK sets `span_kind` automatically for all wrapper-generated spans. For manual spans:
```typescript theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
const { span } = startSpan({
name: "vector-search",
span_kind: "retriever",
provider: "custom",
});
```
See the full [span kinds reference](/reference/span-kinds) for flame graph rendering details.
***
## Trace
A trace is a group of spans sharing the same `trace_id`. It represents one complete end-to-end agent run — from the first operation to the last, across all agent turns, tool calls, and LLM interactions.
For a simple chatbot making one API call per message, a trace contains one span. For a multi-agent workflow that routes, retrieves context, delegates to specialists, and synthesizes a response, a trace may contain dozens of nested spans across multiple agents.
In the dashboard, the **Traces** view shows the flame graph for a full trace:
* The x-axis is wall-clock time from start to end of the root span
* Each bar is a span coloured by status (green = success, red = error, yellow = timeout or rate limited, gray = cancelled)
* Hovering shows span details; clicking opens the full span detail panel
* The total trace cost is the sum of `cost_usd` across all spans
A trace is not an object you create explicitly. It emerges automatically from spans that share a `trace_id`. The SDK generates a new `trace_id` for each top-level agent run and propagates it to all nested operations through context propagation.
***
## Project
A project is an isolated container for agent events. Every project has exactly one API key and all events ingested with that key are scoped to that project. Projects are the primary unit of data isolation.
Use separate projects for:
* **Different applications** — one project per service or agent system
* **Different environments** — or use the `environment` field to keep production and staging in one project with easy filtering
Each project has its own retention period, dashboard metrics, alert rules, guardrails, prompt library, and AI analysis results.
Treat your project API key like a password — never commit it to source control. Rotate it from **Settings → API Keys** if compromised. The old key stays valid for 24 hours to allow smooth rollover.
***
## Organization
An organization is the top-level workspace. It holds your projects, team members, and billing subscription. All usage — events ingested, data retained, members invited — counts against the organization's plan.
| Role | Access |
| ---------- | ------------------------------------------------------------------ |
| **Owner** | Full access including billing, org deletion, and all admin actions |
| **Admin** | Manage projects, API keys, alerts, AI features; read billing |
| **Member** | Read-only dashboard access |
Billing is per organization, not per project or user. Your plan's event quota is shared across all projects and resets monthly.
***
## Cost calculation
Zespan computes `cost_usd` client-side in the SDK before events are sent. Costs appear immediately in the dashboard without any server-side enrichment step.
```
cost_usd = (input_tokens × input_price
+ output_tokens × output_price
+ cached_tokens × cached_price) ÷ 1,000,000
```
Cached tokens — prompt tokens served from the provider's prompt cache — are billed at a reduced rate. The SDK extracts cached token counts from provider responses automatically.
If the SDK encounters a model it does not recognise, `cost_usd` is set to `0` rather than throwing an error. See the [supported models reference](/reference/models) for the full pricing table.
See your cached token savings on the **Costs** page under the Cache Hit Ratio card. Increasing your cache hit rate — by keeping static system prompts at the start of your context — is often the easiest way to reduce agent operating costs.
# Agent Registry
Source: https://docs.zespan.com/dashboard/agent-registry
A live map of every agent in your system, the tools they use, and how they connect to each other.
The Agent Registry builds a topology of your AI system automatically from your traces. Every agent that has run at least one trace appears here with the tools it called, the models it used, and any sub-agents it delegated to.
## What the registry shows
Each agent entry shows:
| Field | Details |
| ------------ | ------------------------------------------------------- |
| Agent name | The identifier set in `withAgent()` or `wrapADKAgent()` |
| Last seen | Timestamp of the most recent trace for this agent |
| Total traces | Count of all recorded runs |
| Tools used | List of tool names called by this agent |
| Models | All model identifiers this agent has used |
| Sub-agents | Agents this agent has delegated to |
## Agent topology
The topology view renders agents as nodes connected by delegation edges. An edge from Agent A to Agent B means A called B as a sub-agent in at least one trace. Edge thickness scales with delegation frequency.
Use the topology to identify:
* Coordinator agents that delegate to many specialists
* Agents with unexpectedly high tool diversity (potential scope creep)
* Isolated agents with no connections (standalone tools)
## Tool usage
Each agent's tool panel lists every tool the agent called, with call counts and failure rates. A tool with a failure rate above 10% is highlighted in the reliability column.
## Filtering
Filter agents by:
* **Last active** — see which agents are actively running vs dormant
* **Model** — find all agents using a specific model
* **Environment** — separate production from staging agents
## Registering agents in code
Agents appear in the registry automatically when you use the SDK's agent context:
```typescript theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
import { zespan, withAgent } from "@zespan/sdk";
await withAgent({ name: "research-agent", model: "gpt-4o" }, async () => {
// all LLM calls and tool invocations inside are attributed to this agent
const result = await openai.chat.completions.create({ ... });
});
```
For Google ADK agents, use `wrapADKAgent()` or `instrumentADK()` — see [Google ADK integration](/sdk/integrations/google-adk).
## Compliance history
Both the registry list and an individual agent's detail page show a compliance badge — pass rate, violation count, and last violation date — pulled from **every project in your organization**, not just the current one. It's the same summary that appears next to a [delegation line in the trace detail view](/dashboard/traces#delegation-compliance).
This answers a different question than the per-agent performance metrics elsewhere in the registry: not "did this agent's calls succeed," but "has this agent — wherever it's actually run across my org — stayed inside its guardrails."
Compliance data rolls up on a 6-hour cycle. A badge reading "No compliance history yet" means this agent name has no recorded guardrail checks anywhere in your org yet — it isn't a failing score.
## Next steps
* [Agents](/dashboard/agents) — per-agent performance dashboard
* [Traces](/dashboard/traces) — drill into individual agent runs
* [Guardrails](/dashboard/guardrails) — the policies this compliance history is measured against
# Agents
Source: https://docs.zespan.com/dashboard/agents
Per-agent performance dashboards — cost, latency, error rate, and tool usage broken down by agent.
The Agents view gives each agent in your registry its own performance dashboard. Where the Traces view shows individual runs, the Agents view aggregates metrics across all runs to answer questions like "which agent costs the most per run?" or "which agent has the worst error rate this week?"
## Agent list
The agent list shows all agents with at least one trace in the current date range, sorted by total cost descending by default.
| Column | Details |
| ----------- | ---------------------------------------- |
| Agent | Agent name from the SDK |
| Runs | Total trace count in the selected period |
| Avg cost | Average USD cost per run |
| Avg latency | Average wall-clock duration per run |
| Error rate | Percentage of runs that ended in error |
| Last run | Timestamp of the most recent run |
Click any agent to open its detail view.
## Agent detail
The agent detail view shows metrics for one agent across the selected time range.
### Cost over time
A time-series chart of total daily cost for this agent. Spikes indicate either increased usage volume or a specific run with abnormally high token consumption — click any spike to see the traces that contributed to it.
### Latency percentiles
P50, P95, and P99 latency for each day. Increasing P99 with stable P50 indicates occasional outlier runs — usually caused by retries or tool failures.
### Error breakdown
Errors grouped by type (model error, tool error, timeout, rate limit). The breakdown helps you prioritize: rate limit errors are solved by quota increases or backoff logic; tool errors need debugging in the tool implementation.
### Tool usage
A ranked list of tools this agent called, with total calls, failure count, and average latency per tool. Tools with high failure rates or high latency are the most common causes of poor agent performance.
### Model usage
All model identifiers used by this agent in the selected period, with token counts and cost attributed per model.
## Comparing agents
Return to the agent list and use the checkboxes to select 2–4 agents for side-by-side comparison. The comparison view shows cost, latency, and error rate on shared axes for the selected period.
## Next steps
* [Agent Registry](/dashboard/agent-registry) — topology view of all agents
* [Costs](/dashboard/costs) — cost attribution across all dimensions
# Anomaly Detection — catch metric deviations before they become incidents
Source: https://docs.zespan.com/dashboard/ai-features
Run on-demand statistical scans over your LLM traffic to detect latency spikes, error rate surges, and cost drifts — with plain-English explanations and automatic incident correlation.
Anomaly Detection scans your recent LLM events for statistical deviations and explains what it found in plain English. Select a time range, click **Detect Anomalies**, and the engine groups events by model, runs Z-score analysis against your baseline, and flags anything that falls outside normal range.
Anomaly Detection requires the **Team** plan or higher.
Looking for other AI features? [ZespanPilot](/dashboard/zespanpilot) handles natural language queries and project actions. [Cost Optimizer](/dashboard/costs#cost-optimizer) surfaces model-switching recommendations on the Costs page. Root Cause Analysis runs automatically on [Incidents](/dashboard/incidents).
## Running a scan
1. Open **AI Features** from the left sidebar.
2. Select a time range: **Last 1 hour**, **Last 24 hours**, or **Last 7 days**.
3. Click **Detect Anomalies**.
The engine groups events by model before running detection so a spike in one model doesn't mask healthy behavior in others.
## What gets detected
| Anomaly type | Trigger condition |
| --------------- | ----------------------------------------------------- |
| `LATENCY_SPIKE` | Z-score > 2.5 (high) or > 4.0 (critical) vs baseline |
| `ERROR_SPIKE` | Error rate > 10% with at least 3 errors in the window |
| `COST_SPIKE` | Per-request cost increased > 1.5× from baseline |
## What anomaly cards show
Each detected anomaly produces a card with:
* **Type** — `LATENCY_SPIKE`, `ERROR_SPIKE`, or `COST_SPIKE`
* **Severity** — `critical`, `high`, `medium`, or `low`
* **Description** — the specific metric values that triggered the detection
* **Affected models** — which models are showing the deviation
* **AI recommendation** — a plain-English suggested next step
## Incident correlation
Anomalies at `medium` severity and above are automatically correlated with your [Incidents](/dashboard/incidents) feed. `high` and `critical` anomalies also fire your configured [alert rules](/dashboard/alerts).
You can also manually open an incident from any anomaly card using the **Open incident** button.
If the scan returns no anomalies, your metrics are within normal range for the selected window. This is the expected result most of the time.
# Alerts
Source: https://docs.zespan.com/dashboard/alerts
Create threshold rules on error rate, cost, latency, and eval scores. Get notified via email, Slack, PagerDuty, OpsGenie, Discord, Jira, Freshservice, or webhook.
Alerts let you define thresholds on agent metrics and get notified the moment one is crossed. Rules are evaluated continuously against your live data.
Alerts require the **Pro** plan or higher.
## Creating an alert rule
Go to **Alerts** and click **Create Alert**.
Give it a descriptive name — e.g. `High error rate — support agent` or `Cost spike — prod`.
| Metric | What it monitors |
| ---------------- | -------------------------------------------------------------- |
| `error_rate` | Fraction of operations that fail, timeout, or are rate-limited |
| `latency` | Average response time in milliseconds |
| `cost` | Total spend in USD for the window |
| `requests` | Total operation count |
| Evaluator metric | Any custom eval score key (e.g. `quality.score`) |
To monitor a custom evaluator score, select your evaluator from the **Evaluator** dropdown and enter the metric key.
Choose an operator (`>`, `<`, `>=`, `<=`) and the threshold value.
Example: `error_rate > 0.05` fires when more than 5% of operations fail.
The window controls how far back data is aggregated before comparing to your threshold.
| Window | Best for |
| ------ | ----------------------------------------- |
| 5 min | Sudden spikes — agent errors, cost bursts |
| 15 min | Sustained error rate increases |
| 30 min | Cost trend monitoring |
| 60 min | Latency degradation patterns |
Add one or more channels. You can mix channel types on a single rule.
**Email** — enter one or more addresses (comma-separated). Receives an HTML alert with metric, current value, threshold, and a direct link to the dashboard.
**Slack** — paste your Slack incoming webhook URL. Alert arrives as a colour-coded message (red = critical, orange = warning, blue = info).
**PagerDuty** — paste your Events API v2 integration key. Alert triggers an incident with severity mapped to the configured level.
**OpsGenie** — paste your OpsGenie API key. Creates an alert with priority P1/P2/P3.
**Discord** — paste your Discord webhook URL. Alert arrives as a colour-coded embed.
**Jira** — enter your Jira base URL, email, API token, project key, and issue type. Creates a Jira issue when the rule fires.
**Freshservice** — enter your Freshservice domain and API key. Opens a ticket with priority mapped to severity.
**Webhook** — paste any HTTPS URL. Zespan POSTs a signed JSON payload. See [Webhooks](/guides/webhooks) for the schema and signature verification.
Use **Test channel** after adding a channel to verify delivery before the rule goes live.
Click **Create Alert** to activate. The rule is evaluated in the next cycle.
## Enabling and disabling rules
Each rule has a toggle on the rule card. Disabled rules are not evaluated and will not fire. Use this to pause a noisy rule without deleting it.
## Alert history
The history table shows every evaluation — both triggered and not triggered — with timestamp, rule name, metric value, and threshold. Click any row to see the full evaluation details.
## Webhook payload
When a rule fires and a webhook channel is configured, Zespan POSTs:
```json theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
{
"projectId": "proj_xyz789",
"alertName": "High error rate — support agent",
"metric": "error_rate",
"threshold": 0.05,
"currentValue": 0.12,
"severity": "critical",
"message": "error_rate is 0.12, threshold is > 0.05",
"triggeredAt": "2026-04-20T14:32:00Z"
}
```
The request includes an `X-Zespan-Signature-256` header for HMAC-SHA256 verification. See [Webhooks](/guides/webhooks) for verification code.
## Model lifecycle alerts
[Model Lifecycle](/dashboard/model-lifecycle) uses this same alert delivery machinery — not a separate notification path — for a distinct rule type, `model-lifecycle`, that fires when the daily deprecation scan raises or re-raises a finding rather than on a metric threshold you configure.
A model-lifecycle finding is **always** recorded and visible in the dashboard (the Overview widget, the Models page Lifecycle column, and the Model lifecycle findings page) regardless of alert configuration. But email and webhook **notification** only happens if the project has at least one **enabled** alert rule of type `model-lifecycle`. With none configured, the delivery worker finds no matching rule and silently skips sending — the finding is still there, it just never reaches an inbox or a channel. If deprecation alerts seem to not be firing, this is the first thing to check.
`AlertRule.type` for this rule kind is not an option in the **Create Alert** flow described above, which is built around metric/threshold rules, and a model-lifecycle rule doesn't appear in this page's alert rules table either — both assume every rule has a metric, condition, and threshold, none of which a lifecycle rule has. Instead, it has its own small, dedicated opt-in: a settings card at the top of the [Model Lifecycle](/dashboard/model-lifecycle#getting-notified) page itself (toggle, notification emails, optional webhook). Enabling it there creates or updates the same `AlertRule` row this section describes, through a separate `GET`/`PUT /v1/projects/{id}/model-lifecycle/alert-rule` pair (see the [API reference](/api-reference/introduction)) rather than this page's alert endpoints.
## Next steps
* [Model Lifecycle](/dashboard/model-lifecycle) — what a finding contains and the 90/30/7/0 re-raise ladder that governs when this alert type fires again
* [Incidents](/dashboard/incidents) — where a breaching alert can escalate into a tracked incident
* [Webhooks](/guides/webhooks) — full payload schema and signature verification for the webhook channel
# Annotation Queues
Source: https://docs.zespan.com/dashboard/annotation-queues
Route a filtered set of traces to a human reviewer for manual pass/fail or score annotation, stored alongside automated judge scores.
## What it is
An **Annotation Queue** is a filtered set of traces routed to a human for manual review and scoring — not an automated LLM-as-judge run. Use one when you need:
* **Ground truth for evaluator calibration** — a human-scored sample to check whether an LLM-judge evaluator's verdicts actually agree with a person's judgment.
* **Human review of ambiguous cases** — traces where an automated judge score doesn't get you sufficient confidence and you want a person to look directly at the input/output before it counts as pass or fail.
Annotation Queues require the **Team** plan or above — the same tier as Datasets and Simulations.
## Where to find it
Go to **Annotations** in the left sidebar (in the **Develop** group, alongside Prompts, Evaluations, Guardrails, and Datasets).
## Creating a queue
Opens the queue-creation dialog.
Give the queue a name (unique within the project) and an optional description.
Pick a **From** and **To** date/time. This is the window of already-ingested traces the queue will be populated from.
Click **Create**. The queue starts empty — click **Populate** on its card to actually pull matching traces into it.
A queue's filter criteria also support narrowing by operation, model, session, and verdict level (pass/fail/warning) via the API, beyond the time-range fields the dashboard's creation dialog currently exposes.
### Populating is a snapshot, not a live feed
Clicking **Populate** runs a bounded scan (up to 500 traces) against everything matching the queue's stored filter criteria at that moment, and adds any traces not already in the queue. It is **not** a live subscription — traces ingested after you populate don't appear automatically. Click **Populate** again later to top the queue up with newly-matching traces; already-added traces are never duplicated.
## Working a queue
Click **Work queue →** on a queue's card to open its review view: an item list on the left, a review panel on the right.
### The item list
Traces are grouped into three tabs — **Pending**, **Annotated**, **Skipped** — so you always know what's left to review and can revisit anything already decided.
### The review panel
Selecting an item shows:
* A **lens** badge — **Response (final output)** for a whole-trace review, or **Step (one call)** when the item is scoped to one specific span within the trace
* The trace's **input** and **output**
* **Retrieved context**, if the trace has a retrieval step, with its chunk count
### Submitting an annotation
For a pending item, you can:
* **Skip** — no score, verdict, or label required. Always available.
* **Submit annotation** — record your review. You can supply:
* A **verdict** — explicit **Pass** or **Fail** buttons
* A **label** (optional free text, e.g. `good`, `bad`, `needs_fix`)
* A **score** (optional, 0–1)
* **Notes** (optional free text)
A categorical (label-only) annotation **always requires an explicit Pass/Fail verdict** — there is no way to submit one without picking one. A label like `bad` or `needs_fix` is arbitrary free text with no inherent pass/fail meaning to Zespan, so it can never stand in for a verdict on its own. The **Submit annotation** button stays disabled until you've either entered a numeric score or explicitly clicked Pass or Fail. (A numeric score alone is sufficient — it implies its own verdict at the ≥0.5 threshold — but an explicit verdict you set alongside a score always overrides that default.)
## How human annotations are stored
Submitting an annotation writes one row into the same `evaluation_scores` data automated evaluators write to, tagged `human_annotated`. This is a deliberate design choice — not a separate "annotations" table — so a human annotation shows up everywhere an automated score already does: the evaluator's trend chart, the [Cost-Quality Frontier](/dashboard/costs#cost-quality-frontier), and per-trace score views. There's no separate dashboard or query path to check for human-reviewed results; they're already there, distinguished from judge-model scores only by the `human_annotated` tag and the reviewing user's identity.
A numeric annotation is stored as a numeric score; a label-only annotation is stored as a categorical score carrying your label and the verdict you explicitly chose.
## Next steps
* [Evaluations](/dashboard/evaluations#retroactive-evaluation) — run an automated evaluator against the same historical traces instead of reviewing them by hand
* [Evaluations — reading results](/dashboard/evaluations#reading-evaluation-results) — how categorical/boolean scores (including human annotations) render across the dashboard
# Blast Radius — what breaks if you change this
Source: https://docs.zespan.com/dashboard/blast-radius
A dependency graph across prompts, agents, models, guardrails, evaluators, and alerts, backing an impact check shown before you release a prompt and a delete-blocking check on evaluators.
Blast Radius answers "what actually depends on this?" for a single resource — a prompt, an agent, a guardrail policy, an evaluator, a dataset, a model, or an alert — by walking a dependency graph built from both your project's configuration and its real production traffic.
It isn't a standalone dashboard page today. It surfaces in two places where the answer changes what you'd do next:
* **Releasing a prompt** — the confirmation dialog for promoting a version to `production` shows an impact summary before you commit. Advisory only; it never blocks the release.
* **Deleting an evaluator** — the delete confirmation shows the same summary plus a full dependents table, and it *does* block: if the evaluator has any dependents (or the check fails to load), you have to explicitly acknowledge before **Delete** becomes clickable.
A full graph visualization and a standalone Blast Radius page are on the roadmap but not built yet. Today the graph is only consumed through the two integration points above, and directly through the API described below.
## Declared vs. observed edges
Every dependency in the graph is one of two kinds, and the UI keeps them visually distinct rather than collapsing them into one number:
| Origin | What it means | Source |
| ---------- | -------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------ |
| `declared` | Configured — someone wired this dependency in your project, whether or not it's ever actually used | PostgreSQL (prompt embeddings, guardrail agent filters, evaluator→alert links) |
| `observed` | Actually called — real production traffic passed through this dependency in the trace window | ClickHouse trace data |
The distinction matters in both directions: a prompt configured as a dependency of six agents but only ever called by one is a much smaller real blast radius than the declared graph alone suggests. Conversely, a dependency nobody explicitly configured but that shows real call volume is exactly the kind of undocumented coupling this feature exists to surface.
A dependent can be reached through more than one edge (for example, a prompt embedded in two different parent prompts, both called by the same agent). When that happens, `hasObservedTraffic`, `totalCallVolume30d`, and `totalCostUsd30d` on that dependent are summed across **every** inbound edge feeding it from within the blast radius — not just the one edge that happened to be discovered first. A dependent reached only through a `declared` edge can still show real observed traffic, if another edge into it (from elsewhere in the same blast radius) is `observed`.
## Dependency direction
Every edge is `{ from, to }`, and **`to` depends on `from`** — read it as "changing `from` affects `to`":
| From | To | What it means |
| ---------------- | ------------- | ---------------------------------------------------------------------------------------------------------- |
| Child prompt | Parent prompt | The parent embeds the child's content by reference; editing the child changes the parent's rendered output |
| Guardrail policy | Agent | The policy is scoped to that agent via its agent filter |
| Model | Agent | The agent actually called that model (observed) |
| Prompt | Agent | The agent actually called that prompt (observed) |
| Evaluator | Alert | The alert rule watches that evaluator |
So "what breaks if I change this prompt?" is answered by walking dependents *from* the prompt — which surfaces every agent that calls it, directly or transitively.
## Node ids
A node id is `:` — for example `prompt:support-reply` or `evaluator:3f9c...`. Keys are **names** for prompts, agents, and models, and **UUIDs** for row-backed kinds (evaluators, guardrail policies, datasets, alerts). Because a key can itself contain a colon or a slash (a prompt legitimately named `checkout/v2` produces the id `prompt:checkout/v2`), the id always travels as the `node` query parameter — never a URL path segment.
## Depth, node, and time-window caps
A blast radius is a breadth-first walk from the root node, bounded on three axes so a hub resource with hundreds of dependents (or a genuine dependency cycle — agent-to-agent delegation produces real ones) can't turn one request into an unbounded scan:
| Parameter | Default | Max | What it bounds |
| ------------ | ------- | ---- | -------------------------------------------------------------- |
| `maxDepth` | 3 | 5 | How many hops from the root to traverse |
| `maxNodes` | 500 | 2000 | Total dependents returned |
| `windowDays` | 30 | 90 | Lookback window for observed (ClickHouse) call volume and cost |
`truncated: true` in the response means the walk hit one of these caps **and** the graph genuinely continues beyond it — not merely that the graph happened to end exactly at the cap. If depth 3 reaches every dependent and stops naturally, `truncated` is `false` even though depth 3 was the limit checked.
## Response shape
```json theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
{
"root": { "id": "prompt:support-reply", "kind": "prompt", "name": "support-reply", "href": "prompts/support-reply", "environments": ["prod"] },
"dependents": [
{
"node": { "id": "agent:CustomerSupportAgent", "kind": "agent", "name": "CustomerSupportAgent", "href": "agents/CustomerSupportAgent", "environments": ["prod", "staging"] },
"depth": 1,
"via": [{ "from": "prompt:support-reply", "to": "agent:CustomerSupportAgent", "origin": "observed", "callVolume": 386, "costUsd": 41.2 }],
"totalCallVolume30d": 386,
"totalCostUsd30d": 41.2,
"hasObservedTraffic": true
}
],
"impact": {
"agents": 1,
"environments": ["prod", "staging"],
"productionAffected": true,
"callVolume30d": 386,
"cost30dUsd": 41.2,
"slosAtRisk": [],
"gates": []
},
"truncated": false,
"computedAt": "2026-08-07T09:54:00.000Z"
}
```
`dependents[].totalCallVolume30d` and `totalCostUsd30d` sum to `impact.callVolume30d` / `impact.cost30dUsd` across the whole response — the per-node numbers and the headline number are the same aggregation at two grains, not two separately-computed figures that can drift apart.
`computedAt` marks the moment this response's graph was built. Every request computes fresh — there's no caching in front of this endpoint today — so `computedAt` is effectively "now," modulo the small delay before a trace becomes queryable. It's still shown on every card and table because a number with no timestamp reads as unexplainably stale the first time it lags even slightly behind a dashboard refresh.
`impact.slosAtRisk` and `impact.gates` are always empty arrays today. They're reserved fields for an SLO model that hasn't shipped yet — present so the response shape won't need to change later, not because either is currently populated. Don't build against them expecting real data.
## Where it's not (yet)
* **No standalone Blast Radius page.** The graph is only reachable through the two integration points above and the API directly.
* **No graph visualization.** `GET /projects/:id/graph` returns the full raw graph (nodes + edges) for a project, sized for a future visualization — nothing in the dashboard renders it yet.
* **Tool and HTTP-target nodes aren't in the graph.** Only prompts, agents, models, guardrail policies, evaluators, datasets, and alerts are covered. MCP tool calls and HTTP targets are candidates for a later pass.
## API
Both endpoints require a dashboard session (browser cookie) with the `dashboard:read` permission — not an `x-api-key` request. See the **Blast Radius** group in the [API Reference](/api-reference/introduction) for full parameter and response documentation.
```bash Get the blast radius for one node theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
curl "$ZESPAN_API_URL/projects/$PROJECT_ID/blast-radius?node=prompt%3Asupport-reply" \
-H "Cookie: "
```
A malformed `node` (missing the `kind:key` separator) returns `400`. A well-formed id for a resource that doesn't exist in the project — or that belongs to a different project — returns `404`; the two cases (typo vs. wrong project) are deliberately indistinguishable in the response, so a cross-project id can't be used to probe for a resource's existence.
## Next steps
* [Prompts](/dashboard/prompts#blast-radius-before-releasing) — the release-confirmation impact card
* [Evaluations](/dashboard/evaluations#deleting-an-evaluator) — the delete-blocking check
* [Changes](/dashboard/changes) — the project-wide "what changed" timeline this feature complements: Changes tells you what happened, Blast Radius tells you what else would be affected before it happens
# Changes — a project-wide timeline of what changed and when
Source: https://docs.zespan.com/dashboard/changes
A single feed of every prompt deploy, agent promotion, policy edit, and pipeline deploy in your project, so 'what changed?' is a page you visit instead of a question you Slack someone.
The Changes page answers the question every investigation starts with: what changed, and when? It merges prompt deploys, agent lifecycle transitions, guardrail/evaluator/alert edits, and deploys you report from your own CI pipeline into one chronological feed — no more cross-referencing the audit log, the prompts page, and a deploy Slack channel by hand.
From the project sidebar, go to **Monitor → Changes**.
Use the range selector (**Last 24 Hours**, **Last 7 Days**, **Last 30 Days**) to set the window. The feed is sorted newest-first.
Click any kind chip to narrow the feed to just that kind — click again to remove it. Chips are additive: select two kinds to see both.
Wire your CI pipeline to `POST /v1/projects/:id/changes` so pipeline deploys show up on the same timeline as everything else. See [Reporting changes from CI](#reporting-changes-from-ci) below.
## The eight change kinds
Every event on the timeline has one of eight kinds:
| Kind | What it captures | Source |
| ------------------ | ------------------------------------------------------------------------------------------------ | ------------------------------ |
| `prompt_deploy` | A prompt version created, edited, relabeled, rolled back, or deleted | Prompt deployments + audit log |
| `agent_lifecycle` | An agent promoted, demoted, or otherwise transitioned state | Agent lifecycle history |
| `policy_change` | A guardrail created, updated, enabled, disabled, or deleted | Audit log |
| `evaluator_change` | An evaluation created for a project | Audit log |
| `alert_change` | An alert rule created, updated, or deleted | Audit log |
| `config_change` | Dataset, project settings, retention, API key rotation, or a ZespanPilot auto-remediation action | Audit log |
| `incident_change` | An incident created, its status changed, or resolved | Audit log |
| `external` | A change reported from your own pipeline via the API | Changes API |
Each row also carries a **severity** (`info`, `notable`, or `high`), shown as a colored left border, and — where the underlying source captured one — a **View diff** popover showing the before/after values.
## Filters
| Filter | Where | Notes |
| -------------------------- | -------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Time range (`from` / `to`) | Range selector | Defaults to the last 7 days |
| Kind (`kinds`) | Kind chips | Comma-separated in the API; an unrecognized kind returns `400` |
| Agent (`agent`) | API only | Restricts results to events whose scope includes the named agent — `Incident` records have no agent field, so no UI surface (including the incident panel) sets this yet; call the API directly to use it |
If any underlying source doesn't respond in time, a banner marks the result as **partial** rather than silently showing an incomplete list as complete. The banner names which source(s) were slow.
## `ChangeEvent` fields
| Field | Type | Description |
| ------------ | ------------------- | ------------------------------------------------------------------------------- |
| `id` | string | Stable and unique across sources (`:`) |
| `kind` | string | One of the eight kinds above |
| `occurredAt` | string (ISO-8601) | When the change happened |
| `actor` | object \| null | `{ type: "user" \| "system" \| "api", id, name }` — who or what made the change |
| `title` | string | One-line summary shown as the row's title |
| `summary` | string \| null | Optional longer description |
| `href` | string | Relative deep link into the dashboard (e.g. `prompts/checkout-system`) |
| `scope` | object | `{ agents?, prompts?, models? }` — what the change affected |
| `diff` | object \| undefined | `{ before, after }` when the source captured a structured diff |
| `severity` | string | `info`, `notable`, or `high` |
## Changes around an incident
The incident detail page has its own "Changes around this incident" panel — the same feed, pivoted on an incident's start time instead of a date range, split into **Before** and **After** columns. See [Changes around this incident](/dashboard/incidents#changes-around-this-incident) on the Incidents page for details.
## Reporting changes from CI
Anything your own deploy pipeline does — a Kubernetes rollout, a feature-flag flip, an infra change — is invisible to Zespan unless you tell it. Report it with `POST /v1/projects/:id/changes` and it shows up on the timeline (and in the incident panel's before/after split) as an `external` event, right alongside prompt deploys and policy edits.
```bash Report a deploy from CI theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
curl -X POST "$ZESPAN_API_URL/v1/projects/$PROJECT_ID/changes" \
-H "x-api-key: $ZESPAN_API_KEY" \
-H "content-type: application/json" \
-d '{"title":"Deployed api@'"$GIT_SHA"'","source":"github","severity":"notable"}'
```
`$ZESPAN_API_URL` defaults to `https://api.zespan.com` — only set it if you're self-hosting. `$ZESPAN_API_KEY` is a project-scoped API key (see [API Keys](/account/api-keys)). An API key has no role attached — it's already scoped to exactly one project — so any valid key for the target project works; there's no separate permission check for API-key callers on this route.
### Request body
| Field | Type | Required | Notes |
| ------------ | ----------------------------------- | -------- | ---------------------------------------------------------------------------------------------------------------------------- |
| `title` | string | Yes | Max 200 characters |
| `summary` | string | No | Max 2000 characters |
| `href` | string (URL) | No | A link to attach to the event, e.g. your CI run |
| `occurredAt` | string (ISO-8601) | No | Defaults to now |
| `severity` | `"info"` \| `"notable"` \| `"high"` | No | Defaults to `"info"` |
| `source` | string | No | Max 50 characters, defaults to `"api"` — use it to identify the pipeline (`"github"`, `"circleci"`, `"terraform"`, ...) |
| `scope` | object | No | `{ agents?, prompts?, models? }` — scope the change to specific agents/prompts/models so it surfaces in agent-filtered views |
A successful call returns `201` with `{ id, title, occurredAt }`.
## Next steps
* [Incidents](/dashboard/incidents) — the before/after change panel on incident detail
* [Alerts](/dashboard/alerts) — notification rules that share the `alerts:manage` permission with reporting changes
* [Audit Log](/platform/audit) — the full, unfiltered event history that most change kinds are drawn from
# Costs — understand and control your LLM spend
Source: https://docs.zespan.com/dashboard/costs
Break down your LLM spend by model and over time, track cache savings, see your month-to-date total, and get a 30-day forecast to avoid billing surprises.
The Costs page gives you a complete picture of where your LLM budget is going. You can see which models are the most expensive, how your daily spend is trending, how much you're saving through token caching, and where your current month is heading. Use this data to make informed decisions about model selection and sampling strategies before your next billing cycle arrives.
Full cost attribution and forecasting require the **Pro** plan or higher. Basic cost totals are visible on all plans.
## Cost by model
The bar chart at the top of the page ranks every model you've used in the selected period by total spend. Each bar shows the model name and its USD cost for the period. Hover over a bar to see the underlying numbers: number of calls, total tokens, average cost per call, and cache hit rate.
This chart answers the most common cost question immediately: "which model is responsible for most of my bill?" If one model towers above the others, that's your highest-leverage target for optimization.
## Cost over time
The line chart below shows your daily spend across the selected date range. Each point represents total cost for that calendar day across all models. Use this chart to spot:
* Sudden spikes that coincide with a deployment or feature launch
* Gradual cost growth that may indicate increasing usage or a model change
* Days with unexpectedly low cost that may point to an outage or misconfiguration
You can change the date range using the selector above the chart. Shorter ranges (7 days) show finer detail; longer ranges (90 days) reveal trends.
## Cache hit ratio
The cache hit ratio card shows what percentage of your input tokens were served from the model provider's prompt cache rather than recomputed from scratch. A higher cache hit ratio means lower cost and lower latency for those requests.
The card displays:
* **Cache hit ratio** — percentage of total input tokens that were cached
* **Cached tokens** — the raw count of tokens served from cache
* **Estimated savings** — the USD amount saved by not recomputing those tokens
To increase your cache hit ratio, structure your prompts so that the static system prompt comes first and only the dynamic user content changes per request. The Zespan SDK tracks `cached_tokens` automatically when your provider reports them.
## Month-to-date spend
The gauge in the upper-right corner shows your cumulative spend for the current calendar month. The gauge fills from zero toward the outer ring, which represents your budgeted monthly limit (if you've set one). The exact dollar amount is shown in the center.
## 30-day forecast
Below the gauge, a forecast card projects your total spend for the next 30 days based on a linear extrapolation of your recent daily averages. This is a straight-line estimate — it does not account for planned changes in traffic — but it gives you an early warning if your current trajectory will exceed your budget.
The forecast uses the last 14 days of data to calculate the daily average. If your usage pattern is highly variable or you recently made a significant change (such as switching models), treat the forecast as a directional signal rather than a precise prediction.
## Cost by user
The cost by user table requires the Team plan or higher.
If your SDK passes a `userId` when creating spans, the cost by user table breaks down spend per user for the selected period. The table shows each user ID, their total cost, number of requests, and average cost per request, sorted by total spend descending.
This view is useful for understanding which users or user segments are the most expensive to serve, and for detecting unusual individual usage that may indicate a bug or abuse.
The cost by user table only populates for requests where your SDK explicitly sets a user ID. If you haven't configured this, see the SDK documentation for how to attach user context to your traces.
## Acting on cost data
The Costs page is most useful when it informs action. Here are two common levers:
If your costs are higher than expected and your application can tolerate missing some traces, reduce the `sampleRate` in your SDK configuration. A `sampleRate` of `0.5` sends half of all events to Zespan, cutting your event quota usage and any associated overage charges in half. Set it in your SDK initialization:
```typescript theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
import { zespan } from "@zespan/sdk";
zespan.init({
apiKey: process.env.ZESPAN_API_KEY,
sampleRate: 0.5, // trace 50% of requests
});
```
If the cost by model chart shows that one expensive model handles a large share of your requests, consider whether a smaller model could handle some of those workloads. Use the [Cost Optimizer](#cost-optimizer) at the bottom of this page to analyze a specific trace and get a model-switching recommendation with a confidence score.
***
## Retroactive Evaluations
Score historical traces against an evaluator you didn't have configured when the traffic actually happened — you don't need to have predicted which quality metric you'd want in advance.
Scroll to the **Retroactive Evaluations** panel on the Costs page.
Choose any existing evaluator — built-in or custom.
Pick the historical window to score, and optionally narrow it to a specific operation or model.
Zespan resolves every matching historical trace and scores it with the same LLM-judge pipeline live auto-evaluation uses. Results are tagged as retroactive so you can tell them apart from live scores.
## Cost-Quality Frontier
Once you have quality scores — live or retroactive — alongside your existing cost data, the Cost-Quality Frontier chart plots every model actually used on a given operation by real cost vs. real quality, joined at the individual LLM-span level rather than a coarser trace-level average, so a multi-model trace doesn't get misattributed to the wrong model.
Each point is one (model, operation) pair. Points with quality data are colored by score; grey points haven't been scored yet — run a [retroactive evaluation](#retroactive-evaluations) to fill them in.
If two models cluster within a couple of quality points of each other on the same operation, that's a real "switch and save \$X per call" opportunity — the frontier chart surfaces it directly instead of you cross-referencing two separate dashboards to find it yourself.
## Cost Optimizer
The Cost Optimizer analyzes a specific trace and tells you whether a cheaper model could handle the same task — and by how much your cost would drop.
**Plan availability:** Pro and above
### How to use it
Enter a trace ID in the Cost Optimizer panel (scroll to the bottom of this page) and click **Analyze**. The engine evaluates the trace's task complexity against model capability and returns a recommendation.
Task complexity is scored on a 1–10 scale:
* **1–3** — simple classification, extraction, short Q\&A
* **4–6** — summarization, structured output generation
* **7–10** — code generation, multi-step reasoning, tool use
A downgrade is only suggested when complexity is 5 or below, and never when the trace contains tool calls or extended reasoning tokens.
### What you get back
| Field | Description |
| ----------------- | --------------------------------------------------------- |
| Current model | The model used in the analyzed trace |
| Suggested model | The recommended cheaper alternative |
| Potential savings | Estimated cost reduction (0–95%) |
| Confidence | `high`, `medium`, or `low` |
| Complexity score | 1–10 score for the task |
| Reasoning | One-sentence explanation of why the switch is appropriate |
Start with your most expensive traces from the **Top sessions by cost** table above — paste those trace IDs into the Cost Optimizer to find the highest-value switching opportunities.
## Next steps
* [Value](/dashboard/value) — this page tells you what you spent; Value tells you what that spend actually produced, joining reported business outcomes to the same per-trace cost data
* [Cost-Quality Frontier](#cost-quality-frontier) — the quality-adjusted view of cost, using evaluator scores instead of reported business outcomes
* [Models](/dashboard/models) — the per-model table behind this page's cost-by-model chart, with latency and error-rate columns alongside cost
* [Model Lifecycle](/dashboard/model-lifecycle) — if a model driving your spend also has a provider-announced end-of-life, this is where that gets flagged
# Datasets
Source: https://docs.zespan.com/dashboard/datasets
Manage evaluation datasets — create from traces, upload CSV, and score your own pipeline's runs against them.
Datasets are collections of input/output pairs used to evaluate your agents systematically. You can build datasets from real traces, upload them as CSV, or populate them manually. Once created, your own code runs against the dataset's items and links the results back as a run, which you then score with an evaluator from the dashboard — see [How dataset runs work](#how-dataset-runs-work) below.
## Creating a dataset
### From traces
The fastest way to build a dataset is from existing traces:
1. Open **Traces** and filter to the runs you want to evaluate
2. Select one or more trace rows using the checkboxes
3. Click **Add to dataset** → choose an existing dataset or create a new one
The trace's input (prompt or agent instruction) and output (completion or agent response) are added as a row.
### By uploading CSV
Upload a CSV file with columns matching the dataset schema. Required columns:
| Column | Description |
| ---------- | ------------------------------------------------------------ |
| `input` | The prompt or instruction sent to the agent |
| `output` | (Optional) The agent's response to evaluate |
| `expected` | (Optional) The ground truth answer for comparison evaluators |
Only `input` is required — a row with no `output` or `expected` value is still added to the dataset.
Go to **Datasets** → **New dataset** → **Upload CSV** and select your file.
### Manually
Add rows one at a time using the **Add row** button. Useful for small curated datasets of known edge cases.
Once a dataset has items, Zespan doesn't run anything against it directly — your own code links a run's results back to the dataset, then you score that run from the dashboard. See [How dataset runs work](#how-dataset-runs-work) and [Scoring a run](#scoring-a-run) below.
## Dataset versioning
Each dataset has a version history. When you add or remove rows, the previous version is preserved. Evaluation runs are tied to a specific dataset version so results remain reproducible.
## How dataset runs work
A dataset run lets you evaluate a pipeline that lives entirely in your own code — a production service, a batch job, a scheduled script, anything that can call the Zespan SDK — instead of asking Zespan to execute it for you.
Unlike [Simulations](/dashboard/simulations), which Zespan runs on your behalf, dataset runs let you bring your own pipeline: your code fetches the dataset's items, calls your own LLM or agent with each one (producing a Zespan trace the same way your integration always does), and links that trace back to a named run. Zespan never executes anything here — it only records the link between a dataset item and the trace your code produced, then scores the linked traces with an evaluator you choose and lets you compare two runs side by side.
### Running against an HTTP endpoint
Two execution modes are the exception to "Zespan never executes anything here": running a candidate [prompt version](/dashboard/prompts#the-quality-gate), and running a registered **HTTP Target**. Both let Zespan produce the run itself, without your own pipeline code in the loop, from the same "Run over dataset" flow — pick which one to use with the **Prompt version** / **HTTP endpoint** toggle.
An HTTP Target is a registered, externally-hosted agent endpoint — a deployed Bedrock or Glean agent, or any chatbot API you don't control or can't instrument with the Zespan SDK. Instead of calling an LLM provider directly (as the prompt-version mode does), Zespan POSTs each dataset item's hydrated request straight to the endpoint you registered and records the raw response as a trace tagged `sdk_name: "zespan-http-endpoint"`. See [HTTP Targets](/dashboard/http-targets) for how to register one, its security model, and how `traceparent` propagation works.
## Linking a run from your code
A typical job does three things:
1. Fetch the dataset's items with the SDK — each item includes its `input` and, if the dataset has one, its `expectedOutput`.
2. Create the run. Calling this again with the same name later (for example, the next time the job starts) just re-attaches to the existing run instead of creating a duplicate.
3. For each item, call your own pipeline as normal, then link the item to the run using the trace ID your call just produced. Pass an `observationId` as well if you want to point at one span within the trace rather than the trace as a whole.
```typescript TypeScript theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
import { zespan, withZespanTrace } from "@zespan/sdk";
import { randomUUID } from "node:crypto";
zespan.init({ apiKey: process.env.ZESPAN_API_KEY! });
const client = zespan.getClient();
// 1. Fetch the dataset's items
const items = await client.datasets.getItems("support-eval-set");
// 2. Create (or re-attach to) a named run — safe to call every time your job starts
const run = await client.datasets.createRun("support-eval-set", "gpt-4o-v2");
// 3. Run your pipeline per item under a known trace ID, then link it to the run
for (const item of items) {
const traceId = randomUUID();
// Any wrapped LLM/agent call inside this callback is tagged with traceId
await withZespanTrace(() => mySupportAgent(item.input), { traceId });
await run.link(item.id, traceId);
}
```
```python Python theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
import zespan
from zespan import get_client, with_zespan_context
zespan.init(api_key="zsp_your_api_key_here")
client = get_client()
# 1. Fetch the dataset's items
items = client.datasets.get_items("support-eval-set")
# 2. Create (or re-attach to) a named run — safe to call every time your job starts
run = client.datasets.create_run("support-eval-set", "gpt-4o-v2")
# 3. Run your pipeline per item under a known trace ID, then link it to the run
for item in items:
with with_zespan_context() as ctx:
my_support_agent(item["input"]) # your own LLM/agent call, traced as usual
trace_id = ctx["trace_id"]
run.link(item["id"], trace_id)
```
For the full SDK walkthrough — run handles, `observationId`, and wiring this into a quality gate — see [Dataset runs in the SDK](/sdk/dataset-runs).
## Scoring a run
Once your job has linked every item, score the run from the dashboard:
Open the dataset and click the **Runs** tab. Find the run your job created — it's created the first time your code calls `createRun`/`create_run`.
Choose which evaluator to score the run's linked traces with.
Zespan looks up each linked trace and scores it with the evaluator you chose. A per-item score appears next to each dataset item, along with the run's overall average once scoring completes.
Scoring a run calls the evaluator's LLM judge, which requires a project **LLM connection**. Without one, scoring fails with "No LLM connection configured — add one in Settings → LLM Connections." Connect a provider key under [LLM Connections](/platform/llm-connections) first.
## Comparing two runs
To see whether a change to your pipeline — a new prompt version, a new model, a new retrieval step — actually did better on this dataset, compare two runs directly:
Open the dataset and click the **Runs** tab.
Select the two runs you want to compare — for example, your previous run and your new candidate run.
Zespan shows a side-by-side table of every dataset item both runs cover, with each run's linked trace and score next to each other, plus each run's overall average. Regressions (10+ point drops) are listed first, improvements (5+ point gains) after.
Use **Export CSV** or **Export HTML** at the top of the comparison view to save the same regressions-first breakdown as a file — CSV for spreadsheet analysis, or a self-contained HTML file you can open directly in a browser or attach to a PR/Slack message without needing to log in to Zespan.
## Regression testing from production failures
Every recurring production failure — clustered as an [Issue](/dashboard/issues) — automatically becomes a regression test case once it's happened 3 or more times. There's no persona-writing involved: the test case is the incident that already happened to real traffic.
### How it works
A background worker captures recurring issues into a **"Production Failures (auto-captured)"** dataset once they've recurred 3 or more times.
Point your CI's existing dataset-run flow at that dataset — the exact same [linking mechanism](#how-dataset-runs-work) described above. This is bring-your-own-execution, same as every other dataset run: Zespan never executes your agent.
A second worker computes the real verdict on the replayed trace and compares it to the original failure's verdict. The case counts as **resolved** only if the replay is now genuinely `healthy` — not merely "didn't error."
### Wiring it into a quality gate
Pass a `regressionRunId` to the prompt quality gate request to require a minimum resolution rate before a prompt or policy change is allowed to ship. It becomes a normal fourth pass/fail signal alongside your existing gate checks.
This sidesteps hand-authoring persona-driven test scenarios entirely: your own incident history already is the test suite. It works because Zespan already has a deterministic verdict system and a bring-your-own-execution architecture — the same primitives every other dataset run on this page relies on.
## Next steps
* [Issues](/dashboard/issues) — where recurring failures get clustered before they become regression tests
* [Evaluations](/dashboard/evaluations) — run and review evaluation results
* [Simulations](/dashboard/simulations) — test prompt changes against a dataset before deploying
# Environments — filter traces, evaluations, and guardrails by deployment stage
Source: https://docs.zespan.com/dashboard/environments
Every project ships with dev, staging, and prod. Add your own, switch between them from the header, and filter any view down to just one.
An **environment** is a deployment stage — `dev`, `staging`, `prod`, or whatever your pipeline actually looks like — that scopes traces, evaluations, guardrails, incidents, and metrics so you can look at one stage of your pipeline without the others in the way. It's a first-class row in the project, not a free-text tag: it has a slug, a display name, a rank (for ordering), a production flag, and an optional monthly event quota field (see [Monthly quota](#monthly-quota-not-yet-enforced) below).
Every project page has an environment switcher next to the project name. It defaults to **All environments** — pick one to scope the current page's data to just that environment.
Go to **Settings → Environments** to create, rename, reorder, or delete environments, and to mark one as production.
Every endpoint the switcher drives also accepts an `environment` query parameter directly — see [Filtering the API](#filtering-the-api) below.
## The three defaults, and custom ones
Every project is seeded with three environments on creation:
| Slug | Display name | Rank | Production |
| --------- | ------------ | ---- | ---------- |
| `dev` | Development | 0 | No |
| `staging` | Staging | 10 | No |
| `prod` | Production | 20 | Yes |
These three are a starting set, not an enum. Slugs are free-form beyond them — `qa`, `uat`, `prod-eu`, `canary` are all valid, as long as the slug is lowercase alphanumeric with hyphens (max 50 characters) and unique within the project. Create one from **Settings → Environments → New environment**, or `POST /v1/projects/:id/environments`.
Creating, renaming, reordering, or deleting an environment requires the `environments:manage` permission (owner/admin/editor); viewing the list only needs `environments:read`.
### Deleting an environment
An environment can carry guardrail configs, alert rules, SLA policies, health thresholds, agent lifecycle gates, prompt deployments, and incidents. Deleting it does **not** silently orphan those — the delete is blocked with a `409` naming exactly what's still attached (e.g. `"3 guardrail config(s), 1 alert rule(s)"`), so you know what to move or delete first before the environment itself can go.
## The switcher and the "omitted means all" rule
The environment switcher writes its selection into the URL as `?env=` — so a filtered link is shareable and survives a page reload. Clearing the selection (**All environments**) removes the param entirely rather than writing a default back into the URL.
This matters for the API too: **omitting the `environment` parameter means all environments**, on every endpoint that accepts it. This is deliberate — every existing integration and saved link that predates this feature keeps working unfiltered, exactly as before. You only see a single environment's data when you (or the switcher) explicitly ask for one.
## Filtering the API
The `environment` query parameter is accepted on 16 read endpoints across traces, evaluations, guardrails, incidents, and metrics — the same set of pages the switcher covers. Pass a slug:
```bash theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
curl "$ZESPAN_API_URL/v1/projects/$PROJECT_ID/traces?environment=staging" \
-H "x-api-key: $ZESPAN_API_KEY"
```
An unrecognized slug returns `400`:
```json theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
{ "message": "Unknown environment \"qa-2\"" }
```
This is the opposite of what happens at ingest time — see [Alias mapping and the ingest asymmetry](#alias-mapping-and-the-ingest-asymmetry) below for why reads reject what ingest would silently accept.
## Alias mapping and the ingest asymmetry
The `environment` column in the underlying event store predates this feature — it's been populated by the SDK's free-text `environment` option (or OTel's `deployment.environment` attribute) since long before environments were a project-level entity. That means historical data uses whatever string a client happened to send, most commonly `production`, `development`, `staging`, and `test`, not the seeded `prod`/`staging`/`dev` slugs.
To keep old and new data queryable together, a shared alias map normalizes the common cases both ways:
| Free-text value | Resolves to slug |
| --------------- | --------------------------- |
| `production` | `prod` |
| `development` | `dev` |
| `staging` | `staging` (already matches) |
Filtering `?environment=prod` matches rows stored as either `prod` or `production` — you never have to know or care which era a given row came from.
**Ingest and reads handle an unrecognized value differently, on purpose:**
* **Reads reject.** `?environment=qa-2` on a project with no `qa-2` environment returns `400`. A read is a request for a specific view; failing loudly beats silently returning nothing or (worse) everything.
* **Ingest keeps and flags.** A trace arriving with `environment: "load-test-3"` that doesn't match any known slug or alias is still stored — under `load-test-3`, unmodified — with a warning logged (throttled to once per project per hour, so a typo'd environment string on every span doesn't flood the logs). Losing telemetry because of an unfamiliar environment string would be a far worse outcome than an imprecise dimension on an otherwise-good trace.
If you see an unexpected environment value showing up, it usually means a client is sending something that isn't yet one of the project's environments — create it (**Settings → Environments → New environment**) with a matching slug and future events will resolve against it directly, no alias needed.
## Monthly quota (not yet enforced)
Each environment has an optional `monthlyEventQuota` field, settable from the same create/edit form and returned by the API. **Nothing currently enforces it** — it does not throttle ingest or trigger an alert on its own. The field exists so environment-level quota enforcement, when it ships, doesn't require a second schema change. Don't rely on it as a working limit today; project-level quotas (see [Billing](/account/billing)) are the ones actually enforced.
## Next steps
* [Traces](/dashboard/traces) — the primary view the environment switcher scopes
* [Guardrails](/dashboard/guardrails) and [Alerts](/dashboard/alerts) — can be scoped to a specific environment via the same `environmentId` relation the delete-block check above walks
* [Environments guide](/guides/environments) — when to use environment filtering within one project versus separate projects entirely
# Errors
Source: https://docs.zespan.com/dashboard/errors
Track, group, and triage errors across your agents — model errors, tool failures, timeouts, and rate limits.
The Errors view aggregates failures from all your agent traces into a structured error log. Instead of hunting through individual traces, you see errors grouped by type and agent so you can triage at scale.
## Error types
Zespan classifies errors into four categories:
| Type | Description |
| --------------- | ----------------------------------------------------------------------------------- |
| **Model error** | The LLM returned an error response (e.g. content policy violation, invalid request) |
| **Tool error** | A tool call threw an exception or returned an error status |
| **Timeout** | A span exceeded its deadline — either an LLM call or a tool invocation |
| **Rate limit** | A 429 response from a model provider |
## Error list
The error list shows all errors in the selected time range, with:
* Error message (truncated to 200 characters)
* Error type
* Agent that produced the error
* Model (for model errors)
* Tool name (for tool errors)
* Timestamp and count (how many times this exact error occurred)
Errors with identical messages and sources are grouped automatically. A group showing 47 occurrences means the same error happened 47 times — not 47 unique bugs.
## Filtering
Filter by error type, agent, model, environment, or date range. The **Only new** toggle shows only errors that first appeared in the selected period — useful for spotting regressions after a deployment.
## Opening the source trace
Click any error row to open the trace that produced it. The flame graph highlights the failed span so you can see the full context — what the agent was doing when the error occurred, what inputs led to it, and what happened immediately before.
## Error rate metrics
The error rate chart at the top of the page shows errors per hour over the selected period. A flat baseline with sudden spikes indicates an external event (provider outage, a bad deployment). A gradually increasing baseline indicates a growing problem.
## Next steps
* [Incidents](/dashboard/incidents) — AI-detected anomalies, including error surges
* [Alerts](/dashboard/alerts) — set up notifications when error rates exceed a threshold
* [Traces](/dashboard/traces) — investigate individual failed runs
# Evaluations — measure and trend custom LLM metrics
Source: https://docs.zespan.com/dashboard/evaluations
Define custom evaluation metrics, run them against your traces, and track results over time to detect quality regressions and measure the impact of prompt changes.
Evaluations let you define custom quality metrics for your LLM outputs and measure them against real production traces.
Instead of guessing whether a prompt change improved quality, you define what "good" means numerically, run an evaluation, and see scores trend over time. Zespan uses an LLM-as-judge approach: your evaluator definition is a prompt that an AI model applies to each trace, producing a numeric score.
Auto-evaluations with 12 built-in LLM-as-judge templates are available on **Solo** and above. Manual evaluation runs are available on all plans including Free.
## How evaluations work
An **evaluator** (`EvaluatorDefinition`) is a named metric — at minimum a name, a metric key, and an optional description. When an evaluator runs (manually, on auto-run, or from a dataset-run score) against a trace, Zespan submits the trace's prompt/completion text to an LLM judge, which returns a numeric score. Scores are stored and displayed as trends on the Evaluations page.
There are two ways to get an evaluator with a real rubric:
* **New Evaluator dialog** — the fast path. It only collects Name, Metric Key, and an optional Description; it does not let you write a scoring prompt. An evaluator created this way scores traces using Zespan's generic default judge prompt ("evaluate output quality against input intent, score 0.0–1.0") rather than a rubric you author.
* **Templates → Deploy** — the path for a custom rubric, score range, and input scope. See [Custom evaluator templates](#custom-evaluator-templates) below.
You can also attach scores programmatically from the SDK using `span.setEvalScore()`, bypassing the judge model entirely. Both sources — programmatic and LLM-as-judge — appear in the same dashboard.
## Creating an evaluator
Navigate to **Evaluations** in the left sidebar.
The evaluator creation dialog opens.
Give it a name (e.g. `Factuality Check`) and a metric key (e.g. `custom_relevance`). The metric key becomes the identifier in trend charts. Optionally add a description.
Click **Save**. The evaluator appears in your evaluator list, ready to run with the default judge prompt described above.
If you need a custom scoring prompt, a specific score range, categorical/boolean verdicts, or an input scope (prompt vs. completion vs. both), don't use the New Evaluator dialog — create a [custom template](#custom-evaluator-templates) instead and deploy it. Deploying a template creates the same kind of evaluator, just with your rubric attached instead of the generic default.
Evaluations work on stored prompt and completion text. If you have disabled prompt storage with `storePrompts: false`, evaluation results will be empty — re-enable it to use evaluations.
Creating an evaluator via the New Evaluator dialog requires the **Team** plan or above. Deploying a template into an evaluator does not have this restriction — see [Custom evaluator templates](#custom-evaluator-templates). Editing or deleting an evaluator — however it was created — also requires **Team**. See [Plan limits](#plan-limits).
## Deleting an evaluator
An evaluator can be depended on — most commonly by an alert rule that watches its score. Deleting it out from under that dependency would silently break the alert, so the delete confirmation runs a [blast radius](/dashboard/blast-radius) check first and shows the result inline: every dependent found (alert rules today), with its call volume and cost where known.
Unlike the equivalent check shown before a [prompt release](/dashboard/prompts#blast-radius-before-releasing), this one **blocks**. If the evaluator has any dependents — or the check itself fails to load, which is treated the same as "there might be dependents" rather than silently letting the delete through — you have to explicitly check an acknowledgement box before **Delete** becomes clickable. An evaluator with no dependents deletes immediately, no extra step.
## Custom evaluator templates
Beyond the built-in library, you can author your own evaluator templates from the **Templates** tab on the Evaluations page. Built-in and custom templates live in the same catalog — you browse both, then deploy any of them into a live evaluator when you're ready. This is where you write an actual scoring prompt/rubric — the New Evaluator dialog above does not expose one.
Navigate to **Evaluations** and select the **Templates** tab.
The **Create Custom Template** dialog opens.
Give the template a name, and a category such as `quality`, `safety`, `performance`, `agent`, or `rag` — or type your own.
This is the rubric the judge model applies to each trace — describe what to look for and how to score it, the same way you would for a built-in template.
The metric key becomes the identifier for this evaluator in trend charts. The threshold (0.0–1.0) is the cutoff score above which a result counts as a pass.
Select how the judge should express its verdict:
* **Numeric** — a score between 0.0 and 1.0
* **Categorical** — the judge picks exactly one label from a list you define, entered as comma-separated values, e.g. `helpful, unhelpful, unclear`
* **Boolean** — a true/false verdict
Override the project's default judge model for this template alone. See [Overriding the judge model per template](#overriding-the-judge-model-per-template) below.
Use **Test on a real trace** to preview scoring before saving — see [Testing a template before you deploy it](#testing-a-template-before-you-deploy-it) below. When you're satisfied, click **Create Template**.
The new template appears in the catalog alongside the built-in ones. Click **Deploy** on its row to turn it into a live evaluator, the same way you would enable any built-in template.
### Overriding the judge model per template
By default, every evaluator uses your project's default judge model. When creating a custom template, you can pin a specific judge for that template instead: choose a provider — **OpenAI**, **Anthropic**, or **Google** — and enter the model name (for example `gpt-4o`, `claude-sonnet-4-20250514`, or `gemini-2.5-flash`). Leave the provider set to its default option to keep using the project's default judge model.
This lets you match judge cost and strength to the rubric — a cheaper, faster model for a simple pass/fail check, a stronger model for a template that requires nuanced judgment.
### Testing a template before you deploy it
Before saving a new template, use the **Preview & Test** panel in the creation dialog to see how it scores. Click **Test on a real trace** to submit your rubric, along with your most recently ingested trace, to the judge model synchronously. The result — a score and the judge's reasoning — appears in the dialog within a few seconds.
A dry run doesn't create an evaluator or save any results. It's a quick check that your rubric produces the score you expect before you commit to deploying it.
Every LLM-judge call — the template dry run above, auto-evaluation, a manual **Run now**, and the ad-hoc playground run described below — requires a project **LLM connection**. Without one, the call fails with "No LLM connection configured — add one in Settings → LLM Connections." Connect a provider key under [LLM Connections](/platform/llm-connections) before using any of these.
### Scope: trace vs. session
Every template above judges a single trace by default (`scope: "trace"`). Setting `scope: "session"` instead makes the deployed evaluator judge an entire conversation — every trace sharing a `sessionId`, folded into one ordered transcript — rather than one call in isolation. This is useful for quality signals that only show up across multiple turns, like the assistant contradicting itself or drifting off-topic over a long back-and-forth.
The **Create Custom Template** dialog doesn't expose a scope toggle yet, so set it via the API when creating or updating a template:
```bash theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
curl -X POST https://api.zespan.com/v1/projects/{projectId}/evaluation-templates \
-H "Authorization: Bearer $ZESPAN_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"name": "Conversation coherence",
"category": "quality",
"evaluatorType": "llm_judge",
"metricKey": "conversation_coherence",
"defaultThreshold": 0.7,
"systemPrompt": "Judge whether the assistant stayed coherent across the whole conversation...",
"scope": "session"
}'
```
Deploy it from the **Templates** tab the same way as any other template — the scope carries over to the live evaluator automatically. A session-scope evaluator never scores an individual trace; it only runs once a session is detected as complete, and its results are written separately from per-trace scores (no `traceId`/`spanId`, a `sessionId` instead). See [Session-level evaluation](/dashboard/sessions#session-level-evaluation) for how "session complete" is detected and where these scores are surfaced.
## Running evaluations
After creating an evaluator, you run it against a selection of traces.
**Auto-evaluation:** Enable **Auto-run** on an evaluator to have it score every new trace automatically as it arrives. This is useful for monitoring ongoing quality in production.
**Manual run:** Click **Run now** on any evaluator to score a batch of recent traces immediately. You can choose how many recent traces to include (up to 500).
Both modes display results in the evaluator's trend chart within a few minutes of completion.
### Scoring a Playground response
You can also run an evaluator outside this page entirely: on the **Playground** page, each output window has a **Run Eval** action that lets you pick any evaluator from this project and score that window's prompt/response pair synchronously — useful for checking how an evaluator would score a draft response before it ever becomes a trace. Like every other judge call described above, it requires a project LLM connection.
## Sampling and filters for auto-evaluation
When an evaluator is running continuously against live traffic, you can scope exactly which traces it judges instead of scoring every single one. Open the **Evaluators** tab and click the gear icon next to an active evaluator to open **Sampling & Filters**.
You can configure:
* **Sample rate** — a fraction from 0 to 1 of matching traces to judge. Set it to `0.1`, for example, to evaluate roughly 1 in 10 matching traces and control judge-model cost; leave it at `1.0` to evaluate every eligible trace.
* **Minimum output tokens** — skip traces whose output is shorter than this token count, so trivially short responses don't consume judge calls.
* **Models** — a comma-separated list of model names to restrict evaluation to, e.g. `gpt-4o, claude-sonnet-4-20250514`.
* **Operations** — a comma-separated list of operation types to restrict evaluation to, e.g. `chat, completion`.
* **Statuses** — a comma-separated list of trace statuses to restrict evaluation to, e.g. `success`.
Leave any filter blank to match everything. Use the **Sampling Enabled** toggle to turn sampling on or off without losing your configured values, then click **Save** to apply.
Combine sampling and filters to scope auto-evaluation to the slice of traffic you actually care about — for example, judge only 20% of `chat` operations on your production model, skipping short responses that wouldn't produce a meaningful score.
## Score direction
By default, a higher judge score means a better result. That's backwards for metrics like toxicity or hallucination rate, where a lower score is the good outcome. Click **Direction** on a deployed **LLM JUDGE**-type evaluator's row (Evaluators tab) to set which way that evaluator's score should be read:
* **Higher is better** — the default. A higher score is a better result.
* **Lower is better** — e.g. toxicity, hallucination rate. A lower score is a better result.
The **Direction** button only appears on `LLM JUDGE` type evaluators — `CLASSIFIER`, `METRIC CHECK`, and `PATTERN DETECT` evaluators don't show it. Score direction isn't just cosmetic: it's used wherever a score is turned into a pass/fail verdict, including trace verdicts, [Issues](/dashboard/issues) clustering, and regression checks — not only on this page.
## Reading evaluation results
A **categorical** or **boolean** evaluator (see [Custom evaluator templates](#custom-evaluator-templates)) displays its result as the judge's chosen label, or `true`/`false`, in the **Evaluators** table on the trace detail view. It no longer renders as a 0–100% bar there: a percentage has no meaning for a label like `helpful` or a boolean verdict. This is scoped to that one surface — the cards and tables on this page, and the generic score chip used elsewhere, still show a raw percentage regardless of score type. Numeric evaluators are unaffected everywhere and still show a 2-decimal score with the usual progress bar.
The **Evaluators** tab lists every evaluator in the project. Each row shows:
* **Name** and a **type badge** — `LLM JUDGE`, `CLASSIFIER`, `METRIC CHECK`, or `PATTERN DETECT`, color-coded so the four types are visually distinguishable at a glance
* **Latest score** — the most recent average score for the selected time window
* **Score distribution** — a green/amber/red bar showing the share of recent scores that passed (0.8 or above), landed in the warning band (0.5 up to 0.8), or failed (below 0.5) within the current window
* **Trend** — in the **Trends** density view, a sparkline of the average score across the selected time window (the window picker above the list — Last 1 hour / 24 hours / 7 days — controls both the sparkline and the score distribution, not a fixed lookback)
* **Sample count** — how many traces were scored in the current window
Switch between **Compact**, **Full**, and **Trends** density using the toggle above the evaluator list — Compact favors row count, Full shows more per-evaluator detail in a card grid, Trends surfaces the sparkline.
If your evaluator score drops sharply after a deployment, open the **Traces** page and filter to the same time window. The evaluation score is shown in the trace detail panel so you can correlate low-scoring traces with specific model calls.
### Evaluation runs
The **Runs** tab lists batch scoring jobs — the ones created from **Run now**, a dataset run, or a retroactive run. Each row shows the run's **Name**, the **Evaluator** it used, a **Status** badge (`completed`, `failed`, `running`, or `pending`, each with its own icon), the run's **Score** (average across its scored items, once completed), how many **Samples** it scored, and when it was **Created**.
## Evaluating RAG pipelines
If your traces include retrieval spans — captured automatically by `recordRetrieval()`, the LangChain integration, or attached manually via `span.recordDocuments()` (`span_kind: "retriever"`) — five evaluator templates can score the retrieval step itself, not just the final answer:
| Template | What it checks | Requires retrieved chunks |
| ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | ------------------------- |
| **Groundedness** | Whether the response is supported by whatever context is available — retrieved chunks if present, the trace's system prompt otherwise | No — degrades gracefully |
| **RAG Faithfulness** | Whether every claim in the response is specifically supported by the retrieved chunks | Yes |
| **Context Relevance** | Whether the retrieved chunks are actually relevant to the query — judges the retriever, independent of the generated answer | Yes |
| **Context Utilization** | What fraction of retrieved chunks were actually used in the response | Yes |
| **Retrieval Hit Rate** | Deterministic, no LLM call — did the retriever return anything at all | Yes |
All five are reference-free, like every other evaluator on this page — no ground-truth answer set required. Deploy any of them from the **Templates** tab under the **RAG** category, the same way you'd deploy any built-in template.
The four RAG-specific templates — everything except Groundedness — only score traces that actually have a retrieval span with chunks. A trace with no retrieval step is skipped for these evaluators entirely: no score row, no LLM call, no cost, rather than a meaningless score against nothing.
## RAG analytics
The **Retrieval** tab on this page rolls RAG evaluator scores up across traces: a trend card per metric, and a ranked list of your worst-performing retrieval operations — the fastest way to spot a retriever that's degraded on one specific operation before it shows up as a wave of bad answers.
If Retrieval Hit Rate is trending down on an operation, check that first. A faithfulness or context-relevance drop is often downstream of the retriever returning nothing, not the judge model getting worse.
## Attaching scores from the SDK
You can attach evaluation scores to any span programmatically using `span.setEvalScore()`. This is useful when you compute quality scores in your own code — for example, using a custom similarity function for RAG faithfulness.
```typescript theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
import { zespan, startSpan } from "@zespan/sdk";
const { span } = startSpan({ name: "rag-pipeline", provider: "custom" });
try {
const answer = await generateAnswer(query, docs);
// Attach your computed scores before closing the span
span.setEvalScore("faithfulness", computeFaithfulness(answer, docs));
span.setEvalScore("relevance", computeRelevance(answer, query));
await span.end({ status: "success" });
return answer;
} catch (err) {
await span.end({ status: "error", error_message: String(err) });
throw err;
}
```
```python theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
from zespan import start_span
with start_span(name="rag-pipeline", provider="custom") as span:
answer = generate_answer(query, docs)
span.set_eval_score("faithfulness", compute_faithfulness(answer, docs))
span.set_eval_score("relevance", compute_relevance(answer, query))
```
Scores attached via the SDK appear in the same trend charts as LLM-as-judge scores. If you attach a score with the same name as an evaluator, both sources are shown together in the detail view — and, as of this dashboard's KPI tiles and metric list, in the aggregate too: `setEvalScore()` results and server-side judge results for the same metric key are combined into one count-weighted average per time bucket rather than only ever appearing in the raw per-trace view. No setup is required on your side; this happens automatically the moment a span carries an eval score.
## Performance & agent evaluators
Four built-in templates score performance and agent-trajectory signals directly from trace data — no LLM call, no judge cost:
| Template | Metric key | What it checks | Requires |
| ----------------------- | --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------- |
| **Latency SLA** | `latency_sla` | Whether the span's `latency_ms` is within a target (default 5000ms) | The span's latency to be recorded |
| **Cost Budget** | `cost_budget` | Whether the span's `cost_usd` is within a budget (default \$0.05) | The span's cost to be recorded |
| **Loop Detection** | `loop_detection` | The longest run of consecutive identical operations across the trace's spans so far — a sign of an agent stuck repeating the same tool call | At least 2 spans in the trace |
| **Error Recovery Rate** | `error_recovery_rate` | Of the trace's error spans, what fraction were followed later by a successful call to the same operation — a retry-and-recover pattern | At least 1 error span in the trace |
Each is a real deterministic check against the trace's own data (`latency_ms`, `cost_usd`, `operation`, `status`) — not a generic LLM quality-judge rubric scoring "output quality" in the abstract. **Loop Detection** and **Error Recovery Rate** skip a trace entirely (no score row, no cost) when there isn't enough signal yet — fewer than 2 spans, or no error spans respectively — rather than fabricate a score against nothing.
`slaTargetMs` (Latency SLA) and `budgetUsd` (Cost Budget) use fixed defaults (5000ms / \$0.05) for every evaluator deployed from these templates. There's no dashboard control to change these per-evaluator yet — reach out if you need a different threshold.
## Retroactive evaluation
An evaluator only auto-scores traffic from the moment it's turned on — but you don't need to have predicted which metric you'd want in advance. The **Retroactive Runs** panel on this page scores historical traces you've already ingested against any evaluator, including one that wasn't configured for auto-run (or didn't exist yet) when that traffic actually happened.
Opens the retroactive-run dialog. Requires at least one evaluator to already exist.
Choose any existing evaluator — built-in or custom, numeric, categorical, or boolean.
Pick a **From** and **To** date/time — the historical window to score. "From" must be earlier than "To".
Enter an operation (e.g. `chat.completions.create`) or model name (e.g. `gpt-4o-mini`) to restrict scoring to a slice of that time range instead of everything in it.
Zespan resolves every matching trace and scores it with the same LLM-judge pipeline a live auto-evaluation run uses.
### Watching progress
Each run appears in the **Retroactive Runs** list with a progress bar and status label:
| Status | Meaning |
| --------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Queued** | The run has been created and enqueued, not yet picked up. |
| **Running — X/Y traces scored** | The worker is actively scoring; X of Y matched traces are done so far. |
| **Completed — X/Y traces scored** | Every matched trace has been scored (or the run finished with zero matches). |
| **Failed** | The run couldn't complete — for example, no LLM connection was configured for the judge model. The failure reason is shown inline when available. |
The list refreshes automatically while any run is still **Queued** or **Running**, so you don't need to reload the page to watch a run finish.
A retroactive run's results land in the same score data as any other evaluator run, so they appear in that evaluator's trend chart and everywhere else its scores are shown — no separate "retroactive results" view to check.
Need a human to review traces directly instead of running them through an LLM judge — for calibration, or for cases too ambiguous to trust to a judge model? See [Annotation Queues](/dashboard/annotation-queues).
## Plan limits
| Plan | Auto-evaluation | Built-in templates | Monthly scored traces |
| ----- | ----------------------- | ------------------ | --------------------- |
| Free | Manual only | — | 500 |
| Solo | Included (12 templates) | 12 | 5,000 |
| Pro | Included | 12 + custom | 25,000 |
| Team | Included | 12 + custom | 100,000 |
| Scale | Included | 12 + custom | Unlimited |
This table covers auto-evaluation and template access, not evaluator management. Deploying a template into a live evaluator has no separate plan gate on any of the plans above (only the `evaluations:manage` permission). Using the **New Evaluator** dialog to create one directly, or editing/deleting any existing evaluator, requires the **Team** plan or above regardless of which plan row you're on.
# Guardrails — configure content safety policies
Source: https://docs.zespan.com/dashboard/guardrails
Create and manage guardrail policies in the Zespan dashboard to block, redact, or flag unsafe LLM inputs and outputs without redeploying your application.
The Guardrails page is where you define the content safety policies that the Zespan SDK enforces at runtime. Policies are evaluated server-side on every check request from the SDK — you can update, enable, or disable them without touching your application code.
**Guardrails is the runtime view. [Policies](/dashboard/policies) is where they come from.** A guardrail rule on this page was either hand-built here, or compiled from a policy — a versioned document authored in the dashboard's Policies section or as a YAML file in your repository. Guardrails shows what's enforcing and what fired; Policies is where you author, review, backtest, and apply the change. This page's ownership model — Code-managed vs. dashboard-managed, and Detach — is the same mechanism either way.
The Guardrails page — and every action on it (create, enable, disable, delete, install from a template) — is available on the **Solo** plan and above. The Free plan has no access.
## How guardrails work
When your SDK is initialized with `guardrails: true` on a provider wrapper, it sends a check request to `POST /v1/guardrails/check` before the LLM call (pre-check) and after the LLM response (post-check). The backend evaluates all active policies for your project against the content and returns a verdict.
The SDK receives the verdict and either:
* **Allows** the call to proceed normally
* **Blocks** it by throwing a `GuardrailBlockedError`
* **Redacts** sensitive content and substitutes the cleaned text
* **Warns** (logs the trigger but allows the call through)
See the [SDK guardrails guide](/sdk/guardrails) for how to handle these verdicts in your application code.
## Creating a guardrail
Every guardrail is created from one page — there's no separate "quick install" grid or creation dialog. **Add guardrail** on the Guardrails page always opens it.
Navigate to **Guardrails** in the left sidebar.
Opens the guardrail-creation page.
Pick a starting point from the **Content** or **Agent safety** template chips at the top of the page — each pre-fills a working configuration (type, phase, action, and default settings) for one of the guardrail types below — or click **Blank** to start from an empty draft. Below the chips, **Preset templates** (bundles Zespan maintains and auto-updates) and **Your templates** (custom bundles saved for this project) are also available — see [Guardrail templates](#guardrail-templates).
| Type | What it does |
| ------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------- |
| `pii` — **PII detection** | Detects personal information (emails, phone numbers, credit cards, SSNs, IPs, names, and more) and can redact it in place |
| `toxicity` — **Toxicity filter** | Flags toxic, harassing, or threatening language at a configurable sensitivity (low/medium/high) |
| `topic_boundary` — **Topic boundary** | Blocks content matching a keyword blocklist, or requires it to match an allowlist |
| `regex` — **Regex block** | Blocks content matching one or more regular expressions you define |
| `format` — **Output format** | Validates that a response is valid JSON and contains any required fields — intended for post-response checks |
| `cost_ceiling` — **Cost ceiling** | Blocks a call whose estimated cost or input token count exceeds a limit you set — intended for pre-call checks |
| `custom_llm` — **Custom LLM judge** (deprecated) | Runs your own evaluation prompt against the content and blocks/warns based on a pass/fail score. Deprecated in favor of `regex` — see the note below |
The following types apply specifically to agent traces — they inspect tool calls and agent names rather than raw prompt/response text:
| Type | What it does |
| ------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------- |
| `agent_rate_limit` — **Agent rate limit** | Caps how many requests, tokens, or dollars an agent can consume in a time window — intended for pre-call checks |
| `tool_misuse` — **Tool misuse** | Blocks disallowed tools, or a tool that's been called too many times in one trace — intended for pre-call checks |
| `loop_detection` — **Loop detection** | Blocks an agent that repeats the same tool call (with the same arguments) too many times in a row — intended for pre-call checks |
| `agent_misuse` — **Prompt injection / goal hijacking** | Detects prompt injection, jailbreak, and goal-hijacking attempts at a configurable sensitivity |
| `scope_enforcement` — **Scope enforcement** | Keeps an agent's output within an allowed set of topics via keyword allow/blocklists |
| `delegation_control` — **Delegation control** | Restricts which agents another agent is allowed to hand off (delegate) to — intended for pre-call checks |
`custom_llm` is deprecated — Zespan steers new policies toward `regex` instead. It still works if you already have one configured, but prefer `regex` (or another pattern-based type above) for new guardrails.
Give it a descriptive name that explains what it protects against, e.g. `block-competitor-mentions` or `toxicity-filter`.
**Phase** selects whether the check runs pre-LLM (the prompt), post-LLM (the completion), or both — the pipeline preview above the fields shows which stages are solid (checked) versus dashed (skipped) as you change it. **Action** selects what happens when the guardrail triggers:
* **Block** — reject the request and throw `GuardrailBlockedError` in the SDK
* **Redact** — remove the matched content and use the cleaned text
* **Warn** — allow the request but surface the trigger as a warning
* **Log** — allow the request and record the trigger in execution history, without surfacing it as a warning (useful while you're still tuning a new policy)
Fill in the fields for the type you picked. For keyword-based types (topic boundary, scope enforcement, tool misuse allow/blocklists), enter the terms. For regex, enter the patterns. For toxicity and prompt-injection types, pick a sensitivity level (low/medium/high). For PII detection, choose a compliance preset and, optionally, fine-tune the confidence threshold under advanced detection. For custom LLM judge, write your evaluation prompt.
Open **Advanced settings** to set max latency, priority (lower numbers run first), whether the rule applies **project-wide** or only to **specific agents**, and whether it's enabled on create.
The right-hand column runs a client-side **Live preview** against sample text as you configure, and a **Deterministic test** panel that evaluates your unsaved draft against the real guardrail engine. Once you're happy with the result, click **Create guardrail** in the sticky footer. The guardrail activates immediately — all subsequent SDK check requests will include it.
Use the **Deterministic test** panel before creating — it runs the real rule synchronously against your draft settings, not just the in-browser preview — see [Testing a guardrail](#testing-a-guardrail) below.
## Promoting a violation to a policy
Every guardrail hit recorded against a trace can become a permanent rule with one click, right from where it happened — no need to reconstruct the pattern from scratch.
Find the trace with the violation you want to codify. The guardrail hit shows as its own span in the flame graph, colored to match its outcome.
Click it to see the policy, the check phase, and the exact value that triggered it.
Zespan pre-fills a new guardrail draft with the same tool, field, and value that triggered the violation.
Adjust the draft if you want — tighten a keyword list, change the action, scope it to specific agents — then save it like any other guardrail.
This is the fastest way to turn a one-off bad response you noticed into a rule that catches every future occurrence.
## Guardrail templates
Instead of creating guardrails one at a time, you can bundle several rules into a reusable **template** and apply that bundle to a project (optionally scoped to specific agents, the same way a single guardrail can be). Templates live on the guardrail-creation page (**Add guardrail**), below the built-in template chips described in [Creating a guardrail](#creating-a-guardrail) — there's no separate templates tab.
Zespan ships a set of **preset templates** that are managed and auto-updated with new threat patterns:
| Preset | What it bundles |
| ----------------- | --------------------------------------------------------------------------------------------------------------------------------- |
| Jailbreak Shield | Prompt-injection detection plus scope enforcement, across all agents |
| PII Protection | PII detection tuned to block SSNs and credit cards, warn on emails and phone numbers, plus regex rules for common API key formats |
| Content Safety | Toxicity filtering plus a topic-boundary warning for off-topic responses |
| Agent Safety Pack | Tool misuse, loop detection, delegation control, and agent rate limiting together |
Under **Preset templates** and **Your templates** you can:
* **Apply** a preset or a custom template to your project, project-wide or scoped to specific agents
* **Clone** a preset into an editable copy you can tweak (presets themselves can't be edited or deleted)
* **Create** your own template from scratch by combining any of the guardrail types above into one bundle
* **Edit** or **delete** your own templates, and **remove** an application without deleting the template itself
Applied templates run alongside any standalone guardrails you've created — both are evaluated on every check request, and both show up as rows in the [Rules table](#managing-guardrails).
## Managing guardrails
The Guardrails page has two tabs: **Rules** and **Activity**.
### Status band
At the top of the page, a single status band answers "what's running and is anything wrong":
* A posture headline — *"Fully protected"*, *"N protections active, M gaps"*, or *"Your agents are running unguarded"* — plus an **N of 5 active** count, scored across five protection categories: PII, Injection, Content, Cost, and Secrets.
* Any category with no active rule shows as a chip (e.g. *Secrets +*) that links straight to **Add guardrail** so you can close the gap.
* A stat line for the selected time range: checks, blocked, redacted, avg check latency, and block rate, with a small block-rate sparkline.
### Rules table
The **Rules** tab lists every guardrail as a row, sorted by phase (pre, then both, then post) and then by priority within each phase:
| Column | Description |
| ------ | ---------------------------------------------------------------------------------------------------- |
| Rule | The guardrail's name, an enabled/disabled dot, and a **Code** badge if it's managed by a policy file |
| Type | The guardrail mechanism (`pii`, `toxicity`, etc.) |
| Phase | Pre, post, or both |
| Action | What happens on trigger (block, redact, warn, log) |
| Checks | Total checks in the selected time range |
| Blocks | Total blocks in the selected time range |
| P90 | P90 check-latency overhead |
Each row has a **⋯** menu with **Configure** (opens the guardrail's detail page), **Enable**/**Disable**, and **Delete**. A code-managed row only offers **Configure** — see [Dashboard-managed vs code-managed](#dashboard-managed-vs-code-managed).
Click a guardrail's name (or **Configure**) to open its detail page, with **Configure**, **Test**, and **Logs** tabs for editing settings, running the guardrail against sample text, and reviewing its recent trigger history. The **Configure** tab also shows the pipeline diagram — the same Pre → Both → Post chain that used to sit on the landing page — highlighting where this specific rule sits.
### Activity tab
The **Activity** tab is the project-wide event log — see [Execution history](#execution-history).
### Dashboard-managed vs code-managed
A guardrail can be owned either by the dashboard or by a policy file in your
repository. Code-owned guardrails carry a **Code-managed** badge in the Rules
table's Rule column, with the file path in the tooltip.
On a code-managed guardrail's detail page:
* The settings form is **read-only**, and a banner names the file that owns it:
*Managed in `policies/hipaa-phi-egress.yaml`. Edit it there, or Detach to take
over from the dashboard.*
* The **Delete** button is replaced by **Detach**.
* Enable/disable and delete are unavailable from the row's **⋯** menu — only **Configure** shows.
This is what keeps the two surfaces from fighting. A dashboard edit to a
code-owned policy would silently diverge from git and be overwritten by the next
`zespan policy apply`, with nothing to warn you it had happened.
#### Detaching
**Detach** hands ownership of that policy to the dashboard: the form becomes
editable immediately, and the next `zespan policy apply` reports the policy as a
conflict and refuses to overwrite it without `--force`. Use it when you need to
change a control faster than a pull request allows — during an incident, for
example — then fold the change back into the file afterwards.
Detaching is recorded in the audit log as `policy.detached`.
Detach requires the same permission as applying policies: **owner** or
**admin**. Editors can view a code-managed guardrail but not take it over.
See [Policy as code](/policies/as-code) for the authoring side.
#### Adopting a dashboard guardrail into code
If you want a guardrail you built here to live in git instead, you do not have
to rewrite it: `zespan policy pull` generates the file from what already exists,
and `zespan policy apply --adopt` takes it over **in place** — the guardrail
keeps its id, so its execution history and metrics carry over unchanged. See
[Adopting a project that already has guardrails](/policies/as-code#adopting-a-project-that-already-has-guardrails).
### Reviewing and applying a policy's changes
For a policy authored in the dashboard, reviewing and applying its pending
changes now lives in [Policies](/dashboard/policies) — open the policy from
its inventory row and use **Review changes** on its detail page. That screen
renders exactly what `zespan policy plan` would print for the same change,
computed through the same endpoint, so the two can never disagree. Reviewing
needs only `policy:read`, which every role has; applying needs
`policy:apply`.
For policy files that haven't been committed to your repository yet, the
equivalent check is `zespan policy plan` from the CLI — see
[zespan policy](/cli/policy).
## Execution history
The Guardrails page's **Activity** tab is a log of recent guardrail events, filterable by rule (and each guardrail's own detail-page **Logs** tab shows just its events), with:
* Timestamp
* The guardrail and check phase (pre/post)
* The action taken (blocked, redacted, warned, logged)
* The reason the guardrail triggered (e.g. which PII types or keyword matched — not the full prompt)
* A link to the underlying trace
From the **Activity** tab, each event has a **Mark as False Positive** button. Feedback you submit rolls up into a false positive rate, so you can see at a glance whether a policy is producing mostly genuine catches or noise.
If a guardrail is triggering frequently, review its execution history and mark any false positives. You can then adjust the keyword list, sensitivity, or confidence threshold without redeploying.
## Testing a guardrail
Every guardrail (and the form for a new one) has a test panel where you can paste sample text and run it through the guardrail's current configuration — this evaluates the real rule synchronously and returns a verdict, without needing a live trace from your application:
* While creating a guardrail, the test panel runs against your unsaved draft settings
* On an existing guardrail's **Test** tab, it runs against the saved configuration
* You can optionally supply a model name, operation name, estimated cost, and input token count — the cost ceiling type checks these directly
* The result shows an overall **Allowed** / **Blocked** verdict, the modified text (if a redact rule matched), and a per-guardrail breakdown of which rule fired, what action it took, whether it passed, and its latency
The test panel evaluates text content only. Agent-context checks that depend on the calling agent's name or its recent tool calls — agent rate limit, tool misuse, loop detection, and delegation control — always pass in the test panel, since that context only exists on a real trace. Validate those types by checking their execution history after live traffic runs through them.
Use this to validate a policy change before it affects real traffic.
## Latency impact
Guardrail checks add latency to your LLM calls. The check runs synchronously before (and optionally after) the LLM call. Typical check latency:
| Guardrail type | Typical latency |
| -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------- |
| Pattern/rule-based (regex, topic boundary, toxicity, scope enforcement, prompt injection, tool misuse, loop detection, delegation control, cost ceiling, agent rate limit) | \< 5ms |
| PII detection | 20–50ms |
| Custom LLM judge (deprecated) | 200–800ms |
Custom LLM judge is deprecated — use `regex` (or another pattern-based type above) for new guardrails instead. It's also the only type that calls a model for every check, adding real latency; every other type runs entirely on pattern matching, keyword lists, or counters.
## Human approval gates
Beyond automatic block, redact, and warn actions, the SDK exposes a real human-in-the-loop primitive: `awaitApproval()` pauses execution until an admin approves or rejects the call from the dashboard — not just a log entry, an actual gate your code waits on.
```typescript TypeScript theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
import { zespan } from "@zespan/sdk";
const client = zespan.getClient();
// Blocks until an admin approves or rejects from the Approvals inbox
await client.awaitApproval("delete_database", { table: "users" });
// throws ApprovalRejectedError / ApprovalTimeoutError, or resolves silently on approval
```
```python Python theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
from zespan import get_client
client = get_client()
# Blocks until an admin approves or rejects from the Approvals inbox
client.await_approval("delete_database", {"table": "users"})
# raises ApprovalRejectedError / ApprovalTimeoutError, or returns silently on approval
```
Pending requests appear in the **Approvals** inbox (Guardrails → **Approvals** tab) with the tool name, its arguments, and the requesting agent. An Owner or Admin approves or rejects each one; your application resumes — or raises the corresponding error — as soon as a decision is made.
Reserve this for genuinely high-risk or irreversible tool calls — deleting data, sending money, publishing externally — where you want a human in the loop before the action executes, not a warning after the fact.
## Near-miss capture and suggested rules
Zespan also learns from traffic that almost triggered a numeric guardrail rule but didn't. When a threshold-based check — `cost_ceiling`, `agent_rate_limit`, and similar types — evaluates close to its limit without firing, that near-miss is logged instead of silently discarded.
A background worker clusters recurring near-misses into **suggested policy rules**, surfaced in a panel on the Guardrails page: *"Your agents have hit this pattern 6 times this week — no rule governs it yet. Want one?"*
* Click **Promote to Policy** on a suggestion you agree with — it reuses the same promote flow described [above](#promoting-a-violation-to-a-policy)
* Click **Dismiss** to clear a suggestion that isn't worth a standing rule
This means your policy set gets measurably stricter over time from real agent behavior, instead of staying a static list someone wrote once.
## Plan requirement
The Guardrails page, and every action on it — creating, editing, enabling, disabling, deleting a guardrail, and installing a template — is available from the **Solo** plan up. The Free plan has no access.
## Next steps
Author, review, backtest, and apply policies from the dashboard.
Manage these policies as YAML in your repository, reviewed in pull requests.
Every field in a policy file, and the supported YAML subset.
validate, plan and apply from the CLI or CI.
Enforce these policies from your application.
# HTTP Targets
Source: https://docs.zespan.com/dashboard/http-targets
Register an externally-hosted agent endpoint so a dataset run can call it directly, without instrumenting it with the Zespan SDK.
## What it is
An **HTTP Target** is a registered, externally-hosted endpoint — a deployed Amazon Bedrock or Glean agent, or any chatbot/agent API you don't control or can't add the Zespan SDK to. Once registered, the ["Run over dataset"](/dashboard/datasets#running-against-an-http-endpoint) flow can POST each dataset item directly to that endpoint and capture its raw response as a trace, without your own pipeline code in the loop.
Use this when you want to evaluate an agent you can only reach over HTTP — you don't own its deployment, or it runs on a stack the SDK doesn't wrap — but you still want it scored against a dataset like any other run.
If you *can* add the Zespan SDK to the service being tested, prefer that: a normal [dataset run](/dashboard/datasets#how-dataset-runs-work) captures your pipeline's real trace tree (retrieval spans, tool calls, sub-agent hops), not just a single request/response pair. HTTP Targets exist for the case where that isn't possible.
## Where to register one
Go to **Project Settings → HTTP Targets**.
## Registering a target
Give the target a name (shown wherever you pick it for a run) and the endpoint's full URL, e.g. `https://agent.example.com/invoke`.
Choose how Zespan authenticates to the endpoint:
| Auth mode | What it sends |
| ------------------ | ---------------------------------------------------------------- |
| **No auth** | No credential attached. |
| **Bearer token** | An `Authorization: Bearer ` header. |
| **API key header** | Your credential in a header you name, e.g. `X-Api-Key: `. |
Like an [LLM Connection](/platform/llm-connections), the credential is encrypted at rest and **write-only** — once saved, it's never returned by the API or shown in the UI again; the list only shows whether a credential is set.
Write the JSON body Zespan should POST to the endpoint, with the literal placeholder `{{input}}` wherever a dataset item's input should be substituted, e.g.:
```json theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
{"message": "{{input}}"}
```
`{{input}}` can also appear in a header value or in the URL itself. At run time it's replaced with the dataset item's `input` (JSON-escaped so the result is always valid JSON when it sits inside a body string).
Editing a target's auth mode or rotating its credential always requires supplying a fresh credential value — a stored secret is never re-readable to pre-fill an edit, the same as an [LLM Connection](/platform/llm-connections).
## Security: URL validation
A target URL must be `http`/`https` and must resolve to a public address. It's validated server-side before it's saved, and validated **again** on the *hydrated* URL immediately before every outbound call — so a dataset item's input can't smuggle a request toward an internal address even if `{{input}}` is substituted into the URL itself. A blocked call is recorded as a failed trace rather than silently skipped or allowed through.
These ranges are rejected:
| Range | What it covers |
| ------------------------------------------------- | ----------------------------------------------------------------------------------------------------- |
| `127.0.0.0/8`, `::1`, `0.0.0.0` | Loopback and unspecified addresses |
| `10.0.0.0/8`, `172.16.0.0/12`, `192.168.0.0/16` | RFC 1918 private networks |
| `169.254.0.0/16`, `fe80::/10` | Link-local, including the `169.254.169.254` cloud instance-metadata address |
| `fc00::/7` | IPv6 unique local addresses |
| `100.64.0.0/10` | RFC 6598 carrier-grade NAT shared address space — **this includes Tailscale's default address range** |
| `224.0.0.0/4` and above | Multicast and reserved space |
| `localhost`, any `.internal` or `.local` hostname | Non-public hostnames |
IPv4-mapped-IPv6 (`::ffff:a.b.c.d`) and NAT64 (`64:ff9b::a.b.c.d`) spellings of a blocked address are rejected too, so a blocked range can't be reached by rewriting it as IPv6.
There is no allowlist or override for HTTP Targets — a target must be reachable at a public address. An agent that's only reachable over a private network or a Tailscale/CGNAT address (`100.64.x.x`–`100.127.x.x`) cannot be registered as an HTTP Target. Expose it at a public hostname, or instrument the service with the [Zespan SDK](/quickstart) and run a normal [dataset run](/dashboard/datasets#how-dataset-runs-work) instead.
## How a run differs from a prompt-version run
Running against an HTTP Target is a distinct execution mode from running against a registered [prompt version](/dashboard/prompts#the-quality-gate):
* **Prompt version:** Zespan calls an LLM provider directly, using your prompt template and an [LLM Connection](/platform/llm-connections).
* **HTTP Target:** Zespan POSTs your hydrated request template straight to the endpoint you registered — no LLM connection is used or required — and records the raw response as a trace tagged `sdk_name: "zespan-http-endpoint"`, so it's easy to tell apart from traces your own SDK-instrumented code produced.
Each outbound call retries on a network error or a `429`/`5xx` response (with backoff), and never follows redirects — a `3xx` response is recorded as a terminal failure rather than followed, since a target agent has no legitimate reason to redirect a dataset-run request.
## Trace propagation
Every outbound call carries a W3C `traceparent` header built from a freshly generated trace/span id pair — the same id pair the resulting Zespan trace is stored under. If the target agent is itself instrumented with its own OpenTelemetry SDK and configured to export to a Zespan-reachable endpoint, and it extracts and continues that incoming trace context, its own spans can land under the identical trace id — giving you a combined view even though Zespan didn't call the agent's code directly.
## Deleting a target
Deleting a target removes it immediately. Dataset runs already created against it keep their captured traces; new runs can no longer select it.
## Next steps
* [Datasets](/dashboard/datasets#running-against-an-http-endpoint) — start a dataset run against a registered HTTP Target
* [LLM Connections](/platform/llm-connections) — the closest analog for how credentials are stored and masked
* [Prompt version runs & the quality gate](/dashboard/prompts#the-quality-gate) — the alternative "Run over dataset" mode that calls an LLM directly
# Incidents — detect, correlate, and resolve LLM issues
Source: https://docs.zespan.com/dashboard/incidents
The Incidents page surfaces active and resolved issues detected across your LLM traffic, correlates related anomalies and errors, and tracks resolution state.
The Incidents page groups related problems — anomalies, error spikes, latency regressions — into a single timeline so you can investigate them as a unit. When Zespan detects that multiple signals are happening at the same time and likely have a common cause, it opens an incident automatically. You can also open incidents manually for any issue you want to track to resolution.
Incident detection and management require the **Team** or **Scale** plan.
## How incidents are created
Zespan creates incidents automatically when:
* An anomaly is detected (cost spike, error rate jump, latency surge) **and** a related alert rule fires within 15 minutes
* Three or more traces with the same error code occur within a 10-minute window
* An AI analysis detects a pattern across multiple traces that it classifies as a systemic issue
You can also open an incident manually from the **New incident** button on the Incidents page, or by clicking **Open incident** on any anomaly card in **AI Features**.
## Incident states
Each incident moves through three states:
| State | Meaning |
| ------------- | --------------------------------------------------------------------------- |
| **Detecting** | The platform is still gathering signals — the issue may still be developing |
| **Active** | Confirmed ongoing issue requiring attention |
| **Resolved** | The issue is no longer occurring |
Zespan auto-resolves an incident when its driving metric returns to baseline for 30 consecutive minutes. You can also resolve an incident manually.
## The incidents list
The main Incidents page shows a table of all incidents, ordered by most recent. Each row shows:
* **Severity** — `high`, `medium`, or `low`, based on impact to cost or error rate
* **Title** — a one-sentence summary of the issue
* **Affected metric** — which measurement is out of range
* **State** — detecting, active, or resolved
* **Duration** — how long the incident has been open
* **Models affected** — which model(s) are involved
Use the **State** filter to see only active incidents, or the **Severity** filter to focus on high-severity issues.
## Incident detail
Click any incident to open its detail view. The detail view shows:
### Timeline
A chronological feed of all signals related to this incident:
* Anomaly detections with their explanation and severity
* Alert rule triggers with threshold and actual value
* Related error traces (grouped by error code)
* Configuration changes that may have contributed (from the audit log)
### Changes around this incident
A before/after view of everything that changed in the project in the six hours on either side of when the incident started — prompt deploys, agent lifecycle transitions, policy/evaluator/alert edits, and changes reported from your own pipeline. Both sides are labeled with their count (**Before (N)** / **After (N)**) so the split itself is the takeaway: what was already true when the incident started, versus what happened next. If the underlying data sources don't all respond in time, a banner marks the result as partial rather than silently showing an incomplete list as complete.
This is the same change feed available project-wide from the **Changes** section of the sidebar, pivoted on this incident's start time instead of a date range.
### Root cause analysis
When an incident is created, Zespan automatically runs AI root cause analysis across the correlated traces. The results appear at the top of the incident detail view under **AI Root Cause Analysis**.
| Field | Description |
| -------------------- | ------------------------------------------------------- |
| Root cause | Single-sentence diagnosis of what caused the failure |
| Contributing factors | Up to 5 conditions that made the problem worse |
| Suggested fix | The single highest-impact action to resolve the issue |
| Prevention tips | Up to 5 steps to avoid the same failure class in future |
| Confidence | `high`, `medium`, or `low` |
The analyzer uses a fast path for common patterns (rate limits, timeouts, context length exceeded, provider 5xx errors) and falls back to AI analysis for novel cases. Trace content is sanitized before analysis — only error details, model metadata, span structure, and latency are used.
If analysis is still running, the section shows "Analysis in progress…" and updates automatically when complete.
### Propose Fix
For an incident ZespanPilot has traced to a single causal prompt deploy — the same case the **Recommended fixes** rollback suggestion already covers — a **Propose Fix** button lets you go one step further than a suggestion: generate an actual fix candidate, test it against this incident's own failures, and route it for approval, all without leaving the incident. No other step on this page changes anything by itself; **Propose Fix** is the one closed loop that goes from "here's what broke" to a tested, human-approved prompt version.
Click **Propose Fix**. Zespan runs the same [Enhancer](/dashboard/prompts#enhance) used on a prompt's **Enhance** tab, grounded only in the real failing traces captured during this incident's own time window — not a generic rewrite. The result is saved as a new, unlabeled draft version of the causal prompt.
Zespan replays this incident's exact failing traces against both the current production prompt and the draft, scores both runs with your project's own [LLM-judge evaluator](/dashboard/evaluations) — the same judge that scores this project's production traces — and compares them with the same [quality gate](/dashboard/simulations#ci-gate-for-batch-runs) used elsewhere in Simulations. Only a higher-is-better judge is used: a "lower is better" rubric (toxicity, hallucination, and similar) would invert the gate's comparison, so those are skipped. If your project has no suitable LLM-judge evaluator deployed, Zespan provisions a reference-free response-quality judge (**Closed-Loop Fix Response Quality**) for this comparison; it is created disabled, so it never scores your live traces or adds cost to ingest.
If the draft doesn't beat production on the gate, the run stops there. The draft prompt version stays a draft — nothing is sent for approval — and the incident shows the gate's reason for failing.
If the draft beats production, a request appears in [ZespanPilot's approval queue](/dashboard/zespanpilot), showing the gate's evidence — score deltas and regression count — alongside the request.
The comparison replays each captured failure's own recorded tool calls, not live ones — that's what makes it a faithful replay of the incident rather than a fresh run against whatever your tools return today. When that replay isn't fully faithful, the run reports `unverified` instead of `passed`, even if the score compared favourably. This happens for either of two independent reasons: a tool call was made but its result was never captured, so the replay had to fall back to a live call; or your project's [retention window](/account/billing#plans) had already aged out the incident's tool-call data by the time the run tried to read it back, so there was nothing to replay against at all. Either way, a favourable score isn't treated as evidence the fix works — **no approval request is created**, the same as an outright gate failure.
Approving lets the reviewer choose whether to promote the fix to `staging` or straight to `production`. Rejecting leaves the draft untouched. The reviewer must be someone **other than** the person who clicked **Propose Fix** — you cannot approve your own fix candidate.
While a run is in progress, the card shows its current status:
| Status | Meaning |
| ---------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `pending` / `generating_fix` | Candidate generation is running |
| `testing` | The candidate and production prompt are being compared on the gate |
| `gate_failed` | The candidate didn't beat production — draft left as-is, nothing proposed |
| `gate_unverified` | The score compared favourably, but the replay wasn't faithful enough to trust — a tool result wasn't captured, or retention had already aged the data out. Treated the same as a failure: nothing proposed |
| `awaiting_approval` | The candidate beat production — waiting on a human in the approval queue |
| `approved` / `rejected` | A reviewer has decided |
| `failed` | The run hit an error — see the error message shown on the card |
**Propose Fix** requires the incident to have exactly one identified causal prompt deploy. If the incident's timeline has zero or more than one candidate cause, the button isn't available — the same requirement the **Recommended fixes** rollback suggestion already applies. Requires the **Team** or **Scale** plan and **both** the `incidents:manage` and `prompts:manage` permissions — because the run creates a new prompt version, it is restricted to **owner** and **admin** roles; an **editor** cannot start one.
Your organization also needs at least one **other** owner or admin besides you. Because a fix candidate can't be approved by the person who requested it, a run started in an organization with a single admin could never be approved — so Zespan refuses it up front rather than spending a generation and test cycle on a proposal that would dead-end in the approval queue.
Nothing is ever auto-deployed. A human approval is always required before any prompt label changes — even when the gate passes cleanly.
### Affected traces
A filtered list of the specific traces associated with this incident, with error codes, latency, and cost. Click any trace to open it in the flame graph view.
### Resolution notes
A free-text field where you can record what you found and how you fixed it. Resolution notes are preserved after the incident closes and appear in the incident history. Use them to build a runbook for recurring issues.
## Resolving an incident
Click **Mark resolved** on any active incident. You'll be prompted to add a brief resolution note. Once resolved:
* The incident state changes to "Resolved"
* The resolution timestamp and note are saved
* If the same underlying issue recurs, a new incident opens automatically — it does not reopen the closed one
Always add a resolution note before closing an incident. Future incidents of the same type will surface the previous resolution notes so your on-call engineer can see what worked before.
## Auto-remediation
On the Scale plan, you can configure auto-remediation rules that ZespanPilot applies automatically when an incident of a specific type opens. For example:
* "When a GPT-4o error spike incident opens, switch to GPT-4o-mini"
* "When a rate-limit incident opens, reduce sample rate to 50%"
Auto-remediation rules are configured in **Settings → Incidents** and require explicit approval from an Owner-level user to activate.
Auto-remediation applies SDK config changes without human confirmation. Only enable it for actions you have validated are safe to apply automatically. All auto-remediation actions are logged in the audit trail.
## Notifications
Incidents trigger the same notification channels as alert rules — email for Pro/Team, and webhooks for Scale. If an incident is opened while an alert for the same metric is active, Zespan deduplicates the notification so you do not receive duplicate alerts.
## Next steps
* [Changes](/dashboard/changes) — the project-wide timeline behind the "Changes around this incident" panel, including how to report your own pipeline deploys
* [Alerts](/dashboard/alerts) — the threshold rules that can open an incident automatically
* [ZespanPilot](/dashboard/zespanpilot) — the approval queue that Propose Fix requests land in
# Issues — recurring failures grouped automatically
Source: https://docs.zespan.com/dashboard/issues
Zespan clusters repeated failed and degraded traces into a single recurring Issue, using the same deterministic verdict system shown on every trace, so you stop re-discovering the same problem one trace at a time.
The Issues page turns N separate failing traces that are really the same underlying problem into one row: a recurring **Issue** with an occurrence count, a first/last-seen timestamp, and a link to a real sample trace.
Issue clustering runs automatically in the background — there's nothing to configure. It requires the **Pro** plan or higher.
## How clustering works
A background worker re-runs the same deterministic verdict classifier used on every trace detail page (the one that decides whether a trace is `healthy`, `degraded`, or `failed`) against recent candidate traces, then groups matches by `(verdict level, primary operation, error code, signal)`. Traces that land in the same group become occurrences of one Issue.
### What counts as a candidate
Clustering used to consider only traces where a span **errored**. It now also considers traces that failed without erroring:
* an evaluator scored the trace below its configured threshold — or above it, for evaluators like `toxicity` and `pii_leakage` where a high score is the bad one
* an evaluator returned a non-passing verdict outright, which is how categorical rubrics (`safe` / `borderline` / `unsafe`) are judged
* a **behavioural signal** fired — see below
* a tool call was retried, or ran far slower than its siblings
This is the class of failure where nothing throws: the agent returns a confident wrong answer, or takes a wrong action, and no log line says so.
Because it's the same classifier your trace detail page already shows you — not a second, fuzzier system guessing from raw scores — an Issue's grouping key means exactly what it says: these traces failed the same way, for the same reason, on the same operation.
## The Issues list
Each row shows:
| Field | Description |
| ------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Verdict level | `degraded` or `failed` |
| Operation | The operation type this issue recurs on |
| Cause | What the traces in this cluster share — an error code where one exists, otherwise the signal that identified them (see [Signals](#signals)). Never blank: an issue with neither reads *No cause recorded* |
| Occurrences | How many traces have matched this issue |
| Last seen | The clustering window this issue spans |
| Actions | **View details**, **View sample trace**, **View related prompt** (if applicable), **Resolve**, **Dismiss** |
Click **View details** to open the [Issue detail page](#issue-detail-page) for the full sample-trace list and the remediation suggestion feature. **View sample trace** jumps straight to one real occurrence in the normal flame graph view, so you can investigate the actual failure without leaving the list.
## Signals
When a trace fails without erroring, the **Cause** column names the signal that identified it rather than an error code.
| Signal | What it means |
| -------------------------------------------------- | ------------------------------------------------------------- |
| An evaluator slug (e.g. `toxicity`, `correctness`) | That evaluator judged the trace's output and did not pass it |
| `user_retry` | The same question was asked more than once in one session |
| `agent_loop` | One tool was called three or more times inside a single trace |
| `tool_retry` | A tool call was retried before succeeding |
| `tool_slow` | A tool call ran far slower than the others in the trace |
`user_retry` and `agent_loop` are **behavioural signals**: observations about how a conversation went, not judgements of what the agent said. A person may rephrase a question for their own reasons, and some agents legitimately call one tool repeatedly. Treat them as evidence worth reviewing, not as proof the agent was wrong. Trace detail labels them **Evidence** for exactly this reason, and never shows them a pass/fail threshold — there is no rubric they were scored against.
Two issues on the same operation with different signals stay separate. A `toxicity` failure and a `user_retry` observation on `chat` are different problems and get their own rows, their own occurrence counts, and their own regression datasets.
## Resolved vs. dismissed
Both actions remove an issue from the open Issues list — the difference is what they communicate, the same distinction [Sentry-style issue trackers](https://docs.sentry.io/product/issues/states-triage/) draw between the two:
| Action | Meaning |
| ----------- | ------------------------------------------------------------------------------------- |
| **Resolve** | "I fixed the underlying problem." |
| **Dismiss** | "Stop showing me this recurring pattern" — regardless of whether it's actually fixed. |
Neither is permanent: if the aggregator detects a fresh occurrence of the same cluster (same verdict level, operation, error code, and signal) after an issue was resolved or dismissed, the issue reopens and reappears in the open Issues list. Resolving an issue you haven't actually fixed just means it comes back sooner.
An issue with a rapidly climbing occurrence count is usually a better place to start than sorting the Traces log by timestamp — it's already told you this isn't a one-off.
## Issue detail page
Clicking into an Issue (`/{orgSlug}/{projectId}/issues/{issueId}`) shows:
* The verdict level, current status, occurrence count, and first/last-seen timestamps
* A link to the associated prompt, if this issue's traces share one
* **Mark resolved** / **Dismiss** actions (same semantics as the list — see [above](#resolved-vs-dismissed))
* A **Sample traces** table — up to 10 representative traces from the cluster, each linking out to its full trace detail view
* The **Generate suggestion** remediation feature, described below
### Generate suggestion (remediation)
Requires the **Pro** plan or higher, same as the rest of the Issues feature.
Click **Generate suggestion** on the Issue detail page to have Zespan investigate the pattern for you: it runs the same [root cause analysis](/dashboard/traces#root-cause) used on individual traces against up to 5 representative sample traces from the cluster, then writes a short suggestion — the likely underlying cause and one concrete next step, reasoned over the *recurring pattern* rather than a single occurrence.
Every statement in that suggestion is checked against the record it refers to before you see it, and each one shows what it rests on — the trace, prompt, policy or metric behind it — linking through where that record has a page.
Two things follow from that check, and both are deliberate:
**Suggestions are shorter and more specific than they used to be.** Statements that cannot be tied to a record in your project are not shown, so text that previously read as confident but rested on nothing no longer appears.
**Sometimes there is no suggestion at all.** Where nothing could be checked against your data you will see *"No suggestion: nothing here could be checked against your data"* rather than a plausible-sounding guess. A suggestion you cannot verify is worse than none — it costs you the time to act on it and the trust you would have placed in the next one. If a run stops early it is labelled incomplete alongside whatever was verified.
This is a **markdown suggestion only**. Generating it never opens a pull request, proposes a code diff, or touches any source repository — Zespan has no repo-connection integration today. Treat it as a starting point for your own investigation, not an automated fix.
The suggestion is cached per issue, so regenerating it (e.g. after revisiting the page) is instant and doesn't re-run the analysis or incur additional cost. If none of the sampled traces can be analyzed — for example, they've aged out of trace retention — you'll see an error instead of a suggestion; try again once the issue has recurred and produced fresher samples.
### Automatic remediation suggestions
Requires the **Pro** plan or higher, same as manual **Generate suggestion**. Free and Solo plan Issues are not auto-analyzed, but you can still generate a suggestion yourself at any time using the manual button above.
Once an Issue has recurred **three or more times**, Zespan generates a remediation suggestion for it on its own — you don't have to ask. The suggestion appears on the Issue detail page labelled with the time it was generated, so opening a recurring failure usually means reading the analysis rather than requesting it.
The analysis reasons **across** the Issue's sample traces rather than about one of them, and says which situation it found:
| Finding | What it means |
| --------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| One shared root cause | Every sampled occurrence points at the same underlying failure. The suggestion describes it once and gives one next step. |
| Several failure modes | The occurrences are different problems that happen to share the same verdict, operation, and error code. The suggestion groups them and gives a next step per group. |
| Single occurrence | Only one sample could be analysed (the others aged out of retention, or carry no error span). The finding is marked provisional. |
Suggestions are text only. Zespan never opens a pull request, edits your prompts, or changes your configuration as part of generating one — you decide what to act on.
You can still press **Generate suggestion** yourself on an Issue that hasn't reached three occurrences, or ask ZespanPilot directly: "investigate the rate limit issue on chat.completions.create."
## Issues feed regression tests
Once an issue has recurred 3 or more times, it's automatically captured as a test case in a **Production Failures** dataset — turning your own incident history into a regression suite your CI can replay against a candidate change. See [Regression testing from production failures](/dashboard/datasets#regression-testing-from-production-failures).
## Next steps
* [Traces](/dashboard/traces#root-cause) — the underlying root-cause analysis that powers remediation suggestions
* [ZespanPilot](/dashboard/zespanpilot) — the AI copilot that writes remediation suggestions
* [Guardrails](/dashboard/guardrails) — if an issue's error code traces back to a guardrail violation, promote that violation into a permanent rule
* [Datasets](/dashboard/datasets) — replay recurring issues as regression tests
* [Incidents](/dashboard/incidents) — for anomaly-driven correlation across metrics, rather than verdict-based trace clustering
# Model Lifecycle — provider deprecation radar
Source: https://docs.zespan.com/dashboard/model-lifecycle
Detects when a model you're actually calling has a provider-announced end-of-life, with real measured call volume, cost, and affected agents — never a fabricated quality or regression comparison.
Model Lifecycle watches the models your project actually calls against a curated catalogue of provider-announced deprecation and retirement dates, and raises a finding when one of your models is heading toward end-of-life. Every number on a finding is measured from your own traffic — nothing here is estimated or simulated.
## What the radar watches
A daily scan (04:00 UTC) sweeps your project's model usage from the last 30 days and matches every model you called — by model id, disambiguated by provider when the same id is published by more than one — against Zespan's bundled catalogue of provider deprecation announcements (`ModelLifecycle`). See [the feed reference](/reference/model-lifecycle-feed) for exactly what's in that catalogue and how it's maintained.
A model only produces a finding when **both** are true:
* It has an announced `retiresAt` date, and that date falls within the detection horizon (90 days by default)
* Your project actually called it at least once in the last 30 days
A model with an announced retirement date that you've never called, or one retiring further out than the horizon, produces nothing — there's no finding to review and no widget to see.
The catalogue is curated and bundled with each release, not scraped from a provider's page at request time. This also means a self-hosted install with no outbound internet access evaluates lifecycle findings normally — there's no live fetch in the critical path.
## Where findings appear
| Location | What you see |
| ---------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Overview** (Engineering view) | A "Model risk" widget showing your single most urgent open finding, plus a count of any others. It renders nothing at all when there are no open findings — it never sits on the dashboard as an empty state. |
| **Models page → Lifecycle column** | Every model with a catalogue entry or an open finding shows a countdown chip (e.g. "Retires in 34 days", "Retired 429 days ago"). Models with no known lifecycle show a plain dash. See [Models](/dashboard/models#lifecycle-column). |
| **Model Lifecycle page** (`/models/lifecycle`) | The full findings list for the project, filterable by **Open**, **Dismissed**, or **All**. This is the page both the overview widget and the Models page banner link to. |
## What a finding contains
Each finding is one (project, model) pair. Everything below is either read directly from the catalogue or measured from your own ClickHouse trace data over the trailing 30 days — nothing is inferred beyond that.
| Field | Source | Notes |
| ---------------------------- | ------------ | ----------------------------------------------------------------------------------------------- |
| Model / provider | Your traffic | The exact model id and provider you called |
| `deprecatedAt` / `retiresAt` | Catalogue | The provider's own announced dates; `retiresAt` may be in the past for an already-retired model |
| Days remaining | Computed | Whole days from now to `retiresAt`, floored on the UTC day. Negative means already retired. |
| Calls / 30d, Cost / 30d | ClickHouse | Real call volume and spend on this model, in this project, over the trailing 30 days |
| Affected agents | ClickHouse | Every agent name that called this model in the window |
| Affected prompts | ClickHouse | Every `name@vN` prompt version that called this model in the window |
| Successor model | Catalogue | The provider's own named replacement, when one was announced |
| Cost comparison | ClickHouse | See below |
| Source / last verified | Catalogue | `announced`, `inferred`, or `manual`, plus when a human last checked the `sourceUrl` |
### Cost comparison
When the provider named a successor and you have real traffic on it too, the finding shows the measured difference in average cost per call between the two models over the same 30-day window. This is a comparison of **your own organic traffic on both models** — not a matched, controlled comparison, since the two models may be serving different operations. If you have no traffic on the successor yet, the finding says so plainly instead of showing a blank or a zero.
Zespan does not run your evaluations or regression suite against the successor model, and does not report a quality delta between the two. Nothing in this platform invokes a model on your behalf — everything shown is measured from calls you already made. Compare quality yourself before switching. See [what's not built yet](#whats-not-built-yet) below.
## The re-raise ladder and dismissing a finding
A finding's urgency is expressed as a **band**: 90, 30, 7, or 0 days remaining (0 meaning the model has already retired). Zespan notifies the first time a finding is created, and again only when the band *tightens* — the deadline gets meaningfully closer. A daily re-scan that lands in the same band is silent by design: a re-run with nothing new to say would train you to ignore the feature by the time it actually matters.
**Dismissing** a finding (with a required reason) suppresses it at its current band. It stays dismissed through further daily scans as long as the band doesn't tighten. When the retirement date moves into a nearer threshold — 90 → 30 → 7 days, or crosses into retired — the finding automatically reopens exactly once and notifies again, clearing the old dismissal reason. A band that *widens* (a feed correction pushes the retirement date further out) never reopens a dismissed finding.
From the Model Lifecycle page, the Models page banner, or the Overview widget.
Call volume, cost, affected agents and prompts, and the cost comparison against the successor if you have traffic on it.
Dismissing requires a short explanation (e.g. "migrating next sprint") so a later reviewer knows why it was suppressed. Leaving it open keeps it visible until you act.
## Getting notified
Findings are always recorded and visible on this page regardless of alert configuration — but by default nobody is notified when a new one appears. A small settings card at the top of this page turns that on:
"Notify me about model deprecations", at the top of this page.
A comma-separated list of notification emails, and optionally a webhook URL. The webhook is validated against the same SSRF-safety check every other alert webhook in Zespan goes through — a URL that resolves to a private or internal address is rejected.
Takes effect on the next daily scan that raises or re-raises a finding — see the re-raise ladder above for when that is.
This is a dedicated, minimal opt-in for this feature only — it does not appear on, and is not part of, the general [Alerts](/dashboard/alerts) page, since a model-lifecycle rule has no metric, condition, or threshold for that page's table to display. See [Alerts → Model lifecycle alerts](/dashboard/alerts#model-lifecycle-alerts) for how this fits into the shared delivery machinery (same email/webhook pipeline every other alert rule uses).
## What's not built yet
* **No quality or regression comparison.** Producing "how would the successor model have performed on your traffic" would mean actually invoking that model on your behalf — a capability that doesn't exist anywhere in Zespan today. The cost comparison above is the only automated comparison this feature makes, and it's explicitly labeled as a cost comparison, never a quality one.
* **No drift detection.** Model Lifecycle only watches for provider-announced retirement dates. It does not detect a provider silently changing a pinned model's behavior over time — that's a separate, harder problem tracked for a future release.
* **No automatic migration.** Nothing here writes to a prompt, a config file, or a model string. A finding is information; acting on it is up to you.
## Next steps
* [Models](/dashboard/models#lifecycle-column) — the Lifecycle column and countdown chip on the model registry table
* [Alerts](/dashboard/alerts#model-lifecycle-alerts) — get notified by email or webhook when a new finding is raised
* [Model lifecycle feed reference](/reference/model-lifecycle-feed) — the exact catalogue fields, how it's curated, and how to report a missing or wrong entry
# Models — per-model usage, reliability, and latency
Source: https://docs.zespan.com/dashboard/models
A per-model table of usage, cost, latency, error rate, and agent adoption, plus a Lifecycle column flagging models with a provider-announced end-of-life.
The Models page (labeled **Model registry** in the dashboard) breaks down every model your project has called over the selected time range, so you can compare cost, latency, and reliability model-by-model rather than only in aggregate.
## Summary stats
Four figures across the top of the page, aggregated over the selected range:
| Stat | What it shows |
| -------------- | ------------------------------------------------------- |
| Models used | Distinct model ids called in range |
| Total requests | Sum of calls across every model |
| Total cost | Sum of spend across every model |
| Avg error rate | Mean per-model error rate (not weighted by call volume) |
## Table columns
| Column | Details |
| --------- | ------------------------------------------------------------------------------------------------------------ |
| Model | Model id, with a share-of-calls bar underneath and a status dot (green under 1% error rate, red at or above) |
| Provider | The provider that served the call |
| Lifecycle | See [below](#lifecycle-column) |
| Calls | Total calls in range |
| Cost | Total spend in range, with a share-of-total bar |
| Cost/req | Average cost per call |
| Latency | p50/p95 latency bar |
| Errors | Error count; blank when zero |
| Agents | Count of distinct agents that called this model |
Sort by cost, calls, errors, or latency using the selector in the page header; search and filter by provider using the controls above the table.
## Lifecycle column
The Lifecycle column shows a countdown chip for any model that either has an open [Model Lifecycle](/dashboard/model-lifecycle) finding in this project, or simply has a known retirement date in Zespan's catalogue — even if that date is far enough out that it hasn't produced a finding yet. This is the one place on this page where a model can show lifecycle information without there being an active alert: the catalogue is broader than "worth notifying about."
The chip's color follows the same urgency ladder as the findings page:
* **Neutral** (grey) — more than 30 days out, or catalogued but not yet been called
* **Warning** (amber) — 30 days or fewer remaining
* **Danger** (red) — 7 days or fewer remaining, or already retired
A model with no catalogue entry and no finding shows a plain dash. When at least one model on the page has an open finding, a banner appears above the table linking to the full [Model lifecycle findings page](/dashboard/model-lifecycle).
## Model detail
Click any row to open that model's own detail page — the follow-up to the table above: "should I switch this workload to something cheaper?"
The header repeats the model id, its provider, an overall health dot (green under 1% error rate), and how long it's been called plus its trace/agent/operation counts. A range selector (7d/30d/90d) scopes every panel below, same as the table.
### Overview
* **Volume and cost** — calls and spend per day
* **Latency** — average response time and time-to-first-token per day
* **Cache effectiveness** — share of input tokens served from cache per day; blank when this model has never returned a cached token
### Cost vs quality
A scatter of every model called in the project over the same range — average cost per call on the x-axis, average eval score on the y-axis, bubble size scaled to call volume. This model's own point is highlighted; points below the axis line have no quality score yet.
Below the chart, a **Cheaper alternatives** table lists every model that costs less per call and scores no worse on quality (a small tolerance absorbs noise near a tie) — the direct candidates for "switch this workload here." A model with no quality score of its own still surfaces here on cost alone, since there's no evidence yet that it's worse.
### Reliability
Daily error rate as a bar chart, plus a **Top error codes** table ranking the most frequent failure reasons for this model in range.
### Quality
Average eval score, pass rate, and evaluator count, followed by a daily score trend and a per-evaluator breakdown (evaluations, average score, pass/fail counts). Shows an empty state instead of zeros when no evaluator has scored a trace using this model in the selected range — attach an evaluator to a prompt, or run a retroactive evaluation, to populate it.
### Usage
Two tables — which agents call this model, and which SDK operations route work to it — each with calls, cost, average latency, and error rate, so you can see not just how a model performs but who's actually depending on it.
Model ids that contain a dot or slash (`gpt-5.6-luna`, `openrouter/some-provider/some-model`) work as expected — the dashboard URL-encodes the model segment and decodes it back before display, API calls, and the page title.
## Next steps
* [Model Lifecycle](/dashboard/model-lifecycle) — the full deprecation radar: findings, the re-raise ladder, and dismissing
* [Costs](/dashboard/costs#cost-by-model) — the cost-by-model chart this table's Cost column complements
* [Model lifecycle feed reference](/reference/model-lifecycle-feed) — where the Lifecycle column's dates come from
# System Health — your agent health at a glance
Source: https://docs.zespan.com/dashboard/overview
The System Health page shows a verdict banner judged against your own thresholds, golden-signal KPI cards, a triage row, and five drill-down tabs — all in one place.
The **System Health** page is the first thing you see after opening a project. Instead of a fixed set of "OK" badges, it computes a live verdict — healthy or degraded — against thresholds *you* define for that project, and it never shows a fake healthy status before you've sent any real traffic.
## Before your first trace
System Health is available on every plan, including Free.
A brand-new project with zero traces doesn't show a dashboard full of sample numbers. It shows an honest empty state:
* A **wait pill** at the top reads "No traces yet" until your SDK sends its first event, then "First trace received," then "N traces · warming up" as data accumulates.
* The **verdict slot** stays blank with an explanation — "No verdict yet" — until there's enough traffic to judge. One trace alone isn't enough to compute an error rate, so the page says so rather than guessing.
* Every KPI slot (error rate, P95 latency, spend today, requests) renders as an empty, dashed placeholder with a `—` value and a **WAITING** badge instead of a fake number.
A floating **Getting Started** checklist walks you through the six steps to get here: create your workspace, set your thresholds, connect the SDK, see your first trace, customize the dashboard, and take a tour (optional). It tracks your progress automatically — the SDK and first-trace steps tick off as soon as real data arrives, no manual confirmation needed.
If a project has ingested traces before but the currently selected time range has none, you get a different, narrower empty state: a "quiet window" message if the gap is under 48 hours (with a one-click jump to a wider range), or a "we haven't heard from your agents" warning with likely causes (rotated API key, SDK not initializing, network/firewall blocking ingest) if the gap is longer.
## Setting your health thresholds
Set your thresholds before you need them — do this from the empty-state screen the first time you open a new project.
Before any traffic flows, you can tell Zespan what "broken" means for this specific project. A 20% error rate is a crisis for a payments agent and a normal Tuesday for a research prototype — only you know which applies here.
| Threshold | Direction | Default |
| ----------------- | --------------------- | ------- |
| Error rate | Breach **above** this | 5% |
| P95 latency | Breach **above** this | 5s |
| Daily spend | Breach **above** this | \$25 |
| Tool success rate | Breach **below** this | 90% |
These values are saved per project and used automatically once real data starts arriving — you don't have to touch them again. The System Health verdict banner judges your live metrics against these numbers, not a generic fixed threshold.
## The verdict banner
Once your project has real traffic, the top of the page shows one of two states:
* **Within all thresholds** (or **All systems healthy** if you never customized thresholds) — a summary line naming your current error rate, P95 latency, and today's spend, plus how many agents and tools are reporting.
* **System degraded** — names exactly which threshold was breached and by how much (e.g. "Error rate 12.0% is above your 5% limit"), with a badge showing the breach count and the active incident count, and quick links to review incidents or see the failing traces.
Tool success is only judged if your project has actually invoked tools — pure-LLM projects with no tool calls never get a false "0% tool success" breach.
Below the verdict, an **active alerts strip** appears if any [alert rules](/dashboard/alerts) are currently firing, showing up to four with a link to view all.
## KPI cards and triage row
Four golden-signal cards sit below the banner — **Error Rate**, **P95 Latency**, **Spend (Today)**, and **Requests** — each with a status badge, a trend arrow versus the prior period, a sparkline, and your threshold as a caption.
Beneath the cards, a two-column triage row surfaces what needs attention next:
* **What to fix first** — up to three ranked, concrete problems (the agent with the worst error rate, an operation exceeding its latency target, a cost spike), each with a direct link to investigate.
* **ZespanPilot digest** — an AI-generated summary of what changed, with quick-action links into the relevant page.
## Drill-down tabs
Five tabs below the triage row let you go deeper without leaving the page:
| Tab | What it shows |
| ------------------ | ------------------------------------------------------------------------------------------------------------------------------- |
| **Health** | Success rate, failed runs, tool success, cost per run, failure reasons, agents ranked by error count, and a recent-errors table |
| **Performance** | P50/P95/P99 latency, throughput, cache hit rate, a latency-over-time chart, and the slowest agent runs |
| **Cost** | Today's spend, 30-day projection, cost per run, token usage, spend by model, and spend by operation |
| **Agents & tools** | The full operations table plus a tool-performance breakdown |
| **Traffic** | Hourly run volume and success rate, top agents by volume, and provider reliability |
## Customizing your dashboard
Click **Customise** in the page header to open a panel where you can toggle which KPI widgets are tracked for your project — grouped into Essentials, Cost & Usage, Performance, Health, and Explore — or apply a starting layout (**Engineering**, **Operations**, or **Leadership**). Your selection is saved to your project's dashboard preferences. The first time you complete onboarding you're also prompted to pick one of these starting layouts.
## Time range
The range selector in the top-right controls the window all KPIs, the trend chart, and the tabs are computed over: **Today**, **Last 24h**, **Last 7 days**, **Last 30 days**, or a custom date range. Data refreshes from a 5-minute cache; use the refresh button next to the range selector to force an update (rate-limited to once every 5 seconds).
If your weekly spend more than doubles versus the prior week, a cost-spike banner appears above the checklist with a direct link to the [Costs page](/dashboard/costs).
# Performance — latency metrics and throughput analysis
Source: https://docs.zespan.com/dashboard/performance
Track P50, P90, and P99 latency, time-to-first-token for streaming calls, and request throughput across models, environments, and time ranges.
The Performance page gives you detailed latency and throughput metrics for your LLM operations. While the Overview page shows a single average latency number, Performance lets you drill into the distribution — including the tail latency that affects your slowest users — and break it down by model, environment, and time period.
## Latency percentiles
The top section of the Performance page shows three percentile cards for the selected time range:
Half of your requests complete faster than this. Represents the typical user experience.
90% of requests complete faster than this. The first signal of performance problems affecting a significant minority of users.
99% of requests complete faster than this. Represents your worst-case latency for regular users. This is the number that wakes people up.
Each card shows a trend arrow comparing the current period to the previous equivalent period, so you can see whether tail latency is improving or degrading over time.
## Latency distribution histogram
Below the percentile cards, a histogram shows the full distribution of response times for the selected period. The x-axis is latency in milliseconds; the y-axis is the number of requests that fell in each bucket.
Use the histogram to understand the shape of your latency distribution:
* **Tight distribution** — most requests take roughly the same time, which means your application is predictable
* **Long tail** — a small fraction of requests are significantly slower, which may indicate timeouts, retries, or large inputs
* **Bimodal distribution** — two distinct peaks often indicate two different code paths (e.g., cached vs. uncached requests, or two different models)
## Time-to-first-token (TTFT)
For streaming LLM calls, time-to-first-token (TTFT) is often more important to users than total latency — it's how long they wait before they see any output. The TTFT card shows your P50, P90, and P99 TTFT for streaming requests.
TTFT is only available for requests where `stream: true` was sent. The SDK captures TTFT automatically for all wrapped streaming calls.
A high TTFT with low total latency suggests the model is spending most of its time on prompt processing before generating output. This is common with large system prompts — consider caching or compressing them.
## Latency by model
The **By model** table ranks all models you've used by their P99 latency. For each model you see:
| Column | Description |
| ------------- | ------------------------------------------- |
| Model | The model identifier |
| P50 | Median latency |
| P90 | 90th percentile latency |
| P99 | 99th percentile latency |
| TTFT P50 | Median time-to-first-token (streaming only) |
| Request count | Number of calls in the period |
Click any model row to filter the histogram and time-series chart to that model only.
## Latency over time
The time-series chart below the table shows how P50, P90, and P99 latency have moved over the selected date range. Toggle individual percentile lines on or off using the legend. Use this chart to:
* Correlate latency changes with deployments
* See whether tail latency is trending up before it becomes a user-facing issue
* Identify time-of-day patterns (e.g., provider slowdowns during peak hours)
## Throughput
The **Throughput** section shows requests per minute (RPM) over time. Use this to:
* Verify that your application handles traffic peaks without dropping requests
* Correlate throughput changes with latency changes (high throughput often correlates with higher tail latency)
* See your peak traffic periods, which is useful for capacity planning
## Filtering
All charts and tables on the Performance page respect the global filter bar:
* **Date range** — select a preset or custom range
* **Model** — filter to one or more models
* **Environment** — separate production, staging, and development data
* **User ID** — drill into performance for a specific user
Filters update all charts simultaneously.
# Playground — compare a prompt across models side-by-side
Source: https://docs.zespan.com/dashboard/playground
Run one versioned prompt against up to three models at once, compare cost, latency, and output, and save the winner as a new prompt version.
The Playground is where you compare **one prompt** across **different models** (or, per window, different prompt variants) to see what actually changes before you commit to it. The design is deliberately narrow: one shared prompt, up to three comparison windows, real execution against your own models — not a free-form chat toy.
Playground execution requires an [LLM connection](/platform/llm-connections) configured for your project — it always runs against your own connected provider, never a shared/managed key. If none is configured, you'll see a **Connect a provider** banner in place of the run controls.
## Opening the Playground
Navigate to **Playground** in the left sidebar for a blank session, or click **Test** in the header of a [prompt's detail view](/dashboard/prompts) to open it pre-filled with that prompt's text, version, and — if there's a recent failing generation — a one-click **Replay a failing trace** shortcut.
## The shared prompt rail
The left-hand rail holds **the prompt** — the single versioned artifact every comparison window runs by default. Edit it once and every window updates:
* **Single mode** — one template with `{{variable}}` placeholders.
* **Chat mode** — a system message plus a list of seed turns (few-shot examples, or the exact state a real conversation was in when it broke).
* **Tools & structured output** (collapsed by default) — shared tool definitions (JSON) and an optional structured-output schema, used by every window.
A **Save as new version** button on the rail versions this shared prompt directly, without needing to run it first.
This panel is what gets versioned. Everything in a comparison window below it — output, cost, latency — is disposable evidence of how the prompt behaves, not the artifact itself.
### Per-window override
Any window can break away from the shared prompt: click **Override** on a window to give it its own prompt text (a **CUSTOM PROMPT** badge marks it), or **Reset to shared** to go back to inheriting the rail. This is how you A/B two different prompt variants — not just two models — inside the same comparison.
## Comparison windows
Each window is an independent run: pick a model, run it, read the result. You can have up to **3 windows** open at once — beyond that stops being a comparison and starts being noise. Use the dashed **Add a window** tile to add one (it duplicates the active window's connection/model/toggles, but never its results), and duplicate or remove windows from each card's header.
### Connection-aware model selection
Each window has its own connection/model picker. The **model list is derived from the LLM connection you pick** — you can't accidentally select a Google model while only an OpenAI connection is configured. Switching a window's connection resets its model to a sensible default for that connection's provider (or keeps your current pick if it's still valid).
### Single vs. Chat mode
A page-level toggle switches every window between:
* **Single** — one request in, one output out. Shows output text, token counts, cost, latency, guardrail results, and (if the prompt calls tools) an interactive tool-call panel where you can submit synthetic tool results and continue the run.
* **Chat** — a live multi-turn conversation per window, seeded from the rail's system prompt and seed turns. Non-baseline windows (window 2 and 3) show a **divergence banner** the moment their tool-call pattern differs from window 1's corresponding turn — turn-by-turn, based on real transcript comparison, not a guess.
### Toggles
Each window has independent **Streaming** and **Guardrails** toggles (guardrails apply your project's configured policies to the run and show pass/fail banners), plus the temperature/max-tokens summary from the shared config.
### Reading results
Non-baseline windows (2 and 3) show latency and cost as a percentage delta against window 1, so you can see at a glance whether a model is faster/cheaper or slower/pricier — never a fabricated number when the baseline hasn't run yet. You can also run any configured evaluator against a window's output directly from its **Run Eval** control.
## The verdict banner
Once window 1 and the last window both have results, a verdict banner summarizes the comparison:
* If a guardrail blocked one window's output, the banner leads with that — a blocked output is a failure, not a saving, and isn't treated as a valid comparison.
* Otherwise it reports whether the outputs matched (single mode only), the latency/cost deltas, and — when the session was seeded from a prompt with known call volume — a projected daily cost delta at that volume.
* If any window is running on an overridden (non-shared) prompt, the banner flags that the comparison isn't a pure model comparison.
From the banner you can **save the candidate window as a new prompt version**, **add it to a dataset**, or **discard** it.
### Run over dataset
When the session was seeded from a prompt, the verdict banner's **Run over dataset** button takes you to that prompt's [Versions tab](/dashboard/prompts#the-quality-gate), where you run the candidate version over a full dataset and put it through the quality gate — a single Playground run is evidence, not proof.
## Importing input
The left rail also includes an **Import from trace** control: paste a trace ID and Zespan pulls that trace's prompt, provider, model, and config into the session, so you can reproduce a specific production call. If the session was seeded from a prompt with a recent failing generation, a **Replay a failing trace** shortcut does the same thing without typing an ID.
## Saving your work
* **Save as new version** (rail or window footer) creates a new, unlabeled draft prompt version from the shared rail or a specific window's effective prompt. It never deploys anything — you still promote it to a label from the prompt's Versions tab.
* **Add to dataset** (per window, or from the verdict banner) records the window's last run as a dataset item, provided that run came from an imported trace and the session was seeded from a prompt.
* **Share** copies a link that encodes the active window's prompt, provider, model, and config, so a teammate opens the same setup.
* **History** keeps your last 50 runs in this browser (per project) so you can reload a previous prompt/model/config combination.
* **Reset** clears all windows back to a single blank one.
Playground sessions and history are stored in your browser's local storage, scoped to the project. They aren't shared across devices or team members — use **Share** or **Save as new version** to hand off a result to someone else.
# Policies — author and apply guardrail policies from the dashboard
Source: https://docs.zespan.com/dashboard/policies
Author guardrail policies as versioned documents in the dashboard, review the exact change set, backtest against real traffic, and apply — through the same plan/apply pipeline and the same refusals as `zespan policy`.
**Policies is where a guardrail policy comes from. [Guardrails](/dashboard/guardrails) is what it does at runtime.** A policy compiles into the guardrail rules a project enforces; Guardrails is the execution history and live configuration of those rules. The split exists so authoring a change and watching it fire are two different, un-confusable views of the same control.
Whether a policy is written as a YAML file in your repository and applied with [`zespan policy apply`](/cli/policy), or authored here and applied from the dashboard, it reaches your project's guardrails through the exact same computation: the same parser, the same `plan` → `apply` pipeline, the same refusals. There is no second, dashboard-only way to write a guardrail rule from a policy — this page is a second **entrypoint** into the pipeline documented in [Policy as code](/policies/as-code), not a second implementation of it.
Policies is available on the **Pro** plan and above. Guardrails themselves start one tier lower, on **Solo** — see [Plan requirement](/dashboard/guardrails#plan-requirement).
## One writer per policy
Every policy has exactly one owner at a time: **Code** (a file in your repository, applied by CI) or **Dashboard** (authored here). The owner decides who may change or remove it — the other side is refused, not silently overwritten. See [Ownership](#ownership-and-handing-a-policy-over) below; this is the same ownership model [Policy as code](/policies/as-code#ownership-what-code-owns-and-what-the-dashboard-owns) already describes for Guardrails, extended to cover policies authored here instead of hand-built guardrail rules.
## The inventory
The main Policies page lists every policy enforcing (or once enforced) in the selected environment — the same environment switcher Traces, Guardrails, and Evaluations use. One row per policy:
| Column | Meaning |
| ------- | --------------------------------------------------------------------------------------------------------- |
| Policy | The policy's name, with its id or repository path underneath |
| Owner | **Code** or **Dashboard** — see [Ownership](#ownership-and-handing-a-policy-over) |
| Posture | `Dry run`, `Warn`, or `Deny` — the enforcement ladder's current rung, or `—` if nothing is live |
| Rules | How many compiled rules this policy contributes in this environment |
| Tested | Whether a backtest vouches for exactly the text that's live — see [Backtest evidence](#backtest-evidence) |
| Status | Whether what's enforced matches what's authored — see below |
**Status** values:
| Status | Meaning |
| ----------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- |
| In sync | What is enforced matches the applied revision |
| Unapplied changes | A newer revision exists that hasn't been applied to this environment |
| Not applied | This policy enforces nothing in this environment yet |
| Detached | Ownership was handed to the dashboard (see [Detaching](/dashboard/guardrails#detaching)); applying from code will report a conflict |
| Unrecognised | What's deployed doesn't match any revision Zespan holds — this should never happen through normal use, and is surfaced rather than hidden if it does |
**Tested** values:
| Status | Meaning |
| ---------- | ----------------------------------------------- |
| Tested | A backtest exists for exactly this text |
| Stale test | The policy changed after it was last backtested |
| Untested | No backtest has been run against this policy |
A filter box searches by name or id, and an owner toggle narrows to **Code** or **Dashboard**. Click a row to open that policy's [detail page](#reviewing-a-policy).
A policy live in more than one environment is a separate inventory row per environment. Switch environments with the same selector Guardrails uses to see another.
If a project has no policies yet, the inventory explains what a policy is and offers both starting points side by side: **Author a policy** here, or run `zespan policy apply ./policies --env prod` from CI.
## Authoring
From the Policies page, click **New policy**.
The **Name** and **id** fields write straight into the document's `metadata.name` and `metadata.id` as you leave each field — the fields and the YAML never disagree, because one is the source for the other. If the id or name line in the YAML has been hand-edited into something the fields can't unambiguously update (duplicated, removed, or re-indented), authoring refuses to guess and asks you to fix it directly in the editor instead.
`Dry run`, `Warn`, or `Deny`, matching [the enforcement ladder](/policies/as-code#the-enforcement-ladder). Start new policies at dry run.
The YAML editor is the source of truth — full syntax highlighting and inline error squiggles, powered by the same server-side validator `zespan policy validate` uses, so an error shown here is the same error the CLI would report. Alongside it, the **rule builder** inserts a generated rule fragment at the end of the document's `rules:` list for the common types (regex, keyword, PII, toxicity, LLM classifier). It only ever inserts — it never reads the document back into itself, so nothing you've hand-written (comments, key order, an unusual structure) is ever silently rewritten. If it can't find a single unambiguous `rules:` list to insert into, it says so and leaves the fragment for you to paste in yourself.
**Save draft** commits the document's first revision. Nothing is enforced yet — saving only creates the record. Review and apply it from the policy's own page.
Saving is disabled until the YAML validates and both Name and id are filled in. That's deliberate: nothing invalid or unidentified reaches a revision.
## Reviewing a policy
A saved policy's detail page has four tabs:
* **Rules** — how many compiled rules this policy contributes in the current environment, with a link to their runtime view in Guardrails.
* **Source** — the applied revision's YAML, read-only, with a revision picker. A policy authored in git shows its repository path instead of an editor — Zespan holds no document for it to render.
* **Tests** — the current test status in plain language: tested, stale, or untested, and what to do about it.
* **History** — when this policy was last applied in this environment and by whom, with a link to the full [apply history](#apply-history).
When a revision hasn't been applied yet, a **Review changes** button appears in the header.
## Review and apply
Review renders the exact plan `zespan policy plan` would print for the same change — `creates`, `updates`, `adopts`, `deletes`, and any `conflicts` — because it's computed by the same server endpoint. Reviewing needs only `policy:read`, which every role has; **Apply** needs `policy:apply` and is disabled while any conflict is unresolved.
Applying goes through the identical set of refusals `zespan policy apply` has always had. Each one opens a dialog naming what happened and offering the action that resolves it, rather than a generic error:
| Refusal | What it means | What the dialog offers |
| ------------------------- | -------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `ENVIRONMENT_NOT_ALLOWED` | This policy's `appliesTo.environments` doesn't list the environment you're applying to | No override — apply to a listed environment, or add this one to the policy |
| `STALE_PLAN` | Something else changed enforcement since this plan was computed | **Re-plan and review** — recomputes the plan against current state |
| `CONFLICTS` | Part of this change is owned by a policy file in your repository | No override here — [detach](/dashboard/guardrails#detaching) the conflicting policy first, or leave it to code |
| `REMOVAL_NOT_ALLOWED` | This apply would stop enforcing one or more policies | Lists every policy that would be removed; type **REMOVE** to confirm, then **Remove them** |
| `ADOPTION_NOT_ALLOWED` | This apply would take over one or more hand-built guardrails from the Guardrails page | **Take them over** — explicit, one click, no undo dialog beyond this one |
| `UNTESTED_DENY` | This would enforce at deny with no backtest behind it | **Run backtest now** (primary) runs one against the last 7 days right there; **Apply without testing** (secondary) overrides and is audited like any other apply |
There is no `--force` equivalent in the dashboard. Taking a policy away from the other writer is always **Detach** — a deliberate, named, audited action on the Guardrails page — never a checkbox on an apply.
The `UNTESTED_DENY` treatment is the one worth using deliberately: it's a speed bump, not a wall, because someone mid-incident has to be able to ship — but the dialog makes the safe path (see what the backtest would have caught) as fast as the override.
A successful apply lands you on [apply history](#apply-history) at the entry it just created.
## Backtest evidence
The **Tested** badge and the `UNTESTED_DENY` gate read the same fact: whether a `PolicyTest` row exists for this policy's exact current text, keyed by content hash. A test run from the dashboard and one run with `zespan policy test` write to the same table, so whichever surface tested a given revision satisfies the deny gate on both.
The one place the dashboard triggers a backtest today is inside the `UNTESTED_DENY` dialog — **Run backtest now** runs it against the last 7 days of traffic and reports what it would have caught and broken, the same report [Policy testing](/policies/testing) describes for the CLI. To backtest against a different corpus (`issues`, or a specific dataset) before you're at the deny gate, use `zespan policy test --against `.
## Apply history
**Activity** (linked from the Policies page header) lists every apply in the selected environment, newest first:
| Column | Meaning |
| -------- | ------------------------------------------------------------------ |
| When | When the apply ran |
| From | **Dashboard** or **CI**, with `(forced)` if the CLI used `--force` |
| Policies | Every policy id the apply touched |
| Changes | Created / updated / removed counts, as `+N ~N -N` |
Every dashboard apply records the exact revision it deployed, so an apply made here always names a specific, immutable piece of text — not just "whatever the document held at the time."
## Ownership and handing a policy over
The **Owner** badge is the whole mechanism that lets git and the dashboard share a project without fighting over it:
| Badge | Meaning |
| --------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Code | Authored in your repository. Read-only here — applying a dashboard document that claims the same id is refused (`CONFLICTS`) rather than silently overwriting it |
| Dashboard | Authored here. A policy file claiming this id is refused unless the apply explicitly adopts it (`ADOPTION_NOT_ALLOWED`) or the policy has been detached |
To move a policy from code to the dashboard — during an incident, or because ownership is genuinely changing hands — use **Detach** on its guardrail card or detail page in Guardrails, exactly as [Policy as code](/policies/as-code#detaching-a-policy) describes. The policy becomes dashboard-owned immediately; the next `zespan policy apply` reports it as a conflict and refuses to overwrite it without `--force`.
There's no dashboard-side equivalent for taking a policy the other direction — from dashboard to code — beyond what already exists: point a file at the same id and run `zespan policy apply --adopt`, the same adoption flow [Policy as code](/policies/as-code#adopting-a-project-that-already-has-guardrails) describes for hand-built guardrails.
Archiving a policy (from its detail page) doesn't remove what it enforces — it retires the document from the inventory while leaving live rows exactly as they are. To stop enforcing it, apply with removal allowed first, then archive.
## Organisation view
Everything above is one project's inventory. The top-level **Policies** entry in the org admin sidebar (`/[org]/admin/policies`) rolls the same facts up across every project in the organisation, for the question a security lead actually asks: "is this policy deployed everywhere it should be?"
Four cards summarise the current scope:
| Card | Meaning |
| ------------------- | ------------------------------------------------------------------------- |
| Policies | Distinct policy ids in scope |
| Projects covered | Projects with at least one policy in scope, out of every project in scope |
| Occurrences at deny | Occurrences enforcing at `deny`, not just observing |
| Untested or stale | Occurrences with no backtest matching the live text |
Below the cards, one row per **policy id**, not per project — every project carrying that id is grouped underneath it. A row shows how many projects carry the policy, a posture breakdown (dry run / warn / deny), how many occurrences need attention (`out of band`, `detached`, `untested`, or `stale`), and the owner split (**Code** vs **Dashboard**). Expand a row to see every occurrence: which project, which environment, posture, rule count, owner, test status, and sync status, each linking to that project's own policy detail page. If projects use different display names for the same policy id, the row flags it — that disagreement is itself a governance signal, not noise to hide.
The **Environment** selector defaults to **all environments** — not production. Every count and the "present in N of M" figure describe whatever scope is currently selected. Switching to **Production only**, or to one named environment, narrows every number on the page; read the scope before reading the numbers.
**Present in 9 of 12 projects** is coverage, not compliance: it means 9 of the 12 projects measurable in the current scope carry at least one occurrence of that policy id. Under a narrowed scope (Production only, or one named environment), a project with no matching environment at all was never measurable and is excluded from both the numerator and the denominator, rather than counted as absent for a reason that has nothing to do with policy coverage.
This page reports what is deployed. It does not, and cannot, declare a project **non-compliant** — no standard exists yet in Zespan for a project to be measured against; that's a later capability. "Present in 9 of 12 projects" and "Not present in: \" are facts about what's deployed, never a verdict on the projects that don't carry a policy.
A **Recent applies** section below the table lists the most recent applies across the whole organisation, newest first — project, environment, the policy ids touched, the change counts, and which entrypoint ran it (**from CLI** or **from dashboard**). This section itself has no time window or row cap — it is cursor-paginated, newest first, and you can keep paging back through the org's full apply history. The 90-day window and 2,000-row cap belong to `lastAppliedAt` on the rows further up: a policy's row can still show `Not applied` even while this section shows recent activity for the same project, because activity outside that window, or on a different environment, doesn't move `lastAppliedAt` on the row.
## What is not in this release
* **Editing a policy again after it's been saved.** The dashboard supports authoring, reviewing, backtesting, and applying a policy today; changing an already-saved policy's text from here is not yet available.
* **Restoring or diffing a specific past apply from the Activity page.** Both exist as API endpoints today — restore resolves an old apply back to a plan you confirm through the ordinary apply flow — but neither has a dashboard button yet.
* **A cross-environment view on a policy's detail page.** The detail page shows one environment at a time, whichever the switcher has selected; switch environments to see the same policy's posture elsewhere.
## Next steps
The runtime side: execution history, code-managed badges, and Detach.
The authoring loop and ownership model from the repository side.
Every verb and flag, and the same six refusals from the CLI's perspective.
Every field a policy document supports, and the supported YAML subset.
How a backtest is scored, and what the deny gate is protecting against.
Handle a guardrail verdict in your application code.
# Prompts — version, deploy, and gate your prompt library
Source: https://docs.zespan.com/dashboard/prompts
Store, version, label, and deploy prompt templates through a quality gate. Fetch prompts at runtime via the SDK so you can update them without redeploying.
The Prompts page is your prompt version control and release system. Every prompt change creates a new immutable version — previous versions are never overwritten. You promote a version to a label (`production` or `staging`), your running application fetches prompts by label at runtime using the SDK, and — before a risky change ships — you can run a **quality gate** that scores a candidate version against the current production baseline and blocks the release if it regresses.
## The prompt library
The main Prompts page shows all prompts in your project as a list. Each row shows the prompt name, the number of versions, which labels are active, and when it was last updated.
Prompts are grouped by folder. Ungrouped prompts appear first, followed by each folder's prompts — see [Organizing prompts into folders](#organizing-prompts-into-folders) below.
Click any prompt to open its detail view.
## Creating a prompt
On the main Prompts page, click **New prompt**.
Enter a slug-style name for the prompt, e.g. `support-reply` or `product-description-generator`. This name is used by the SDK to fetch the prompt at runtime.
Assign the prompt to an existing folder, or leave it ungrouped. Folders are purely organizational — you can move the prompt into a different folder later without affecting its name or version history.
* **Text** — a single string with `{{variable}}` placeholders for dynamic content
* **Chat** — an array of message objects (`role`, `content`) with placeholder support
Enter your prompt in the editor. Use `{{variable_name}}` syntax for any values you want to substitute at runtime.
Optionally fill in suggested model, temperature, and max\_tokens. These are stored with the prompt (and versioned along with it) for reference but are not enforced by the SDK — you decide whether to use them.
Assign a label, typically `staging`, to this initial version.
Write a short description of what this version contains. Commit messages appear in the version history.
Click **Save**. Version 1 is created and ready to fetch via the SDK.
## Organizing prompts into folders
As your prompt library grows, group related prompts into folders — for example, one folder per feature or per customer-facing surface. Moving a prompt to a folder never changes its name, so anything fetching it via the SDK is unaffected.
From the main Prompts page, click the actions menu next to the prompt you want to organize.
Select **Move to folder** from the menu.
Start typing a folder name — existing folder names autocomplete as you type. Enter a name that doesn't exist yet to create a new folder on the fly.
Click **Move**. The prompt now appears under that folder on the main Prompts page.
| Action | What it does |
| -------------------------- | ------------------------------------------------------------------ |
| Move to folder | Assigns the prompt to an existing or new folder |
| Move to a different folder | Re-assigns the prompt; its name and version history are unchanged |
| Remove from folder | Moves the prompt back to the ungrouped list at the top of the page |
## The prompt detail view
Opening a prompt takes you to a detail view with seven tabs:
| Tab | What it's for |
| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Versions** | Version history, labels, the quality gate, blast radius, and deployment history — the release-management surface for this prompt. |
| **Playground** | A quick side-by-side run of this draft version against whatever currently holds the `production` label, to sanity-check a change before you commit to a full comparison. |
| **Enhance** | An AI rewrite grounded in this version's real production failures — see [Enhance](#enhance) below. |
| **Analytics** | Calls, cost, and error-rate trend charts for this prompt, with version deploys marked on the timeline. |
| **Generations** | The live trace calls linked to this prompt (by version or across all versions), with per-trace correct/wrong/unsure annotation that can seed a dataset. |
| **Configuration** | Temperature, max tokens, top P, tags, and folder — editing any of these creates a new version, since a config change can affect production output as much as a wording change. |
| **Use** | Copy-paste REST, TypeScript SDK, and Python SDK snippets for fetching this prompt by label, plus the CI quality-gate snippet. |
Each version's row on the **Versions** tab also shows call count, error rate, and cost/call for the selected time window, and a **Test** button in the page header jumps you into the full [Playground](/dashboard/playground) seeded with this prompt's text.
## Labels and promotion
Labels are mutable pointers to specific prompt versions. The recommended workflow is:
1. Create a new version — it starts as a draft with no label.
2. Click **Set staging** to test it, or click **Release** to promote it directly to `production`.
3. Your application code fetches prompts by label:
```typescript theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
const prompt = await promptClient.get("support-reply", { label: "production" });
```
The label pointer updates immediately when you promote. Your running application picks up the new version on its next prompt fetch (respecting the 5-minute cache TTL, or immediately if you clear the cache).
Promoting a version to `production` fires an automatic regression check: Zespan compares evaluator scores between the new production version and the version it replaced, and — if any evaluator dropped more than 10 percentage points — creates a `quality_regression` notification. The **Versions** tab also shows this inline as an alert with **Roll back** and **Fix with AI** actions.
## Protected labels
Some labels — such as `production` — can be locked so that only users with **Admin** or **Owner** roles can move them. Configure protected labels in **Settings → Prompts**. This prevents accidental promotion from a Member-level account.
## The quality gate
Before you trust a draft version, run it through the quality gate: it scores a dataset run of the candidate version, compares that score against a scored baseline run (by default, the run tied to whatever currently holds the `production` label), and returns a pass/fail verdict.
The gate checks three things:
| Check | Default threshold |
| -------------------------------------------------------------------------------- | --------------------- |
| Average evaluator score drop vs. baseline | ≤ 3 percentage points |
| Items that regressed more than 10pp | 0 |
| Tool-call accuracy (did the agent pick the right tool, with the right arguments) | ≥ 90% |
To run the gate from the **Versions** tab:
Choose the dataset to score against and the evaluator to use as the judge.
Either click **Run over dataset** to have Zespan execute this version against every item in the dataset for you, or produce the run yourself from your own pipeline/CI (a code snippet is shown if no run is linked yet) and select it as the candidate.
Click **Run gate**. If the candidate run isn't scored yet, Zespan scores it first (this can take a few seconds — the UI polls automatically), then compares it to the baseline.
A pass/fail banner, the three threshold tiles, a per-item score table (regressions sorted first), and a per-tool accuracy breakdown.
**Run over dataset** and gate scoring (when the candidate run isn't already scored) both execute real model calls, so they require an [LLM connection](/platform/llm-connections) configured for your project. If none is configured, you'll see a **Connect a provider** prompt instead of a run.
The exact same comparison logic backs your CI pipeline: link a dataset run from your own agent code, call `POST /v1/prompts/{name}/versions/{version}/gate`, and a prompt edited in the dashboard and one edited in a pull request are held to the identical bar. See the **Use** tab for the linking snippet, or use the [`zespan-gate` CLI](/sdk/cli) to poll the route and turn the verdict into a CI exit code without writing the polling logic yourself.
## Blast radius
The **Versions** tab also shows a blast radius card: which agents (by `agent_name`) called the version currently holding the `production` label in the last 24 hours, and how many calls each made. This tells you who's actually exposed before you promote or roll back — a change that looks safe on paper can still be high-blast-radius if a critical agent depends on it.
## Blast radius before releasing
Separately from the card above, clicking **Release** opens a confirmation dialog that shows a broader impact summary before the promotion goes through: every agent, environment, and — going further than the last-24-hours card — every prompt/model/policy dependency reachable from this prompt across your full dependency graph, plus 30-day call volume and cost. It flags explicitly when production is among the affected environments.
This check is **advisory only** — it never blocks a release, unlike the equivalent check on [deleting an evaluator](/dashboard/evaluations#deleting-an-evaluator), which does block. See [Blast Radius](/dashboard/blast-radius) for the full model: how declared (configured) and observed (actually-called) dependencies differ, how depth/traversal is bounded, and the underlying API.
## Deployment history
Every promotion and rollback is recorded and shown newest-first on the **Versions** tab, including which version moved to which label and whether the quality gate passed for that release.
## Rollback
**Roll back** re-points the `production` label to the version it pointed at immediately before the most recent promotion — it undoes the last release, not an arbitrary older version. Click **Roll back** next to the current production version, or from the regression alert if one is showing.
You can also roll back via ZespanPilot: "Roll back the support-reply prompt."
## Enhance
The **Enhance** tab rewrites a text prompt using its own real production failures, not generic advice. It pulls the version's recent failing generations from your traces, clusters them by root cause with an AI call, and proposes a rewrite that targets the top cause — every claim in the rewrite is backed by real trace IDs you can open.
Pick one or more goals: fix the top failure cause, cut token cost, tighten tone/format, or add safety constraints.
Click **Generate rewrite**. Zespan needs at least 5 failing generations linked to this version to ground a rewrite — with fewer, it tells you honestly rather than fabricating a generic suggestion.
The result shows the failure clusters found, the proposed rewrite, and a projected impact — clearly labeled as an estimate, not a measurement.
Edit the proposed text if you want, then save it as a new (unlabeled) draft version, or save and promote it straight to production.
Enhance is available on the **Pro** plan and above. Unlike Playground, run-over-dataset, and the quality gate's judge scoring, it does **not** require you to configure your own [LLM connection](/platform/llm-connections) — it works out of the box.
## Dependency tracking
A prompt can embed another prompt's content by reference, using an inline `@@@zespanPrompt:name=|version=@@@` (or `|label=