Issues — recurring failures grouped automatically

The Issues page turns N separate failing traces that are really the same underlying problem into one row: a recurring Issue with an occurrence count, a first/last-seen timestamp, and a link to a real sample trace. In the sidebar, Issues is a row in the Fix group, and Errors is its second tab. See Finding your way around the dashboard.

​

How clustering works

A background worker re-runs the same deterministic verdict classifier used on every trace detail page (the one that decides whether a trace is healthy, degraded, or failed) against recent candidate traces, then groups matches by (verdict level, primary operation, error code, signal). Traces that land in the same group become occurrences of one Issue.

​

What counts as a candidate

Clustering used to consider only traces where a span errored. It now also considers traces that failed without erroring:

• an evaluator scored the trace below its configured threshold — or above it, for evaluators like toxicity and pii_leakage where a high score is the bad one
• an evaluator returned a non-passing verdict outright, which is how categorical rubrics (safe / borderline / unsafe) are judged
• a behavioural signal fired — see below
• a tool call was retried, or ran far slower than its siblings

This is the class of failure where nothing throws: the agent returns a confident wrong answer, or takes a wrong action, and no log line says so. Because it’s the same classifier your trace detail page already shows you — not a second, fuzzier system guessing from raw scores — an Issue’s grouping key means exactly what it says: these traces failed the same way, for the same reason, on the same operation.

​

The Issues list

The range control offers 24h, 7d, 30d, and 90d and opens on 7d. When issues need attention, the verdict band sits above the strip of key figures, so the headline is the first thing you read. Each row shows: Filter the list with the chips above it: search, Verdict, Type (only the types that have issues in the selected range are offered, each with its count) and, when attribution is on, Layer. The Agent column appears only when at least one issue has an agent recorded, so you never see a column of dashes. Row actions stay out of the way until you need them: Generate fix (shown only for an issue with no suggestion yet, it opens the issue so you can generate one), Resolve, and Dismiss appear when you hover a row, select it, or tab to it with the keyboard. Click View details to open the Issue detail page for the full sample-trace list and the remediation suggestion feature. View sample trace jumps straight to one real occurrence in the normal flame graph view, so you can investigate the actual failure without leaving the list.

​

Kinds of issue

An issue has a Type (a column in the list, a small tag on the issue page) that says what kind of problem it is, so you can tell a provider outage from an agent that quietly did the wrong thing. Zespan works the kind out from the issue itself; nothing is configured and no extra data is stored, so it is the same on every project. When the issue itself does not say which kind it is, it gets no tag rather than a guess: an issue keyed on a custom evaluator (a custom check could be about quality, speed or cost), on the built-in cost budget evaluator, or an older degraded issue with no signal.

​

Signals

When a trace fails without erroring, the Cause column names the signal that identified it rather than an error code. Two issues on the same operation with different signals stay separate. A toxicity failure and a user_retry observation on chat are different problems and get their own rows, their own occurrence counts, and their own regression datasets.

​

Where it started

When an issue’s cause has been proven from its spans, the issue page shows where it started under the root cause: a chip such as Caused by Retrieval or Likely Tool, how sure Zespan is, why, the role that owns a fix for that layer (for example Fix owner: Data and search), and the line Failed at: Tool step. A weaker claim reads Possibly …, muted and always with its evidence. When two layers fit equally it reads Either … or … with the step that would settle it. See Stack layers for the layers, the tiers and when a claim is shown. Under the chip, Was it Retrieval? Yes / No, it was… records your answer so Zespan can measure how often it is right, see Confirm or correct the layer. On the list, a Layer filter keeps the issues that started in one layer, and a line above the table says how many issues it covers, for example 9 analysed issues, 6 in Retrieval. Each analysed issue’s row carries a small tag with its layer next to the cause.

​

Resolved vs. dismissed

Both actions remove an issue from the open Issues list — the difference is what they communicate, the same distinction Sentry-style issue trackers draw between the two: Neither is permanent: if the aggregator detects a fresh occurrence of the same cluster (same verdict level, operation, error code, and signal) after an issue was resolved or dismissed, the issue reopens and reappears in the open Issues list. Resolving an issue you haven’t actually fixed just means it comes back sooner.

​

Issue detail page

Clicking into an Issue (/{orgSlug}/{projectId}/issues/{issueId}) shows:

• The verdict level, current status, occurrence count, and first and last seen. Times read as relative (“4h ago”); hover one for the exact local time. A link to the associated prompt appears when this issue’s traces share one. The agent shows when the traces recorded one, and nothing shows when they did not
• Escalate to incident, Mark resolved and Dismiss actions (same semantics as the list — see above)
• The Root cause card, then the Suggested remediation card when it has something to add, then the Sample traces table (up to 10 representative traces from the cluster, each linking to its full trace)

​

Root cause, Fix and Prevent

The Root cause card is computed from the issue’s own spans with no AI, so it is there with AI processing off and before any suggestion exists. It has:

• A badge: Verified from spans (proven) or Auto-detected (matched by a rule from the recorded error, not proven)
• One paragraph saying what happened. Times inside it read as relative times, never raw timestamps
• An evidence line that reconciles the numbers on the page, for example: “12 occurrences. Found 2 matching failures in 2 retained traces; 3 are needed to confirm, so this cause is not confirmed yet.” Occurrences are how often the pattern happened; the traces are the ones still retained. A cause is confirmed only once enough matching failures show it; when it is not confirmed for another reason, the line quotes that reason. For an agent loop it says how many of the retained traces checked showed the loop
• Fix and Prevent, each a numbered list of steps. Fix has Open sample trace. A Prevent step that sets up an alert on a metric Zespan alerts on (error rate, tool error rate, latency, cost, guardrail block rate, blocked, truncated or empty-response rate) has Add alert, which opens the new alert dialog on the Alerts page with that metric and the step as its name, for you to review and save. A step whose subject has no alert metric (token counts, stream finish signals, evaluator pass rates) has no button. A step that adds a guardrail has Create guardrail, which opens the guardrail builder. On the shared demo these two ask you to sign up instead

The fix and prevention steps are listed once, here. Nothing repeats them below.

​

The Suggested remediation card

The card appears only when it adds something beyond the Root cause card: A card that would only say “pending” for something that will not come is not shown. On the shared demo, Zespan never generates a remediation for a visitor; an issue shows one only when a saved suggestion was generated beforehand, labelled “Generated by ZespanPilot on from N sample traces”.

​

Generate suggestion (remediation)

Click Generate suggested remediation on the Issue detail page to have Zespan investigate the pattern for you: it runs the same root cause analysis used on individual traces against up to 5 representative sample traces from the cluster, then writes a short suggestion — the likely underlying cause and one concrete next step, reasoned over the recurring pattern rather than a single occurrence. Every statement in that suggestion is checked against the record it refers to before you see it, and each one shows what it rests on — the trace, prompt, policy or metric behind it — linking through where that record has a page. The suggestion is cached per issue, so regenerating it (e.g. after revisiting the page) is instant and doesn’t re-run the analysis or incur additional cost. If none of the sampled traces can be analyzed — for example, they’ve aged out of trace retention — you’ll see an error instead of a suggestion; try again once the issue has recurred and produced fresher samples. If the analysis couldn’t run because the AI provider was unavailable, the error says so instead — try again later. Sample traces whose analysis was unavailable are never counted toward the pattern. For an issue raised by an evaluator (for example toxicity or user retries), the analysis is given that evaluator’s failing verdict and the content it judged, and its explanation must cite them. If an explanation can’t be checked against that failure, you see what was detected instead of a guess.

​

Automatic remediation suggestions

Once an Issue has recurred three or more times, Zespan generates a remediation suggestion for it on its own — you don’t have to ask. The suggestion appears on the Issue detail page labelled with the time it was generated, so opening a recurring failure usually means reading the analysis rather than requesting it. The analysis reasons across the Issue’s sample traces rather than about one of them, and says which situation it found: You can still press Generate suggestion yourself on an Issue that hasn’t reached three occurrences, or ask ZespanPilot directly: “investigate the rate limit issue on chat.completions.create.”

​

Prevent

When an issue has a cause proven from its spans, its detail page can show a Prevent section (off by default; see Policies). It always starts with the written advice for that cause, whatever your plan. Below it:

• No draft yet. Zespan can build a policy from the agent’s own recorded traces and test it. Create draft needs the Pro plan and permission to draft.
• Proven on your traffic. For example “Caught 12 of 12 similar traces, 0 false hits, over 14 days.” Review opens the drafted policy on the review screen. Apply… explains that applying starts at dryrun, where nothing is blocked, and sends you to the same review and apply screen. Nothing is applied from the issue page.
• Not proven. The plain reason, for example that the draft would also have stopped too many normal traces. No policy is offered and the advice above still applies.
• Confirm before a destructive call. For a finding where a tool you declared destructive ran with no confirmation step, the draft reads Waiting for confirmation data until your SDK sends confirmations (with code to copy and a link to Confirmations), then Would have stopped N of M calls in shadow (dryrun logs only) and how many of those followed a plain request, where a missing confirmation is a judgement call. This version can only warn; it cannot block a call, and it checks that your app recorded a confirmation, not that the end user agreed. See Policies.
• Below the Pro plan. You see the advice, a line that drafting is on the Pro plan and above, and the kinds of prevention that exist for that cause. No draft is built or shown.

Only causes with a draft template can be drafted; for the rest the section shows the advice and says a draft is not available yet.

​

Issues feed regression tests

Once an issue has recurred 3 or more times, it’s automatically captured as a test case in a Production Failures dataset — turning your own incident history into a regression suite your CI can replay against a candidate change. See Regression testing from production failures.

​

Next steps

• Traces — the underlying root-cause analysis that powers remediation suggestions
• ZespanPilot — the AI copilot that writes remediation suggestions
• Guardrails — if an issue’s error code traces back to a guardrail violation, promote that violation into a permanent rule
• Datasets — replay recurring issues as regression tests
• Incidents — for anomaly-driven correlation across metrics, rather than verdict-based trace clustering
• Stack layers — what the layer chip means and when it is shown
• Policies — where a drafted policy shows its proof, and how it is applied
• Instrumentation — see what Zespan can observe, set who gets each layer, and confirm or correct a layer
• Detectors — choose which detector packs run (Off, Preview or On) and confirm what your tools do