Changelog

​

Platform updates — Evaluation trust & CI gating

Released: July 2026

• Vector-DB tracing for Pinecone, Chroma, Weaviate, and Qdrant — calling a vector-DB client directly (no LangChain/LlamaIndex retriever in between) is now auto-traced by zespan.autopatch(), the same one-liner that already covers LLM providers: reads (query/search/near_vector) emit a retriever span with rag_contexts, writes (upsert/add/insert) emit an embedding span with metadata.vector_count — no code changes to your DB client calls. A new recordVectorSearch()/record_vector_search() manual helper covers pgvector and other raw-SQL vector stores, which have no client library to auto-patch — see Pinecone, Chroma, Weaviate, Qdrant, and Manual spans
• Vector-DB read spans now include metadata.top_k and metadata.filter — every retriever span from the auto-patched Pinecone/Chroma/Qdrant/Weaviate clients now also carries the requested result count (metadata.top_k — always recorded for Pinecone, whose top_k is required; recorded when the caller passes one for Chroma/Qdrant/Weaviate, whose equivalent option is optional) and the query’s filter/where clause (metadata.filter, redacted and stringified, only when storePrompts/store_prompts is on). Both fields are only observable nested under metadata — the ingest pipeline persists the metadata object verbatim but drops other unrecognized top-level fields — see each provider’s integration page for the exact source field per client
• Corrected built-in evaluators — Latency SLA, Cost Budget, Loop Detection, and Error Recovery Rate now compute a real deterministic check against the trace’s own latency_ms/cost_usd/operation/status data instead of silently falling back to a generic LLM quality-judge score — see Performance & agent evaluators
• Categorical/boolean scores now render correctly — a categorical or boolean evaluator’s result shows as the judge’s chosen label, or true/false, everywhere it appears (trace detail, per-trace score chip, evaluator tables) instead of a meaningless percentage bar — see Reading evaluation results
• Unified evaluation score sources — scores attached from the SDK via span.setEvalScore() now appear alongside server-side judge scores in the Evaluations dashboard’s KPI tiles and metric trend list, not only in the raw per-trace view — see Attaching scores from the SDK
• POST /evaluation-runs now actually scores — creating an evaluation run through the API now enqueues real scoring and the run list shows its aggregated score, instead of the run sitting at pending forever
• zespan-gate CLI — a new dependency-free @zespan/cli package wraps the prompt quality gate for CI: poll for a pending run, fail fast with a linking snippet if nothing’s scored yet, and exit non-zero on a failed or misconfigured gate — see CI quality gate
• Evaluation & dataset MCP tools — the hosted Zespan MCP server now exposes datasets, evaluators, evaluation runs, and trace verdicts to any connected coding agent (Claude Desktop, Cursor, Claude Code), alongside the existing trace/metrics and prompt-management tools — see Zespan MCP server
• Issue, session, retroactive-eval, and cost MCP tools — the hosted Zespan MCP server now covers 35 tools, adding open Issues (list, detail with sample traces, resolve/dismiss, and cached markdown remediation suggestions), sessions (recent-session list, full rollup + turn-by-turn transcript, session-level eval scores), retroactive evaluation runs (create, list, status), and cost attribution by agent/tool/model/user/operation. Write tools enforce the same role permissions and plan limits as the dashboard — a personal API key can only do what its owner’s role allows — and remediation stays advice-only: Zespan never opens a PR or touches your source. See Zespan MCP server
• HTTP Targets & HTTP-endpoint dataset runs — register an externally-hosted agent endpoint you don’t control or can’t instrument with the SDK (a deployed Bedrock/Glean agent, or any HTTP-reachable chatbot API), then run a dataset against it directly from “Run over dataset” — Zespan POSTs your hydrated request template to the endpoint (with retry, SSRF protection, and W3C traceparent propagation) and records the response as a tagged trace, no LLM connection required — see HTTP Targets and Running against an HTTP endpoint
• Wider SSRF blocking for HTTP Targets — the address-range check applied to a registered HTTP Target URL (and re-applied to the hydrated URL before every call) now also rejects RFC 6598 carrier-grade NAT space 100.64.0.0/10 — which includes Tailscale’s default range — plus multicast and reserved space from 224.0.0.0 up. The full list of rejected ranges is now documented; there is no override, so an agent reachable only over a private or Tailscale address has to be exposed publicly or instrumented with the SDK instead — see Security: URL validation
• Retroactive evaluation trigger UI — score historical traces without waiting for online eval to have been configured, from a new Retroactive Runs panel on the Evaluations page: pick any evaluator, a time range, and optional operation/model filters, then watch per-run progress live — see Retroactive evaluation
• Annotation Queues — a new dashboard section for routing a filtered set of traces to a human reviewer: filter by time range (and, via the API, operation/model/session/verdict), work through pending items with a trace input/output review panel, and submit a pass/fail verdict, optional label, score, and notes. A categorical (label-only) annotation always requires an explicit Pass/Fail choice — there’s no way to submit one without picking a verdict. Annotations are written into the same evaluation_scores data as automated judge scores, tagged human_annotated, so they appear in every existing trend and cost-quality view alongside judge scores — see Annotation Queues
• Session-level evaluation — a new scope: "session" option on evaluator templates judges an entire conversation’s transcript instead of one trace, so multi-turn quality problems (contradictions, lost context, topic drift) can be caught even when no single call looks wrong. “Session complete” is detected by quiescence — 30 minutes of no new activity by default, checked on a 10-minute scan — since the SDK has no explicit “close this session” call. Results appear as a Conversation: Pass/Fail · N evaluators badge on the session detail page, kept visually distinct from per-trace scores — see Session-level evaluation
• Session-level scores no longer skew the trace-level Evaluations views — a scope: "session" score judges a whole conversation and belongs on the session detail page, but it was also being counted by the Evaluations page’s metric trends, traces/spans-evaluated tiles, and Retrieval tab. Those four aggregations are now trace-scoped, so a session evaluator can’t drag a per-trace metric average down, add a phantom trace to the evaluated count, or rank session among your worst retrieval operations — see Session-level evaluation
• Sessions time-range filter — the Sessions list now defaults to the last 7 days instead of scanning full project history on every load, with a date-range picker to widen or narrow the window — see Filtering sessions
• Issue detail page, resolve action, and remediation suggestions — Issues now open into a detail page with their full sample-trace list; a new Resolve action sits alongside Dismiss with Sentry-style semantics (resolved = fixed, dismissed = stop showing me this pattern — either can reopen on a fresh occurrence); and a Generate suggestion button runs root-cause analysis across representative sample traces and asks ZespanPilot for a cached markdown remediation suggestion — likely cause and one concrete next step, never an automatic PR or repo change — see Issues
• Multi-turn conversation scenarios are now fully functional — Simulations’ Conversation (multi-turn) target, where a simulated user and your target agent alternate turns, now runs end-to-end. Conversation scenarios also gained structured persona configuration (name, goal, expertise level, and tone compose the simulated user’s system prompt, with the existing free-text Simulated User Prompt as a fallback), and a run’s detail view now shows the full turn-by-turn Conversation Transcript with per-turn latency and cost plus a total conversation cost from real per-model pricing — see Persona configuration for conversation scenarios and Reading run results
• Conversation scenarios now honor their inner target’s prompt template — the Prompt template you configure for a Conversation (multi-turn) scenario’s inner target was previously discarded, so the target agent ran with no template and no system prompt. It’s now applied to every turn, with {{input}} resolving to that turn’s message, and a configured system prompt is rendered against the run’s variables just like a single-turn prompt target — see Scenario target types
• Plan retention is now enforced on reads — traces, evaluation scores, guardrail events, and incident events older than your plan’s retention window (14 days on Free through 1 year on Scale) are excluded from every API response and dashboard view. Widening a date filter past the window no longer returns older data — see Data retention
• OTLP trace ingest reaches parity with the native SDK path — /v1/traces now enforces the same monthly event quota and 1 MB body limit as /v1/ingest, plus its own 512-span-per-request cap (matching the OpenTelemetry SDK/Collector’s own default max_export_batch_size, rather than reusing /v1/ingest’s 100-span cap, which was sized for the native SDK’s batching and would have silently dropped most spans from a default-configured OTel exporter), with the overflow reported through OTLP’s partialSuccess.rejectedSpans — see OpenTelemetry integration
• OTLP spans now honor organization custom pricing — spans ingested over OTLP are priced through the same resolver as SDK spans (custom ModelPrice rows first, built-in table as fallback) instead of always using built-in rates. Orgs with negotiated or self-hosted model pricing will see corrected costs on OTLP traces
• /v1/metrics and /v1/logs now return 501 instead of a misleading 202 — these OTLP signals were never stored; the endpoints previously accepted and discarded the payload while reporting success. They now fail honestly and non-retryably. Point only your trace exporter at Zespan — see OpenTelemetry integration
• Per-trace sampling in both SDKs — a sampleRate/sample_rate below 1.0 now keeps or drops a whole trace, decided from the trace id, instead of rolling per event. Sampled traces are complete: no orphaned child spans, no wrong span counts, and the same verdict on every service a distributed trace passes through. This also fixes a bug where a trace arriving via OpenTelemetry (or from another service in a distributed trace) always sampled in at 100%, ignoring sampleRate entirely — if you use OTel with sampleRate below 1.0, upgrading will reduce your ingest volume down to your configured rate, which is the fix working as intended, not new data loss — see sampleRate
• Corrected span counts across the dashboard — agent summaries, the AI Hub top-agents panel, top-cost and slow-trace tables, the ops expensive-traces panel, and session summaries now count distinct spans instead of raw rows, so their span and error totals match the trace detail view
• Multi-trace root-cause investigation — ZespanPilot can now investigate a recurring Issue across its sample traces instead of one trace at a time, and reports whether the occurrences share a single root cause or are several distinct failure modes colliding on the same verdict/operation/error-code cluster key — so an Issue that is quietly two problems says so — see Automatic remediation suggestions
• Automatic remediation suggestions — an Issue that recurs three or more times gets a remediation suggestion generated for it without anyone asking, ready on the Issue detail page with the time it was produced. Still text only: no pull request, no prompt edit, no config change — see Automatic remediation suggestions
• Evaluator authoring from chat — describe what you want scored (“check whether replies stay on topic”) and ZespanPilot maps it to the right built-in metric and creates the evaluator, instead of pointing you at the dashboard — see What ZespanPilot can do
• ZespanPilot conversations survive navigation — leaving the Pilot page and coming back resumes the same conversation with its transcript instead of silently starting a new one, and reopening a conversation from History now shows its messages — see Opening ZespanPilot
• Export a dataset run comparison — the two-run comparison view on a dataset’s Runs tab now has Export CSV and Export HTML buttons, so a report can be saved, attached to a PR, or shared without needing dashboard access — see Comparing two runs

---

​

Platform updates — Agentic reliability & RAG evaluation

Released: July 2026

• Enforcement-linked trace forensics — guardrail hits now appear inline in the trace flame graph as their own spans, colored by outcome, instead of a separate log. Promote any violation directly into a permanent guardrail rule with pre-filled tool/field/value — see Promoting a violation to a policy
• Human approval gates — client.awaitApproval() (TypeScript) / client.await_approval() (Python) blocks execution until an admin approves or rejects the call from a new Approvals inbox — a real human-in-the-loop primitive for your own agents’ high-risk tool calls
• Near-miss capture & suggested policy rules — numeric guardrail checks that evaluate close to their threshold without firing are now logged; a background worker clusters recurring near-misses into suggested rules you can promote or dismiss
• Verdict-based issue clustering — a new Issues page automatically groups recurring failed/degraded traces into a single Issue using the same deterministic verdict classifier shown on every trace, with occurrence counts and one-click promotion to a guardrail rule
• Retroactive evaluations — score historical traces against an evaluator you didn’t have configured at the time, from a new panel on the Costs page
• Cost-Quality Frontier — a new chart plots every model used on an operation by real cost vs. real quality, span-level joined so multi-model traces aren’t misattributed — see Cost-Quality Frontier
• Regression testing from production failures — recurring Issues that hit 3+ occurrences are auto-captured into a “Production Failures” dataset your own CI can replay, with verdict-comparison scoring and an optional quality-gate signal — see Regression testing from production failures
• Agent-to-agent trust ledger (Phase 1: intra-org) — a compliance badge (pass rate, violation count, last violation) now appears on delegation lines in the trace detail view and on the Agent Registry, pulled from anywhere in your org rather than only the current project
• RAG quality evaluators — four new reference-free evaluator templates (RAG Faithfulness, Context Relevance, Context Utilization, Retrieval Hit Rate) score the retrieval step of a RAG pipeline directly, plus a fix so the existing Groundedness evaluator actually receives retrieved context instead of judging blind — see Evaluating RAG pipelines
• RAG analytics — a new Retrieval tab on the Evaluations page trends RAG evaluator scores over time and ranks your worst-performing retrieval operations
• Redesigned trace detail — every trace now opens into a plain-English health verdict, three evaluation lenses (final response / trajectory / per step), and a resizable span navigator + inspector, plus Flow (hierarchical/chronological node graph) and Flame (chronological/icicle) visualizations with full-screen, a Retrieval panel showing retrieved RAG chunks, an A2A task-lifecycle panel, a Session conversation view, and per-span deep-link permalinks — see Traces
• Bedrock & Vertex LLM connections — bring-your-own-key now covers Amazon Bedrock (static access keys or STS assume-role, no long-lived secret stored) and Google Vertex AI (service-account JSON), usable by Playground, Evaluations, and Simulations just like the direct-provider connections — see LLM Connections

---

​

SDK updates — July 2026

Released: July 2026 · TypeScript (@zespan/sdk) and Python (zespan) New:

• RAG / retrieval tracing — recordRetrieval() / record_retrieval() records a retriever span with the retrieved documents in one call; span.recordDocuments() / record_documents() and a documents argument on span.end() attach chunks to a span you’re timing. Accepts plain strings, objects, or framework nodes (LangChain Document, LlamaIndex NodeWithScore). Chunk text follows your storePrompts and redaction settings. The LangChain and LlamaIndex integrations now capture retrieved document content automatically. See Recording retrieved documents

Bug fixes:

• Google wrapper no longer crashes generate_content (Python) — a TypeError on the token-usage fields (None where a number was expected) took down every call after the model had already responded; fixed
• Google embedding cost is now computed — embedding calls that report only a total token count (e.g. gemini-embedding-2) previously showed $0; cost is now calculated from the correct input-token count in both SDKs
• Manual span kind always recorded (TypeScript) — startSpan({ span_kind }) previously dropped the kind unless the span was created inside an agent context; it’s now always emitted, matching Python

---

​

Platform updates — July 2026

Released: July 2026

• Prompt folders — organize prompts into folders, with a move-to-folder action and folder autocomplete
• Prompt webhooks — fire a webhook on version created, label assigned, or version deleted, delivered to Slack or a signed, retrying HTTPS endpoint, with a test-delivery button
• Dataset experiment runs — link your own pipeline’s results to a named run against a dataset via new SDK methods (getItems/get_items, createRun/create_run, run.link()), score runs with an evaluator, and compare two runs side by side
• MCP prompt tools — the hosted MCP server now exposes prompt management (list, get, create, update tags, set label) as tools an AI assistant can call directly
• Chat prompt message placeholders — declare a placeholder slot in a chat prompt and fill it with a caller-supplied message array at compile time (e.g. conversation history)
• SDK prompt-fetch resilience — PromptClient.get() now serves a stale cached value or a caller-supplied fallback instead of throwing when the API is unreachable
• Custom evaluation templates — author your own LLM-judge rubrics (numeric or categorical scoring), pin a specific judge model per template, and dry-run a template against a real trace before deploying it
• Auto-evaluator sampling and filters — scope continuous evaluation to a sample rate and filters by model, operation, or status instead of judging every matching trace

---

​

SDK v1.0.1 — TypeScript

Released: June 2026 Bug fixes:

• Fixed API key prefix validation warning (now expects zsp_ prefix)
• Minor internal reliability improvements

---

​

SDK v1.0.0

Released: June 2026 Initial public release of the Zespan SDK (TypeScript and Python).

---

​

Middleware v1.0.0

Released: June 2026

• zespan-autogen — observability middleware for Microsoft AutoGen (agentchat + legacy pyautogen)
• zespan-crewai — observability listener for CrewAI agents
• zespan-fastapi — FastAPI middleware for automatic request/response tracing
• zespan-flask — Flask middleware for automatic request/response tracing