Dataset runs — running your pipeline over a dataset with the SDK

A dataset run executes your prompt or agent over every item in a dataset and links each produced trace back to the item that generated it. Zespan never calls your model for you here — your own code (a script, a CI job, a scheduled batch) does the work; the SDK just records which trace answered which dataset item. Once every item is linked, you score the run with an evaluator from the dashboard, compare it against a baseline run, and — for prompts — feed it into the quality gate before promoting a version to production. DatasetsClient is the SDK interface for this workflow, available in both the TypeScript and Python SDKs with the same method names and behavior (Python uses snake_case). See Datasets for the dashboard-side concepts (creating datasets, versioning, scoring, comparing runs) that this page assumes.

There’s no separate “start” call — createRun/create_run is idempotent. Calling it again with the same run name (for example, the next time a scheduled job starts) re-attaches to the existing run instead of creating a duplicate, so linking is always additive.

Getting a `DatasetsClient`

import { zespan } from "@zespan/sdk";

zespan.init({ apiKey: process.env.ZESPAN_API_KEY! });

// Via the client (recommended)
const datasets = zespan.getClient().datasets;

// Or import and construct directly
import { DatasetsClient, getZespanClient } from "@zespan/sdk";
const datasets = new DatasetsClient(getZespanClient());

import os
import zespan
from zespan import get_client

zespan.init(api_key=os.environ["ZESPAN_API_KEY"])

# The Python SDK only exposes DatasetsClient through the initialized client —
# there is no standalone `DatasetsClient` import (unlike PromptClient).
datasets = get_client().datasets

Step 1 — Fetch the dataset’s items

getItems(datasetName) (TypeScript) / get_items(dataset_name) (Python) resolves the dataset by name and returns its items. Each item carries the input your pipeline should run, and — if the dataset has one — an expectedOutput for comparison-style evaluators.

const items = await datasets.getItems("support-eval-set");
// items: { id: string; datasetId: string; input: unknown; expectedOutput?: unknown; metadata?: Record<string, unknown> }[]

items = datasets.get_items("support-eval-set")
# items: list[dict] — each with "id", "datasetId", "input", and optionally
# "expectedOutput" / "metadata" (the SDK returns the raw API response, so
# keys are camelCase even in Python)

string

required

The dataset’s name (not its id). Resolved to an id internally and cached in-memory for the lifetime of the DatasetsClient instance. Named dataset_name in Python.

Step 2 — Create (or re-attach to) the run

createRun(datasetName, runName, options?) (TypeScript) / create_run(dataset_name, run_name, description=None) (Python) creates a named run under the dataset, or returns the existing one if a run with that name already exists. The returned handle is what you call link() on.

const run = await datasets.createRun("support-eval-set", "gpt-4o-v2", {
  description: "gpt-4o with the v2 support-reply prompt",
});
// run: DatasetRunHandle — run.run is { id, datasetId, name, description }

run = datasets.create_run(
    "support-eval-set",
    "gpt-4o-v2",
    description="gpt-4o with the v2 support-reply prompt",
)
# run: DatasetRunHandle — run.run is {"id", "datasetId", "name", "description", ...}

string

required

The dataset to run against. Named dataset_name in Python.

string

required

A name for this run, unique within the dataset (e.g. a model + prompt version combination). Reusing a name re-attaches to the existing run instead of erroring. Named run_name in Python.

string

Optional human-readable description. In TypeScript this is a key on the third options object ({ description }); in Python it’s a plain keyword argument on create_run — the two SDKs shape this parameter differently.

Step 3 — Run your pipeline per item and link the trace

For each item, call your own LLM or agent (traced the same way it always is — through a wrapped provider client, withAgent, etc.) and then call link(datasetItemId, traceId, observationId?) on the run handle with the trace ID that call produced. Pass observationId as well if you want to point at one span within the trace rather than the trace as a whole. The link call itself doesn’t need to know how the trace was produced — only its ID. The most direct way to know that ID ahead of time is to establish the trace context yourself with withZespanTrace (TypeScript) or with_zespan_context (Python) before calling your pipeline, rather than trying to read the ID back off a manually created span.

import { withZespanTrace } from "@zespan/sdk";
import { randomUUID } from "node:crypto";

for (const item of items) {
  const traceId = randomUUID();

  // Any wrapped LLM/agent call made inside this callback is tagged with traceId
  await withZespanTrace(() => mySupportAgent(item.input), { traceId });

  await run.link(item.id, traceId);
}

from zespan import with_zespan_context

for item in items:
    with with_zespan_context() as ctx:
        my_support_agent(item["input"])  # any patched client call here shares ctx["trace_id"]
        trace_id = ctx["trace_id"]

    run.link(item["id"], trace_id)

string

required

The id of the dataset item this trace answers. Named dataset_item_id in Python.

string

required

The trace ID your pipeline call produced. Named trace_id in Python.

string

Optional — points the link at a single span within the trace instead of the trace as a whole. Named observation_id in Python.

Re-linking the same datasetItemId on the same run updates the trace pointer rather than erroring — safe to call again if a step in your job fails partway through and retries.

Complete example

This example fetches a dataset’s items, creates (or re-attaches to) a run, runs a wrapped OpenAI call per item, and links every result.

import { zespan, withZespanTrace } from "@zespan/sdk";
import { randomUUID } from "node:crypto";
import OpenAI from "openai";

zespan.init({ apiKey: process.env.ZESPAN_API_KEY! });
const openai = zespan.wrapOpenAI(new OpenAI());
const datasets = zespan.getClient().datasets;

async function mySupportAgent(input: unknown): Promise<string> {
  const response = await openai.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: String(input) }],
  });
  return response.choices[0].message.content ?? "";
}

async function runEvalJob() {
  // 1. Fetch the dataset's items
  const items = await datasets.getItems("support-eval-set");

  // 2. Create (or re-attach to) a named run — safe to call every time this job starts
  const run = await datasets.createRun("support-eval-set", "gpt-4o-v2", {
    description: "gpt-4o with the v2 support-reply prompt",
  });

  // 3. Run your own pipeline per item, then link the result back to the run
  for (const item of items) {
    const traceId = randomUUID();
    await withZespanTrace(() => mySupportAgent(item.input), { traceId });
    await run.link(item.id, traceId);
  }

  console.log(`Linked ${items.length} items to run "${run.run.name}"`);
}

runEvalJob();

import os
import openai
import zespan
from zespan import get_client, with_zespan_context

zespan.init(api_key=os.environ["ZESPAN_API_KEY"])
zespan.patch_openai()
datasets = get_client().datasets
openai_client = openai.OpenAI()

def my_support_agent(input_text) -> str:
    response = openai_client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": str(input_text)}],
    )
    return response.choices[0].message.content or ""

def run_eval_job():
    # 1. Fetch the dataset's items
    items = datasets.get_items("support-eval-set")

    # 2. Create (or re-attach to) a named run — safe to call every time this job starts
    run = datasets.create_run(
        "support-eval-set",
        "gpt-4o-v2",
        description="gpt-4o with the v2 support-reply prompt",
    )

    # 3. Run your own pipeline per item, then link the result back to the run
    for item in items:
        with with_zespan_context() as ctx:
            my_support_agent(item["input"])
            trace_id = ctx["trace_id"]

        run.link(item["id"], trace_id)

    print(f"Linked {len(items)} items to run \"{run.run['name']}\"")

run_eval_job()

Scoring and gating the run

Linking is the last thing the SDK does — scoring happens in the dashboard, against the linked traces:

Open the run

Open the dataset in Datasets, click the Runs tab, and find the run your job created (it’s created the first time your code calls createRun/create_run).

Score it

Pick an evaluator and click Score. Zespan looks up each linked trace and scores it, showing a per-item score and the run’s overall average.

Compare or gate

Compare two scored runs side by side from the Runs tab, or — for a prompt version — run this same dataset/run through the quality gate to get a pass/fail verdict against a baseline before promoting.

Scoring a run calls the evaluator’s LLM judge, which requires a project LLM connection. Without one, scoring fails with “No LLM connection configured — add one in Settings → LLM Connections.” Connect a provider key under LLM Connections first.

No-code alternative

If you don’t want to wire up the SDK loop above, the dashboard can execute the dataset run for you instead of your own code: the Run over dataset button, available from a prompt’s Versions tab and from a seeded Playground session, runs the candidate prompt version against every item in the dataset and links the results automatically. This still requires an LLM connection, since Zespan is making the model calls on your behalf in that path — unlike the SDK-driven workflow on this page, where your own code (and your own inference spend) produces the traces.

Next steps

Datasets — creating datasets, versioning, scoring, and comparing runs
Prompt management — fetch and compile the prompt version your pipeline is testing
LLM Connections — required to score a run or use the no-code Run over dataset button

Tracing & Observability

Evaluations

Datasets & Experiments

Guardrails

Prompt Management

Alerts & Incidents

Cost & Analytics

ZespanPilot (AI Copilot)

Dataset runs — running your pipeline over a dataset with the SDK

Getting a `DatasetsClient`

Step 1 — Fetch the dataset’s items

Step 2 — Create (or re-attach to) the run

Step 3 — Run your pipeline per item and link the trace

Complete example

Scoring and gating the run

No-code alternative

Next steps

​Getting a DatasetsClient

​Step 1 — Fetch the dataset’s items

​Step 2 — Create (or re-attach to) the run

​Step 3 — Run your pipeline per item and link the trace

​Complete example

​Scoring and gating the run

​No-code alternative

​Next steps

Getting a `DatasetsClient`

Step 1 — Fetch the dataset’s items

Step 2 — Create (or re-attach to) the run

Step 3 — Run your pipeline per item and link the trace

Complete example

Scoring and gating the run

No-code alternative

Next steps