Playground execution requires an LLM connection configured for your project — it always runs against your own connected provider, never a shared/managed key. If none is configured, you’ll see a Connect a provider banner in place of the run controls.
Opening the Playground
Navigate to Playground in the left sidebar for a blank session, or click Test in the header of a prompt’s detail view to open it pre-filled with that prompt’s text, version, and — if there’s a recent failing generation — a one-click Replay a failing trace shortcut.The shared prompt rail
The left-hand rail holds the prompt — the single versioned artifact every comparison window runs by default. Edit it once and every window updates:- Single mode — one template with
{{variable}}placeholders. - Chat mode — a system message plus a list of seed turns (few-shot examples, or the exact state a real conversation was in when it broke).
- Tools & structured output (collapsed by default) — shared tool definitions (JSON) and an optional structured-output schema, used by every window.
Per-window override
Any window can break away from the shared prompt: click Override on a window to give it its own prompt text (a CUSTOM PROMPT badge marks it), or Reset to shared to go back to inheriting the rail. This is how you A/B two different prompt variants — not just two models — inside the same comparison.Comparison windows
Each window is an independent run: pick a model, run it, read the result. You can have up to 3 windows open at once — beyond that stops being a comparison and starts being noise. Use the dashed Add a window tile to add one (it duplicates the active window’s connection/model/toggles, but never its results), and duplicate or remove windows from each card’s header.Connection-aware model selection
Each window has its own connection/model picker. The model list is derived from the LLM connection you pick — you can’t accidentally select a Google model while only an OpenAI connection is configured. Switching a window’s connection resets its model to a sensible default for that connection’s provider (or keeps your current pick if it’s still valid).Single vs. Chat mode
A page-level toggle switches every window between:- Single — one request in, one output out. Shows output text, token counts, cost, latency, guardrail results, and (if the prompt calls tools) an interactive tool-call panel where you can submit synthetic tool results and continue the run.
- Chat — a live multi-turn conversation per window, seeded from the rail’s system prompt and seed turns. Non-baseline windows (window 2 and 3) show a divergence banner the moment their tool-call pattern differs from window 1’s corresponding turn — turn-by-turn, based on real transcript comparison, not a guess.
Toggles
Each window has independent Streaming and Guardrails toggles (guardrails apply your project’s configured policies to the run and show pass/fail banners), plus the temperature/max-tokens summary from the shared config.Reading results
Non-baseline windows (2 and 3) show latency and cost as a percentage delta against window 1, so you can see at a glance whether a model is faster/cheaper or slower/pricier — never a fabricated number when the baseline hasn’t run yet. You can also run any configured evaluator against a window’s output directly from its Run Eval control.The verdict banner
Once window 1 and the last window both have results, a verdict banner summarizes the comparison:- If a guardrail blocked one window’s output, the banner leads with that — a blocked output is a failure, not a saving, and isn’t treated as a valid comparison.
- Otherwise it reports whether the outputs matched (single mode only), the latency/cost deltas, and — when the session was seeded from a prompt with known call volume — a projected daily cost delta at that volume.
- If any window is running on an overridden (non-shared) prompt, the banner flags that the comparison isn’t a pure model comparison.
Run over dataset
When the session was seeded from a prompt, the verdict banner’s Run over dataset button takes you to that prompt’s Versions tab, where you run the candidate version over a full dataset and put it through the quality gate — a single Playground run is evidence, not proof.Importing input
The left rail also includes an Import from trace control: paste a trace ID and Zespan pulls that trace’s prompt, provider, model, and config into the session, so you can reproduce a specific production call. If the session was seeded from a prompt with a recent failing generation, a Replay a failing trace shortcut does the same thing without typing an ID.Saving your work
- Save as new version (rail or window footer) creates a new, unlabeled draft prompt version from the shared rail or a specific window’s effective prompt. It never deploys anything — you still promote it to a label from the prompt’s Versions tab.
- Add to dataset (per window, or from the verdict banner) records the window’s last run as a dataset item, provided that run came from an imported trace and the session was seeded from a prompt.
- Share copies a link that encodes the active window’s prompt, provider, model, and config, so a teammate opens the same setup.
- History keeps your last 50 runs in this browser (per project) so you can reload a previous prompt/model/config combination.
- Reset clears all windows back to a single blank one.