Skip to main content
LLM Connections are separate from your Zespan API key. An API key authenticates the SDK sending traces to Zespan. An LLM Connection is a provider credential you supply so Zespan can call an LLM on your behalf, billed to you.

​
What it is

Some Zespan features need to call an LLM to do their job — running a prompt in the Playground, scoring an evaluation with an LLM judge, or executing a simulation scenario. These features are bring-your-own-key (BYOK): you connect a provider credential in Project Settings, and Zespan uses it to run those calls. You pay your provider directly for the inference — Zespan doesn’t mark it up or meter it. Credentials are encrypted at rest with AES-256-GCM and are write-only — once saved, the raw key is never returned by the API or shown in the UI again. Every connection in the list only ever displays a masked placeholder for the key.
ZespanPilot and Zespan’s built-in AI features (cost optimization, anomaly detection, root-cause analysis, natural-language queries, overview summaries) don’t need a connection — they work out of the box. LLM Connections only affect the BYOK features listed below.

​
Where to connect

Go to Project Settings → LLM Connections.

​
Connection types

Connect a key directly from OpenAI, Anthropic, or Google. Give the connection a name and paste the provider’s API key. Use this when you already have a standing account with one of these providers and want Zespan to call it directly.
Bedrock and Vertex don’t authenticate with a single API key, so they take structured credentials instead. Everything is encrypted at rest, and no secret field is ever returned to the browser. For Bedrock, prefer Assume role so Zespan never stores long-lived AWS keys.

​
Default connection and multiple connections

The first connection you add for a project is automatically set as its default. You can change the default at any time from the connections list — each connection has a Set default action, and only one connection per project can be default. When a run needs credentials and doesn’t specify a connection explicitly, Zespan resolves them in this order:
  1. The connection you explicitly picked for that run (if any)
  2. The project’s default connection
  3. If no default is set, the most recently created connection
If a project has two or more connections, a compact LLM connection picker appears on the run surface (Playground window, quality gate, run-over-dataset, simulations, eval dry-run) so you can choose which one runs — and pays — for that execution. With zero or exactly one connection, the picker stays hidden: with one connection there’s nothing to choose, so it’s used automatically and silently.

​
Features that require a connection

The following surfaces are fail-closed — without at least one LLM connection configured for the project, they show a “Connect an LLM provider” state instead of running:
  • Playground
  • Evaluations — the LLM-judge dry-run, auto-eval scoring, and Run Eval
  • Simulations
  • Run-over-dataset
If you delete or run out of a connection mid-use, the next run attempt returns a clear error rather than falling back to any shared account.
Prompt Enhance is a Pro feature and does not need an LLM connection — it works out of the box. ZespanPilot and Zespan’s other built-in AI features are likewise exempt — see the callout above.

​
Testing a connection

Each connection in the list has a Test button. For key-based connections (OpenAI, Anthropic, Google, gateway, OpenRouter) it sends a small, cheap authenticated request to the provider — for example, listing available models — to confirm the credential is valid and reachable, without running a real completion. For Bedrock and Vertex there’s no provider-wide models-list to hit with just credentials, and which models are enabled varies by account and region — so the test runs a one-token generation against the model you name. That’s the truest check: it exercises the real auth path (assume-role / access keys / service-account) and confirms the model is enabled where you’re pointing. If you test without naming a model, Zespan confirms the credentials were stored but tells you plainly that it didn’t run a live check. Either way the result (success, or an error) is shown inline, and no secret field is ever echoed back.

​
Deleting a connection

Deleting a connection removes it immediately — any run relying on it will no longer resolve credentials. If the connection you delete was the project’s default, Zespan automatically promotes the most recently created remaining connection to default (if any are left).

​
Self-hosting

Self-hosted deployments must set LLM_CONNECTION_SECRET — a 32-byte hex string used to encrypt connection keys at rest with AES-256-GCM. If unset, it falls back to SSO_SECRET. See Environment variables and Self-hosting.

​
Next steps