> ## Documentation Index
> Fetch the complete documentation index at: https://docs.zespan.com/llms.txt
> Use this file to discover all available pages before exploring further.

# LLM Connections (BYOK)

> Connect your own LLM provider keys per project to run Playground, Evaluations, and Simulations on your own inference spend.

<Note>
  LLM Connections are separate from your Zespan **API key**. An API key authenticates the SDK sending traces to Zespan. An LLM Connection is a provider credential *you* supply so Zespan can call an LLM on your behalf, billed to you.
</Note>

## What it is

Some Zespan features need to call an LLM to do their job — running a prompt in the Playground, scoring an evaluation with an LLM judge, or executing a simulation scenario. These features are **bring-your-own-key (BYOK)**: you connect a provider credential in Project Settings, and Zespan uses it to run those calls. You pay your provider directly for the inference — Zespan doesn't mark it up or meter it.

Credentials are encrypted at rest with AES-256-GCM and are **write-only** — once saved, the raw key is never returned by the API or shown in the UI again. Every connection in the list only ever displays a masked placeholder for the key.

<Note>
  **ZespanPilot and Zespan's built-in AI features (cost optimization, anomaly detection, root-cause analysis, natural-language queries, overview summaries) don't need a connection** — they work out of the box. LLM Connections only affect the BYOK features listed below.
</Note>

## Where to connect

Go to **Project Settings → LLM Connections**.

## Connection types

<Tabs>
  <Tab title="Direct provider key">
    Connect a key directly from **OpenAI**, **Anthropic**, or **Google**. Give the connection a name and paste the provider's API key. Use this when you already have a standing account with one of these providers and want Zespan to call it directly.
  </Tab>

  <Tab title="OpenAI-compatible gateway">
    Connect any gateway that exposes an OpenAI-compatible API — a base URL plus a key. This covers proxies like **LiteLLM**, **Portkey**, **Cloudflare AI Gateway**, and **Vercel AI Gateway**. Use this when you route model calls through a gateway for centralized key management, caching, or multi-provider routing, and want Zespan's runs to go through the same gateway.
  </Tab>

  <Tab title="OpenRouter (OAuth)">
    Click **Connect OpenRouter** to start a one-click OAuth flow — no key to copy or paste. Zespan requests a project-scoped key from OpenRouter on your behalf and stores it as a connection automatically. Use this when you want access to OpenRouter's model catalog without manually generating and pasting a key.
  </Tab>

  <Tab title="Amazon Bedrock">
    Run Claude, Llama, Titan, and other Bedrock-hosted models with your AWS account. Under the **Cloud provider** tab, choose **Amazon Bedrock** and a region, then pick how Zespan authenticates:

    * **Access keys** — paste an AWS access key ID and secret access key (plus an optional session token for temporary credentials).
    * **Assume role** — paste an IAM **role ARN** and Zespan assumes it via STS at call time. **No long-lived secret is stored.** The role must trust Zespan's AWS identity and allow `bedrock:InvokeModel`. This is the recommended path for organizations that don't allow standing access keys.

    The model you use is the Bedrock model ID, e.g. `anthropic.claude-3-5-sonnet-20241022-v2:0`.
  </Tab>

  <Tab title="Google Vertex AI">
    Run Gemini (and Vertex-hosted Claude) with your GCP project. Under the **Cloud provider** tab, choose **Google Vertex AI** and provide:

    * **Project ID** — the GCP project that hosts Vertex AI.
    * **Location** — the Vertex region, e.g. `us-central1`.
    * **Service account JSON** — paste the full service-account key file. It's encrypted at rest and never shown again.

    The model you use is the Vertex model name, e.g. `gemini-2.5-flash`.
  </Tab>
</Tabs>

<Note>
  Bedrock and Vertex don't authenticate with a single API key, so they take structured credentials instead. Everything is encrypted at rest, and no secret field is ever returned to the browser. For Bedrock, prefer **Assume role** so Zespan never stores long-lived AWS keys.
</Note>

## Default connection and multiple connections

The **first** connection you add for a project is automatically set as its default. You can change the default at any time from the connections list — each connection has a **Set default** action, and only one connection per project can be default.

When a run needs credentials and doesn't specify a connection explicitly, Zespan resolves them in this order:

1. The connection you explicitly picked for that run (if any)
2. The project's **default** connection
3. If no default is set, the most recently created connection

If a project has **two or more** connections, a compact **LLM connection** picker appears on the run surface (Playground window, quality gate, run-over-dataset, simulations, eval dry-run) so you can choose which one runs — and pays — for that execution. With zero or exactly one connection, the picker stays hidden: with one connection there's nothing to choose, so it's used automatically and silently.

## Features that require a connection

The following surfaces are **fail-closed** — without at least one LLM connection configured for the project, they show a "Connect an LLM provider" state instead of running:

* **Playground**
* **Evaluations** — the LLM-judge dry-run, auto-eval scoring, and Run Eval
* **Simulations**
* **Run-over-dataset**

If you delete or run out of a connection mid-use, the next run attempt returns a clear error rather than falling back to any shared account.

<Note>
  **Prompt Enhance** is a Pro feature and does **not** need an LLM connection — it works out of the box. ZespanPilot and Zespan's other built-in AI features are likewise exempt — see the callout above.
</Note>

## Testing a connection

Each connection in the list has a **Test** button. For key-based connections (OpenAI, Anthropic, Google, gateway, OpenRouter) it sends a small, cheap authenticated request to the provider — for example, listing available models — to confirm the credential is valid and reachable, without running a real completion.

For **Bedrock** and **Vertex** there's no provider-wide models-list to hit with just credentials, and which models are enabled varies by account and region — so the test runs a one-token generation against the model you name. That's the truest check: it exercises the real auth path (assume-role / access keys / service-account) *and* confirms the model is enabled where you're pointing. If you test without naming a model, Zespan confirms the credentials were stored but tells you plainly that it didn't run a live check.

Either way the result (success, or an error) is shown inline, and no secret field is ever echoed back.

## Deleting a connection

Deleting a connection removes it immediately — any run relying on it will no longer resolve credentials. If the connection you delete was the project's default, Zespan automatically promotes the most recently created remaining connection to default (if any are left).

## Self-hosting

Self-hosted deployments must set `LLM_CONNECTION_SECRET` — a 32-byte hex string used to encrypt connection keys at rest with AES-256-GCM. If unset, it falls back to `SSO_SECRET`. See [Environment variables](/platform/environment-variables) and [Self-hosting](/platform/self-hosting).

```bash theme={"theme":{"light":"github-light","dark":"one-dark-pro"}}
# Generate a suitable value:
openssl rand -hex 32
```

## Next steps

* [Security](/platform/security) — encryption and data handling across Zespan
* [Environment variables](/platform/environment-variables) — full self-hosting reference