Skip to main content
Available via: native OpenTelemetry callback (any language calling the proxy). This is not a Zespan SDK wrapper — see Send traces from any OTel-instrumented app for how the underlying endpoint works.
This page is for LiteLLM Proxy — the self-hosted gateway server that sits in front of your LLM providers and traces every call that passes through it, regardless of which app or language made the call. If your app calls the litellm Python module or client directly instead, use patch_litellm()/wrapLiteLLM() — that’s a client-side SDK wrapper, a different integration point than this page.
LiteLLM Proxy has a built-in otel logging callback — enable it in your proxy config and every request that flows through the gateway (from any calling app, in any language) is traced, with zero changes to the apps calling it.

Setup

In your proxy’s config.yaml:
Set the exporter destination as environment variables before starting the proxy:
LiteLLM’s otlp_http exporter sends protobuf. Zespan accepts Protobuf. Spans LiteLLM sent before the October 2026 release may be missing. See OTel integration.

What gets captured

Every request routed through the proxy — across every provider and model configured in model_list — arrives as spans. Zespan stores the model, provider, token counts, finish reason and the calling user (llm.user) from the attributes LiteLLM sets. Details to know:
  • One model call per request. LiteLLM also exports a span for the proxy request itself (Received Proxy Server Request), guardrail spans and, when message logging is on, a raw request span. Zespan stores them as gateway spans of the same trace; they are not counted as model calls. A failed LiteLLM request span is stored as a failed attempt (a gateway span with its provider) when a successful call of the same proxy request (same trace and same parent span) arrives in the same export; a request that failed for good stays a failed model call, also when your app’s traceparent puts several requests into one trace. LiteLLM exports no retry or fallback marker, so attempts are recognised only this way.
  • Cost. Zespan prices each call from the model and token counts. LiteLLM’s own figure (gen_ai.cost.total_cost) is kept as evidence next to the span and is not used as the price, unless the model has no Zespan price and GATEWAY_COST_FALLBACK=on is set on a self-hosted deployment.
  • Prompts and completions. LiteLLM sends prompt and completion text by default unless you set turn_off_message_logging. Whether Zespan keeps that text follows your project’s prompt storage setting, and redaction applies to what is stored. With message logging off, LiteLLM omits the operation name; Zespan still stores the span as a model call.
  • Identity metadata is dropped. LiteLLM attaches the user’s e-mail, the requester’s IP address, the key hash and the team to its spans (with LITELLM_OTEL_V2 under litellm.api_key.*, litellm.team.* and litellm.metadata.*), and its management endpoint spans carry request and response bodies such as a newly generated key. Zespan removes these before storing (see Security). The end-user id you pass as user stays.
  • Streaming (llm.is_streaming) is recorded. LiteLLM 1.106.2’s exporter sends no cache-hit attribute, so a LiteLLM cache hit is priced like any other call.
Attribute names come from LiteLLM’s published documentation and can change between versions. Use the Gateway and OTLP data card to see what actually arrives.
This is the common pattern for teams that standardize LLM access behind a self-hosted gateway rather than calling providers directly from each service.

Verify

Send one request through the proxy, then open Settings → Instrumentation → Gateway and OTLP data and press Refresh. LiteLLM should appear with tokens and span kind near 100%. See Instrumentation.

Using the SDK too

If an app also uses the Zespan SDK for the same call, Zespan can count its tokens and cost once when merging is on (see the linked section). LiteLLM continues an inbound traceparent and puts the provider’s response id on its span, so both the trace link and the response id can match. Turn on gatewayCorrelation for your proxy’s host to nest the proxy’s span under the SDK span. Without it the response id is used.

Next steps