Skip to main content
TrustGate captures a structured metrics event for every proxied request — model, tokens, cost, latency breakdown, routing attempts, and the policy chain — and publishes it through an asynchronous worker so measurement never blocks request handling.
There is no Prometheus /metrics scrape endpoint. TrustGate is a data-plane proxy: it emits a rich event per request to your sinks (Kafka by default, OpenTelemetry/OTLP optionally) rather than exposing scrape-style counters. Point your analytics or a Kafka consumer at the stream — see Telemetry for the event shape.

​
How it works

  1. A proxy middleware opens a request trace at the start of each request and records timings, routing attempts, and per-policy decisions as the request flows.
  2. When the response finishes (including after a fully-streamed SSE response), the trace is handed to an in-memory worker queue — never inline on the response path.
  3. Worker goroutines drain the queue, build the event, and publish it to the configured exporters (and, for playground requests, a short-lived trace store).
Because the build-and-publish step runs on background workers, a slow or unavailable sink (e.g. Kafka) never adds latency to or fails a user request.

​
Configuration

VariableDefaultMeaning
TELEMETRY_ENABLEDtrueMaster switch — records and emits the per-request event.
METRICS_QUEUE_SIZE1000Capacity of the in-memory event queue.
METRICS_WORKER_COUNT1Worker goroutines draining the queue.
METRICS_FLUSH_INTERVAL5sHow often workers flush buffered events.
Trace depth is tuned with TELEMETRY_ENABLE_REQUEST_TRACES and TELEMETRY_ENABLE_PLUGIN_TRACES (both default true) — see Configuration.

​
What the event carries

Each event is a single record with identity, request, response, usage, cost, a latency breakdown (total / provider / policies / routing / gateway), per-registry routing attempts, and the policy chain. The exact schema and how to consume it are documented in Telemetry. For interactive, single-request inspection (rather than a stream), the Admin plane also exposes the playground trace for a given trace_id.