Skip to main content
TrustGate (open source, Apache-2.0) is a purpose-built reverse proxy for LLM and agent traffic. Point any OpenAI-, Anthropic-, or Responses-API client at it and TrustGate normalizes, routes, load-balances, governs, and observes every call — without changing your application code beyond a base URL and two headers. It is built from scratch in Go on top of Fiber, tuned for low latency and high concurrency, and ships as a single static binary, a Docker image, and Kubernetes manifests.

​
Why a gateway

Putting TrustGate between your apps and your model providers gives you one control point for:
  • Multi-provider access — first-class adapters for OpenAI, Anthropic, Azure OpenAI, AWS Bedrock, Google Gemini, Vertex AI, Groq, Mistral, and DeepSeek (plus any OpenAI-compatible endpoint), behind one OpenAI-compatible surface.
  • Smart routing & load balancing — round-robin, weighted, least-connections, random, and semantic strategies, with health checks and fallback chains.
  • Cost & abuse control — request rate limiting, token/dollar LLM budgets, per-model cost caps, request-size guards, and embedding-based semantic caching to cut spend on repeated prompts.
  • Tool & prompt governance — allow-list, validate, and reshape the tools an agent can call; inject and version system prompts; restrict which models a consumer may reach.
  • Guardrails — built-in TrustGuard, OpenAI Moderation, Azure Content Safety, and AWS Bedrock guardrail policies to inspect prompts and responses inline.
  • Multi-tenancy & auth — per-gateway consumers authenticated by API key, OAuth2, OIDC, or mTLS, with policies scoped globally or per consumer.
  • Observability — rich per-request telemetry (model, tokens, cost, latency breakdown, routing attempts, policy chain) streamed to Kafka by default, with an opt-in per-gateway OpenTelemetry (OTLP) exporter and TrustLens forwarding, where it powers detection alerts.
  • Agent tooling — a dedicated MCP plane exposes MCP servers and tools to agents with full OAuth2 support.

​
The building blocks

You configure TrustGate through its Admin API (or the console), then send traffic to the proxy. Six objects make up a gateway:
ObjectWhat it is
GatewayThe top-level tenant, addressed by a slug. Owns everything below.
RegistryAn upstream backend — an LLM provider endpoint or an MCP server.
ConsumerThe calling application’s identity. Owns routing and credentials, addressed by a slug in the URL.
AuthA credential (API key, OAuth2, OIDC, mTLS) that authenticates as a consumer.
PolicyA governance rule that runs at request/response stages — rate limiting, budgets, caching, tool governance, guardrails, CORS, and more.
RoleRouting config selected from OIDC token claims, for identity-based routing.

​
How a request flows

client ──▶ Proxy :8081  /{consumer_slug}/v1/chat/completions
              │  X-AG-Gateway-Slug, X-AG-API-Key
              ├─ resolve gateway + consumer + policies
              ├─ apply policies (rate limit, cache, CORS, …)
              ├─ load-balance across the consumer's registries (+ fallback)
              ├─ forward to the provider adapter (stream when supported)
              └─ emit telemetry → Kafka
A client never names a provider URL or key — it names a model, and the gateway resolves the registry, applies policies, and forwards. See Architecture for the full lifecycle.

​
Where to go next

Quickstart

Zero to a forwarded completion in six API calls.

Architecture

Planes, the request lifecycle, and infrastructure.

Core concepts

Gateways, registries, consumers, auth, policies, roles.

Admin API

The full open-source REST API.