Skip to main content
TrustGate (open source, Apache-2.0) is a purpose-built reverse proxy for LLM and agent traffic. Point any OpenAI-, Anthropic-, or Responses-API client at it and TrustGate normalizes, routes, load-balances, governs, and observes every call — without changing your application code beyond a base URL and two headers. It is built from scratch in Go on top of Fiber, tuned for low latency and high concurrency, and ships as a single static binary, a Docker image, and Kubernetes manifests.

​
Why a gateway

Putting TrustGate between your apps and your model providers gives you one control point for:
  • Multi-provider access — first-class adapters for OpenAI, Anthropic, Azure OpenAI, AWS Bedrock, Google Gemini, Vertex AI, Groq, and Mistral, behind one OpenAI-compatible surface.
  • Smart routing & load balancing — round-robin, weighted, least-connections, random, and semantic strategies, with health checks and fallback chains.
  • Cost & abuse control — request and token rate limiting, request-size guards, and embedding-based semantic caching to cut spend on repeated prompts.
  • Multi-tenancy & auth — per-gateway consumers authenticated by API key, OAuth2, IDP JWT, or mTLS, with policies scoped globally or per consumer.
  • Runtime security — attach TrustGuard to inspect prompts and responses inline for jailbreaks, PII, toxicity, and tool abuse.
  • Observability — built-in Prometheus metrics and rich per-request telemetry streamed to Kafka (and optionally TrustLens).
  • Agent tooling — a dedicated MCP plane exposes MCP servers and tools to agents with full OAuth2 support.

​
The building blocks

You configure TrustGate through its Admin API (or the console), then send traffic to the proxy. Six objects make up a gateway:
ObjectWhat it is
GatewayThe top-level tenant, addressed by a slug. Owns everything below.
RegistryAn upstream backend — an LLM provider endpoint or an MCP server.
ConsumerThe calling application’s identity. Owns routing and credentials, addressed by a slug in the URL.
AuthA credential (API key, OAuth2, IDP, mTLS) that authenticates as a consumer.
PolicyA governance rule that runs at request/response stages — rate limiting, caching, CORS, and more.
RoleRouting config selected from IDP token claims, for identity-based routing.

​
How a request flows

client ──▶ Proxy :8081  /{consumer_slug}/v1/chat/completions
              │  X-AG-Gateway-Slug, X-AG-API-Key
              ├─ resolve gateway + consumer + policies
              ├─ apply policies (rate limit, cache, CORS, …)
              ├─ load-balance across the consumer's registries (+ fallback)
              ├─ forward to the provider adapter (stream when supported)
              └─ emit telemetry → Kafka
A client never names a provider URL or key — it names a model, and the gateway resolves the registry, applies policies, and forwards. See Architecture for the full lifecycle.

​
Where to go next

Quickstart

Zero to a forwarded completion in six API calls.

Architecture

Planes, the request lifecycle, and infrastructure.

Core concepts

Gateways, registries, consumers, auth, policies, roles.

Admin API

The full open-source REST API.