Skip to main content
TrustGate (open source, Apache-2.0) is a purpose-built reverse proxy for LLM and agent traffic. Point any OpenAI-, Anthropic-, or Responses-API client at it and TrustGate normalizes, routes, load-balances, governs, and observes every call — without changing your application code beyond its base URL and consumer credentials. SaaS gateways are reached by subdomain; Private/Hybrid also accepts X-AG-Gateway-Slug. It is built from scratch in Go on top of Fiber, tuned for low latency and high concurrency, and ships as a single static binary, a Docker image, and Kubernetes manifests.

​
Why a gateway

Putting TrustGate between your apps and your model providers gives you one control point for:
  • Multi-provider access — first-class adapters for OpenAI, Anthropic, Azure OpenAI, AWS Bedrock, Google Gemini, Vertex AI, Groq, Mistral, and DeepSeek (plus any OpenAI-compatible endpoint), behind one OpenAI-compatible surface.
  • Smart routing & load balancing — round-robin, weighted, least-connections, random, and semantic strategies, with health checks and fallback chains.
  • Cost & abuse control — request rate limiting, token/dollar LLM budgets, request-size guards, and embedding-based semantic caching to cut spend on repeated prompts.
  • Tool & prompt governance — inject operator-authored tools, rate-limit tool calls, inject and version system prompts; restrict which models a consumer may reach via model_policies.
  • Guardrails — built-in TrustGuard, OpenAI Moderation, Azure Content Safety, AWS Bedrock, and Regex Replace policies to inspect or rewrite prompts and responses inline.
  • Multi-tenancy & auth — per-gateway consumers authenticated by API key, OAuth2, OIDC, or mTLS (see Auth). LLM consumers use API key, OAuth2, or OIDC; MCP consumers use OAuth2.
  • Observability — rich per-request telemetry (model, tokens, cost, latency breakdown, routing attempts, policy chain) exported through OpenTelemetry (OTLP) and forwarded to TrustLens, where it powers detection alerts.
  • Agent tooling — a dedicated MCP plane exposes MCP servers and tools to agents with full OAuth2 support.

​
The building blocks

You configure TrustGate through its Admin API (or the console), then send traffic to the proxy. Six objects make up a gateway:

​
How a request flows

A client never names a provider URL or key — it names a model, and the gateway resolves the registry, applies policies, and forwards. See Architecture for the full lifecycle.

​
Where to go next

Quickstart

Create a gateway, connect a provider, and send your first request from the console.

Architecture

Planes, the request lifecycle, and infrastructure.

Core concepts

Gateways, registries, consumers, auth, policies, roles.

Admin API

The full open-source REST API.