In Hybrid you run the data plane; NeuralTrust runs the control plane and the inference plane in your tenant’s region, EU or US. Raw prompts and responses stay in your PostgreSQL. Only configuration requests, metadata and the text the ML detectors score reach NeuralTrust, and your cluster opens every one of those connections.
This page is the whole install, written to be followed from top to bottom. Read Architecture once to know what you are installing, tick off Before you start, then run Install step by step. To install one product on its own, follow the TrustGate data plane or TrustGuard data plane guide instead; the prerequisites and network rules on this page apply to both.Before you start
Have each of these ready. The install fails, or half-works, without them.- Your tenant’s region, EU or US — see Choose your region
- A cluster with an ingress controller, sized per Prerequisites, and either ntctl or Helm 3.8+
- A base domain you can point at the cluster ingress, for example
platform.example.com - PostgreSQL and Redis the cluster can reach — see Check your datastores first
- The registry key NeuralTrust sent when your account was provisioned
- Egress to your region’s hosts, and the inbound rule agreed with your network team — see Network
- Console access in your region, to create a private TrustGate and, if you run it, a private TrustGuard
Architecture
How the planes connect
Config sync and DataBridge are outbound only. Your cluster dials them;
nothing at NeuralTrust initiates those, so you open egress and never ingress
for them.
The console is the one exception. The Dataplane URL you register there is a
URL NeuralTrust calls, so your published LLM and MCP entry points must accept
inbound HTTPS from a single NeuralTrust source IP. Agree that rule with your
network team early — it is the requirement security reviews most often push
back on. The address is in Network.
TrustGate and TrustGuard never talk to SaaS directly for metadata. They emit
OTLP to a collector co-located with DataAgent inside your cluster, and that
collector is what egresses. Defaults come from environment variables,
TELEMETRY_EXPORTERS_METADATA=otlp and
TELEMETRY_EXPORTERS_RAW=postgres, so metadata leaves the cluster and raw
payloads stay in your PostgreSQL.
What runs
Products are selected with
global.products.trustgate,
global.products.trustguard, and global.products.dataPlane. All default to
off and at least one must be true.
DataAgent is worth understanding because it is unusual: it exposes no inbound
service. It dials out to DataBridge and holds the stream open, and queries
arrive over that stream. There is nothing to expose and nothing to allow
inbound. It also runs single-replica by design — two would register duplicate
streams. Each enabled product needs its own enrolled DataAgent; only a
red-teaming-only install (global.products.dataPlane alone) runs without one.
TrustGate admin, the console, and analytics are not deployed in Hybrid —
they run on SaaS. There is no in-cluster ClickHouse.
Choose your region
Your tenant lives in one NeuralTrust region, and a Hybrid data plane dials that region for everything: config sync, DataBridge, telemetry and — through the firewall JWT the same console issues — the inference plane. One value selects it:- It moves every control-plane connection together. Config sync, DataBridge and telemetry all derive from it, so there are no per-endpoint hosts to set.
- Credentials are region-scoped. A config-sync token or DataAgent enrollment token issued by the EU control plane does not authenticate against the US one. Create the console objects in step 2 in the same region you set here.
- Anything other than
euorusis rejected at render time, so a typo cannot quietly send a data plane to the wrong region.
saasRegion: "us" under
global in the values file you write in step 4, and allow the US
hosts in Network.
Prerequisites
- A Kubernetes cluster with an ingress controller, and either ntctl or Helm 3.8+
- 2 nodes of 8 vCPU / 16 GiB with all products and the NeuralTrust-hosted Firewall, or 3 of 8 vCPU / 32 GiB with the Firewall in your cluster, as the minimum of an autoscaled node pool; GPU and single-product options are in Capacity
- A base domain you can point at the cluster ingress, for example
platform.example.com - The network rules in Network below
- A reachable PostgreSQL and Redis
- The registry key NeuralTrust sends you when your account is provisioned — Container images
Check your datastores first
The two most common first-install failures are a datastore that is not actually reachable and one that is reachable but silently wrong. Redis is required. Both gateways validateREDIS_HOST at boot and refuse to
start without it, even though a Hybrid data plane does not use PostgreSQL for
its own configuration. TrustGate uses it for rate-limit and budget counters on
the request path.
PostgreSQL is where your raw payloads live. Prompts and responses are
written there and never leave your cluster.
The chart deploys both in-cluster by default so a proof of concept can start
without provisioning anything. Production should set deploy: false and point
at managed instances outside the cluster — see
Managed stores.
An unreachable store fails loudly. The pattern to watch for is a component
whose default quietly points at localhost: it comes up healthy and looks fine
until you use the feature that needs it. Set every host explicitly.
Network
Allow TCP 443 from your cluster egress — and from any NAT or proxy in front of it — to the hosts of your region only. Prefer hostname rules where your firewall supports DNS-based allowlists; the IPs are for static ACLs. If DNS resolves differently in your region, trust DNS and ask NeuralTrust support to refresh the list.- EU (saasRegion: eu)
- US (saasRegion: us)
trustguard-configsync host, and the Firewall host is needed by TrustGuard,
and by TrustGate only when you give it a firewall JWT for smart routing.
Product metadata OTLP does not leave the pods directly — the co-located egress
collector forwards to the ingest endpoint for your region, so that is the host to
list if your policy enumerates every external destination. Opt-in
operational telemetry
uses the same collector and host, so turning it on adds no rule.
Allow the Firewall host your TrustGuard’s firewall JWT names, which matches
your control plane’s region. It is not needed when the Firewall runs in your
cluster (global.firewall.deploy: true). The hosted Firewall can also be
reached over Google Cloud Private Service Connect; ask NeuralTrust for the
service attachment.
Also allow outbound HTTPS to your container registry (or mirror), your LLM
upstreams, and your PostgreSQL and Redis.
Inbound
Hybrid is not egress-only. Your published TrustGate LLM and MCP entry points must accept inbound HTTPS from:
Scope the rule to your network boundary or ingress for those two hosts. Nothing needs to
reach TrustGuard, Firewall, the data-plane API, or DataAgent from outside the
cluster.
Two of these carry long-lived streams. If an egress proxy or middlebox reaps
idle connections, config-sync and DataBridge drop and reconnect on that
interval — check its idle timeout. A TLS-intercepting proxy breaks certificate
verification unless its CA reaches the client: put it in the bundle you point
dataagent.databridge.tlsCa, <product>.configSync.tlsCa and
global.clickstack.egress.tlsCaSecretName at, since each of those replaces the
system roots rather than adding to them.
Install
- ntctl
- Helm
Install ntctl, sign in to the
console of your tenant’s region, and run the install:Leave out
ntctl install asks only for what it cannot work out, and each question has a
default: the base domain, the platform (detected from the cluster), the
products, operational telemetry, where
images come from (the registry key file NeuralTrust sent you, or your own
registry), your PostgreSQL and Redis (or in-cluster ones), and, with TrustGuard,
whether the Firewall runs in your cluster or at NeuralTrust.Signed in, it creates a private TrustGate and TrustGuard in the console, or
reuses the ones you name with --trustgate-instance / --trustguard-instance,
and fetches their tokens itself. That replaces the console steps of the Helm
path. It also sets global.saasRegion from the region you signed in to. Without
ntctl login, it asks for each token with input hidden.Then it:- Checks the cluster: storage, ingress, capacity on the nodes each workload can run on, every Secret the install reads, and that the control plane resolves. A check that fails stops the install with a fix; see Preflight.
- Shows the cluster and the choices it is about to apply, and asks for confirmation.
- Creates the Secrets, installs the chart, waits until every workload is
Ready, and prints the DNS records to create. If the install does not become
ready, it prints what
ntctl doctorfinds wrong. - Writes the values it used, without credentials, to
neuraltrust-values.yaml. Keep that file: it is what you pass tohelmor a GitOps tool if you move off ntctl.
-f. --yes takes the defaults for the rest:NT_POSTGRES_URL and NT_REDIS_URL to run both in the cluster. For a
US tenant without ntctl login, pass -f with global.saasRegion: "us".Behind a Gateway API Gateway instead of an ingress controller, add
--gateway <namespace>/<name>. The endpoints are exposed as HTTPRoutes on that
existing Gateway, which terminates TLS, so its certificate and DNS must cover
every name under the domain, including *.llm. and *.mcp..When only your platform team may create Secrets, --secrets-mode print
checks the cluster and saves the values, then prints the kubectl commands that
create the Secrets, with a placeholder for every credential, and changes
nothing. Run those commands, then install with
ntctl install -f neuraltrust-values.yaml.To add a product later, run ntctl install again. On an existing install it
asks only which products to add; see
Add or remove a product.Verify
Running is not the same as working. A Hybrid data plane’s readiness probe includes a snapshot check, so it reports Ready only after it has pulled configuration from SaaS.ntctl status shows each workload’s readiness, the pods that need attention,
and the DNS records to create. When something is not Ready,
ntctl doctor
says why.
With all three products enabled you should see agentgateway-proxy,
agentgateway-mcp, trustguard-data-plane, data-plane-api, dataagent, and
dataagent-trustguard, plus firewall and its workers with
global.firewall.deploy: true. In-cluster
control-plane-postgresql and redis appear only while deploy: true.
A pod that is Running but never Ready is the signature of config-sync
failing. Three causes, in order of likelihood:
- The token is wrong, or was pasted with trailing whitespace
- Egress to your region’s hosts on 443 —
*.neuraltrust.aior*.us.neuraltrust.ai— is blocked, orglobal.saasRegiondoes not match the region that issued the token — see Network and Choose your region - A TLS-intercepting proxy is present and its CA is not trusted
configSync.tlsCa. Do not reach for
configSync.tlsInsecure — the runtimes refuse it under a deployed APP_ENV.
Config sync covers the mechanism and its
diagnostics.
Expose both entry points
Hostnames derive fromglobal.domain:
TrustGate listens on two ports for two different protocols, and they need
separate hostnames in production:
What “metadata only” means concretely
Raw prompts and responses are written to your PostgreSQL. What leaves your cluster as telemetry is request metadata, timings, detector verdicts, and counts. The one exception is classification. With the NeuralTrust-hosted Firewall, the default, TrustGuard sends the text each ML detector must score to the Firewall in your control plane’s region, over TLS, and gets a score back. The Firewall is stateless: the text is evaluated in memory and not stored. If your policy does not allow text to leave your environment, run the Firewall in your cluster withglobal.firewall.deploy: true.
The egress path is worth knowing. A clickstack-egress-collector runs alongside
your primary DataAgent. It exchanges the DataAgent enrollment JWT for a
short-lived OTLP access token through a loopback broker on 127.0.0.1:9465,
then exports over OTLP with that token. There is no long-lived bearer token on
the application pods, and the broker only listens on loopback.
This is also why the chart refuses to install with a product enabled but no
DataAgent enrollment configured: without it the egress collector has nothing to
exchange, so telemetry would silently never leave.
Operational telemetry is opt-in
Everything above is product telemetry, and it is always on. Separately, TrustGate and TrustGuard can export their own health traces and metrics — request rate, errors, latency — so NeuralTrust support can see how your data plane is doing without asking you for logs. That is off by default: setglobal.observability.operationalTelemetry: true to send it. It takes the same
egress collector and enrollment credentials, so it needs no new hosts, Secrets,
or firewall rules, and it carries no prompt or response payloads. Turning it on,
at least while a support case is open, is the recommended practice. See
Operational telemetry
for what it contains and how to switch it.
The checks above confirm the install came up. Before handing it over, work
through the validation checklist — it
covers what kubectl cannot see: DNS and TLS from the caller’s network, MCP
tested separately from the LLM route, policy decisions, and telemetry actually
arriving.
Upgrades
- ntctl
- Helm
StoreInstallations gRPC service for DB-less
MCP data planes), so a data plane that lags behind the hosted control plane may stop
receiving configuration or fail MCP Store calls. Upgrade promptly when NeuralTrust
announces a control-plane release, and in a Central
setup upgrade the control plane and its data planes together.
Session revocations travel in that snapshot as session_revocations. A hybrid,
self-hosted, or other customer-run data plane applies them only after it runs a
TrustGate build that reads session revocations from the config sync (post-#1142).
An older plane ignores the field and keeps serving, so the member’s existing
MCP sign-ins on that plane keep working until it is upgraded. The control plane
can be upgraded first. See Session revocations.
One caveat: workloads that read configuration through envFrom carry no
checksum of the ConfigMap, so a values change that only touches a ConfigMap
updates the ConfigMap without restarting the pods. The change then takes effect
at the next unrelated restart. If you changed something behavioural, restart the
affected deployment yourself:
High availability
The availability ladder is the same for every self-hosted model, so it lives on one page: what each tier survives, the replica and disruption settings the chart does and does not expose, the failure table, and the promotion runbook.High availability
The four tiers, from one cluster across zones to two active regions, the two invariants, and exactly what you change in the chart.
Each cluster pulls its own configuration
Both clusters use the same gateway scope and the same credentials for it, and each pulls from NeuralTrust SaaS independently, keeping its own last-known-good copy. There is no shared configuration store between them and nothing to replicate — which is why a second cluster costs you no configuration work. The passive cluster is warm: its workloads run, pass health checks, and keep synchronizing, but it receives no production traffic.Only the active cluster runs DataAgent
Two DataAgents on one gateway scope register duplicate streams. Stop it in the passive cluster and start it only as a step in the promotion — never leave both running “ready to go”.Losing SaaS is not losing the request path
If the control plane or your egress to it goes down, both clusters keep enforcing their last synchronized configuration and traffic is unaffected. Only configuration changes stop, and they apply once sync resumes. The exception is a pod that restarts during the outage: it needs the last-known-good file on persistent storage, or it will not become Ready. See restarts during a control-plane outage.Hybrid at tier 3
Both cluster edges must accept the NeuralTrust inbound source IP, and the global
LLM/proxy and MCP URLs must resolve to the same cluster — keep both protocols
together through a promotion.
Next steps
Console setup
Create both products and map their tokens to chart Secrets.
Configuration
Managed stores, ingress, TLS, and every values switch.
Secrets
What the chart generates and what you must supply.
Cloud notes
Ingress, certificates, and managed stores per provider.
Validation checklist
What to prove before the install counts as done.
Troubleshooting
Install, pod, config-sync, and telemetry failures.