Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.neuraltrust.ai/llms.txt

Use this file to discover all available pages before exploring further.

​
What this covers

LangChain applications and LangGraph workflows that call LLM providers through ChatOpenAI, ChatAnthropic, ChatGoogleGenerativeAI, AzureChatOpenAI, or ChatBedrock. TrustGate inspects each LLM hop independently — not the LangChain orchestration itself — so policies apply to every model call the chain or graph makes.
  • Surface: Gateway
  • Who is this for: Python (langchain, langgraph) and TypeScript (@langchain/core, @langchain/langgraph) stacks.

​
Architecture

LangChain / LangGraph app ──► model client ──► TrustGate Gateway ──► LLM provider
                                                        │
                                                        └── one inspection per LLM hop
Every node or chain step that calls llm.invoke(...) becomes a discrete request in the Gateway, with its own route match and detector decisions.

​
Step-by-step setup

Every upstream provider your chain or graph touches is registered once as an Integration; the Gateway then exposes each one on its own Route. Your chain code never sees the provider’s native API key — only the TrustGate Gateway key.
1

Register each provider as an Integration

Open Integrations → Add Integration and register every provider your chain calls — OpenAI, Anthropic, Azure OpenAI, Google, Bedrock, Mistral, etc. Enter the provider’s credentials on the Integration form. Give each one a name you’ll recognize (e.g. openai-prod, anthropic-prod).
2

Create a Gateway Integration

Integrations → Add Integration → Gateway. Pick Serverless or Dedicated, name it (e.g. chains-prod), save, and note the Endpoint from Gateway → Overview.Default Routes for every provider Integration from Step 1 are created automatically — one for OpenAI (/v1/chat/completions), one for Anthropic (/v1/messages), one for Bedrock, and so on. Under Gateway → Routes you can add Tags (for example rag, retrieval) so you can later author tighter policies for retrieval-heavy hops without creating routes manually.
3

Issue a Gateway API key

On the Gateway Integration’s API Keys tab, create a key. One key covers all Routes on that Gateway — chains with mixed providers use a single TrustGate key.
4

Point every model client at the Gateway

Swap base_url / baseURL and api_key / apiKey on each LangChain model client. Prompts, tools, agent graphs, and runnables stay unchanged. Snippets below.
5

Verify in Runtime → Explorer

Invoke a chain and confirm each LLM hop shows up as an independent Explorer entry with its Route, detector decisions, and token usage.

​
Client code

​
LangChain (Python)

from langchain_openai import ChatOpenAI
from langchain_anthropic import ChatAnthropic

gateway = "https://<gateway>.neuraltrust.ai"
key = "<trustgate-api-key>"

openai_llm = ChatOpenAI(
    model="gpt-4o",
    base_url=f"{gateway}/v1",
    api_key=key,
)

anthropic_llm = ChatAnthropic(
    model="claude-3-5-sonnet-latest",
    base_url=gateway,
    api_key=key,
)
Use them anywhere you use a regular LangChain LLM: 
from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_messages([("user", "{q}")])
chain = prompt | openai_llm
chain.invoke({"q": "Summarize the contract below..."})

​
LangChain (TypeScript)

import { ChatOpenAI } from "@langchain/openai";

const llm = new ChatOpenAI({
  model: "gpt-4o",
  configuration: {
    baseURL: "https://<gateway>.neuraltrust.ai/v1",
    apiKey: "<trustgate-api-key>",
  },
});

​
LangGraph

LangGraph nodes use the same LangChain model clients. Configure them once and pass them into your graph as usual: 
from langgraph.graph import StateGraph
from typing_extensions import TypedDict

class State(TypedDict):
    input: str
    output: str

def think(state: State) -> State:
    reply = openai_llm.invoke(state["input"])
    return {"output": reply.content, **state}

graph = StateGraph(State)
graph.add_node("think", think)
graph.set_entry_point("think")
graph.set_finish_point("think")
app = graph.compile()

​
Tools and agents

For tool-calling agents (create_react_agent, AgentExecutor, LangGraph prebuilt agents), the LLM hop is protected by the Gateway. To also protect tool execution — for example, a tool that calls an internal HTTP API — wrap that tool’s endpoint with a separate Gateway route or an API integration.

​
Correlate hops into one conversation

To group hops into a single conversation in Explorer, forward a stable conversation_id header on every model client: 
openai_llm = ChatOpenAI(
    model="gpt-4o",
    base_url=f"{gateway}/v1",
    api_key=key,
    default_headers={"x-conversation-id": conversation_id},
)

​
Policies to apply

Chains and graphs amplify whatever content flows into a model call — a single poisoned RAG document reaches every downstream hop. Because TrustGate inspects each LLM call independently, you get one policy decision per hop, and policies composed across hops stack using the block ▶ mask ▶ log precedence described in Policies & Enforcement. Scope policies with the Gateways or Routes filter so RAG-heavy chains and customer-facing chains can have different rules.

​
Block prompt injection — including indirect injection from RAG

  • Where — Gateway + filter Gateways = <your-gateway>
  • When — Input · Triggers · Prompt Injection, Jailbreak
  • Then — Block
This fires on every hop, so an injection embedded in a retrieved document trips the policy on the hop that consumes it.

​
Mask PII across every chain step

  • Where — Gateway + filter Gateways = <your-gateway>
  • When — Input or Output · Triggers · Email Address, Phone Number, Credit Card, Social Security Number
  • Then — Mask

​
Block credential leakage in outputs

  • Where — Gateway + filter Gateways = <your-gateway>
  • When — Output · Triggers · API Key / Secret
  • Then — Block

​
Tool-call guard for agents

  • Where — Gateway + filter Routes = <agent-routes>
  • When — Tool Call · Triggers · Suspicious Arguments, Prompt Injection
  • Then — Block
Prevents a compromised hop from emitting a tool invocation that exfiltrates data or executes unsafe operations.

​
Log RAG context for audit

  • Where — Gateway + filter Routes = <rag-routes>
  • When — empty (every request in scope)
  • Then — Log
Baseline visibility into what retrieved context a chain is passing to the model. Authored in Log mode first, these five patterns give you an end-to-end view of every hop before you flip the switch to Mask or Block.

​
Limitations

  • Per-hop inspection: detectors run on each LLM call independently. Multi-turn context is inferred from the messages you pass in; use a conversation header to correlate.
  • Streaming: streamed responses are inspected once the stream completes; Mask and Block apply to the final consolidated payload.
  • Tool execution: LangChain tool calls that hit external APIs are not automatically covered. Wrap those endpoints with their own Gateway or API integration if they handle sensitive data.
  • Custom runnables: anything that bypasses the standard chat models (direct httpx calls, custom providers) will not be inspected unless it also uses the Gateway base URL.