NeuralTrust Architecture
NeuralTrust provides enterprise-grade AI security protection, testing, and vulnerability management through a distributed architecture designed for scalability, security, and performance. The platform operates with a unique data sovereignty model where your Data Plane runs entirely within your cloud environment while the Control Plane operates in NeuralTrust’s secure infrastructure.High-Level Architecture
Deployment modes
NeuralTrust provides several deployment modes to suit your needs:| Mode | Control Plane location | Data Plane location | Ideal for |
|---|---|---|---|
| 🏢 On-Premises | Your infrastructure | Your infrastructure | Maximum data sovereignty and regulatory control |
| ☁️ SaaS | NeuralTrust infrastructure | NeuralTrust infrastructure | Fully managed operations with minimal upkeep |
| 🔗 Hybrid | NeuralTrust infrastructure | Your infrastructure | Balance of managed services with local data residency |
LLM endpoint
Some Neuraltrust componentes requires access to a LLM endpoint to perform their operations. Such as LLM as a Judge evaluators, Topic summarizer, etc.Take into account that the cluster where you deploy the Data Plane components needs to have access to the LLM endpoint.
| OpenAI | Direct REST integration |
| OpenAI-compatible APIs | Any provider implementing the OpenAI API shape |
| Google Gemini | Direct REST integration |
| Ollama | Self-hosted models served from your infrastructure |
| Anthropic | Claude models across regions |
| Azure OpenAI | Azure-hosted OpenAI endpoints |
In SaaS mode you can use our LLM endpoint
LLM target/upstream
Some Neuraltrust componentes requires access to your LLM that you want to protect or test.- For testing purposes check our TrustTest documentation.
- For protecting purposes check our TrustGate documentation.
Take into account that the cluster where you deploy the Data Plane components and TrustGate needs to have access to the LLM target/upstream.
Data Plane Components
The Data Plane components manage your private data and endpoints, like user conversations, traces, metrics and more.API
Your Data Plane API serves as the primary ingestion endpoint for AI monitoring data from your applications and systems. This high-performance service provides:- High-Throughput Ingestion: Handles millions of AI monitoring events per second with minimal latency
- Real-Time Validation: Validates and enriches incoming data streams for immediate processing
- Secure Authentication: Implements robust API key management and authentication mechanisms
- Auto-Scaling: Automatically scales based on incoming data volume and processing demands
Worker
The Data Plane Worker performs all observability metrics processing operations.- Real-Time Analysis: Processes AI monitoring events for immediate insights and anomaly detection
- Privacy-Preserving Analytics: Performs analysis while keeping raw data within your environment
- Alert Generation: Triggers immediate alerts for critical AI security and performance events
Message Queue
The Data Plane Message Queue provides reliable, high-performance message processing between Data Plane components:- Event Sourcing: Maintains complete audit trails of all AI monitoring events
- Guaranteed Delivery: Ensures no data loss during processing with configurable durability
- Stream Processing: Enables real-time data processing and analysis workflows
- Backpressure Management: Automatically manages load balancing during traffic spikes
Currenlty we only support Kafka for the Data Plane Message Queue. Being deployed in the same cluster or as external service.
Database
The Data Plane Database stores all sensitive AI monitoring, tests, conversations and more data within a controlled environment:- Complete Data Sovereignty: All raw AI monitoring data remains in your VPC
- High-Performance Analytics: Optimized for real-time queries and complex analytics
- Automated Encryption: All data encrypted at rest with your managed keys
- Compliance Ready: Designed for GDPR, HIPAA, SOX, and other regulatory requirements
Currenlty we only support ClickHouse for the Data Plane Database. Being deployed in the same cluster or as external service.
Control Plane Components
The Control Plane components provide centralized management, advanced AI security capabilities.API
The Control Plane API provides the backbone for all Control Plane operations and customer interactions:- Customer Portal Integration: Powers the NeuralTrust dashboard and customer interfaces
- Multi-Tenant Architecture: Securely isolates customer environments and data
- Global Orchestration: Coordinates across multiple customer Data Plane deployments
Application
The Control Plane Application web app delivers the comprehensive NeuralTrust user experience:- Real-Time Dashboards: Interactive dashboards showing AI security and performance insights
- Global Analytics: Cross-environment insights that don’t compromise individual data privacy
- Alert Management: Centralized alert configuration and notification management
Scheduler
The Control Plane Scheduler manages automated testing and monitoring operations:- TrustTest Job Scheduling: Schedules and orchestrates TrustTest execution jobs across customer environments
- TrustScan Job Scheduling: Schedules and coordinates TrustScan security analysis jobs and vulnerability assessments
- Metrics Check Scheduling: Coordinates periodic metrics checks for alert generation and monitoring
- Automated Test Execution: Manages the timing and execution of AI testing and scanning workflows
Database
The Control Plane Database stores only basic operational data for CRUD operations:- Zero Raw Data: Never stores customer’s sensitive AI monitoring data
- Configuration Management: Stores system configurations and user preferences
- User Management: Handles user accounts, permissions, and authentication data
- Operational Metadata: Basic CRUD operations for system management and administration
Currenlty we only support PostgreSQL for the Control Plane Database. Being deployed in the same cluster or as external service.
Data Flow and Privacy Model
NeuralTrust’s architecture ensures complete data sovereignty while enabling comprehensive AI security protection, testing, and vulnerability management through privacy-preserving data flows.AI Application Data Flow (Your Environment)
- AI Applications → TrustGate Gateway Service: AI applications route through the Gateway service for real-time validation against configured security policies
- TrustGate Gateway Service → DP Queue: Validated and protected AI interaction data is written directly to the queue for processing
- DP Queue → DP Worker: Workers process and analyze data while keeping everything in your VPC
- DP Worker → DP Database: All raw AI interaction data is stored in your controlled environment
- TrustGate Admin Service: Configure security policies, routing rules, rate limits, and access controls that govern the Gateway service behavior
Cross-VPC Security Operations (Privacy-Safe Only)
Privacy-Preserving Security Analysis: NeuralTrust’s security products provide comprehensive AI security analysis through different operational models:- TrustTest: Operates in NeuralTrust Control Plane for comprehensive red teaming and testing capabilities
- Local Analysis: TrustScan and AISPM perform analysis within your Data Plane using your complete data
- Privacy-Safe Results: Only aggregated, anonymized security insights cross the VPC boundary
- No Raw Data Transfer: Customer prompts, responses, and user data never leave your environment for vulnerability scanning and posture management
- TrustTest Red Teaming: Runs in NeuralTrust Control Plane to maintain centralized, up-to-date jailbreak database and threat intelligence
- TrustScan Vulnerability Assessment: Conducts security scanning within your environment, shares only privacy-safe threat detection results
- AISPM Posture Management: Scans your complete AI environment including cloud artifacts, Hugging Face models, MCP servers, and external AI service integrations to assess overall security posture and provide comprehensive risk metrics and improvement recommendations
Why TrustTest Runs in Control Plane TrustTest operates exclusively within NeuralTrust’s Control Plane infrastructure to maintain a centralized, continuously updated jailbreak database containing the latest AI attack techniques, prompt injection methods, and adversarial strategies. This centralized approach is critical because new jailbreak techniques emerge daily across the AI security landscape, and maintaining an up-to-date threat intelligence database requires continuous research, monitoring of security communities, and analysis of emerging attack vectors. By running TrustTest in our Control Plane, we ensure that all customers immediately benefit from the latest red teaming capabilities without each organization needing to independently research and catalog new attack methods. Our security research team continuously updates this database with novel jailbreak techniques, sophisticated prompt injection attacks, adversarial examples, and emerging AI vulnerabilities discovered across the global AI security community.
Multi-Cloud Deployment Support
Universal Cloud Provider Support
Your Data Plane can be deployed on any major cloud provider while maintaining identical functionality:- Amazon Web Services (AWS): Full integration with AWS services and global regions
- Google Cloud Platform (GCP): Native GCP service integration and optimization
- Microsoft Azure: Complete Azure ecosystem integration and enterprise features
Cloud-Agnostic Benefits
- Consistent Experience: Identical features and capabilities across all cloud providers
- Provider Independence: No vendor lock-in to any specific cloud platform
Security and Compliance
Data Sovereignty Guarantees
- Legal Guarantees: Your data never leaves your cloud environment or jurisdiction
- Audit Trails: Complete documentation of all data flows and access patterns
- Compliance Ready: Meets requirements for GDPR, HIPAA, SOX, PCI, and other frameworks
Zero-Trust Architecture
- Mutual Authentication: All communication requires cryptographic verification
- Encrypted Channels: End-to-end encryption for all data in transit
- Least Privilege: Minimal necessary permissions for all system components
- Continuous Verification: Ongoing validation of all system connections and data flows
Cloud Provider Support
NeuralTrust provides managed deployment across all major cloud providers, handling infrastructure complexity while ensuring your data remains within your environment and under your control.Amazon Web Services (AWS)
Our comprehensive AWS integration provides native support for AWS services and capabilities across all regions. The Data Plane deploys seamlessly into your AWS VPC with full integration into AWS security services, encryption capabilities, and monitoring tools. Getting Started: AWS Deployment GuideMicrosoft Azure
Our enterprise Azure integration provides seamless connectivity with Microsoft’s cloud platform and enterprise services ecosystem. Deploy your Data Plane into Azure VNets with native integration to Azure security and monitoring services. Getting Started: Azure Deployment GuideGoogle Cloud Platform (GCP)
Our Google Cloud native architecture leverages GCP’s advanced capabilities for enhanced functionality. Data Plane deployment integrates with Google Cloud security services and monitoring capabilities. Getting Started: GCP Deployment GuideGetting Started
Deployment Process
- Cloud Environment Setup: Configure your cloud environment with required permissions
- Data Plane Deployment: Deploy Data Plane components in your VPC
- TrustGate Configuration: Establish secure connection to NeuralTrust Control Plane
- AI System Integration: Connect your AI applications to the DP API
- Monitoring Activation: Begin receiving insights through the Control Plane dashboard
Support and Maintenance
- 24/7 Support: Round-the-clock technical support for deployment and operations
- Managed Updates: Automated security updates and feature deployments
- Performance Optimization: Continuous monitoring and optimization of Data Plane performance
- Compliance Assistance: Expert help with audit preparation and compliance reporting
Data Sovereignty First: NeuralTrust’s unique architecture ensures your sensitive AI data never leaves your environment while providing enterprise-grade AI security monitoring and insights through our advanced Control Plane services.