Plugin Scopes & Configuration

​

Plugin Types and Functions

• NeuralTrust Guardrail Provides a comprehensive security plugin within TrustGate, designed to protect Large Language Models (LLMs) from harmful, unsafe, or malicious content. It operates in real-time, inspecting incoming data to detect and mitigate risks before they affect the model or end-users.
• Bedrock Guardrail Provides security checks and validation against policy violations using predefined guardrails. (Operator variants may exist for different environments, e.g., in consumer groups vs. global gateway settings.)
• Code Sanitation Inspects and cleans code input to prevent harmful or unwanted operations. (Ensures that any code submitted is safe to execute.)
• Data Masking Identifies sensitive data (such as secret keys, passwords, or personal identifiers) and masks them in logs or outputs. (Utilizes keyword and regex-based rules to apply the appropriate masking.)
• External API Facilitates integration with external services or APIs. (Acts as a bridge to external systems, handling necessary authentication and communication.)
• Injection Protection Protects the system from injection attacks (e.g., SQL injection, script injection) by sanitizing inputs. (Focuses on validating and cleaning inputs to ensure no malicious code is processed.)
• Prompt Moderation Analyzes and moderates prompts, ensuring that inappropriate or harmful content is filtered out before processing. (Crucial for platforms handling user-generated content.)
• Rate Limiter Controls the rate of requests, protecting the backend from abuse by limiting the number of requests per unit time. (Helps maintain system performance and availability.)
• Request Size Limiter Restricts the size of incoming requests to prevent excessively large payloads from impacting the system. (Ensures that the system processes only manageable request sizes.)
• Token Rate Limiter Specifically limits the rate at which tokens (or specific operations) can be used, adding an extra layer of protection against misuse. (Operates on a token basis rather than on request count alone.)
• Toxicity Azure & Toxicity OpenAI Both plugins assess and filter toxic or harmful language from inputs or outputs.
• CORS Enforces Cross-Origin Resource Sharing (CORS) policies to control and secure how your AI Gateway resources are accessed from different origins.
• Contextual Security Analyzes prompts in real time to detect context-specific risks, ensuring sensitive or harmful requests are identified before reaching your AI systems.
• Semantic Cache Stores and retrieves responses based on semantic similarity, reducing latency and improving efficiency by reusing relevant answers from past interactions.
• Bot Detector Provides advanced protection against automated traffic and bot activity by analyzing client-side data to identify suspicious patterns and behaviors commonly associated with bots, scrapers, and automated tools.

​

Plugins Table

​

Plugin Scopes

• Global Gateway Level Plugins defined here apply to all requests processed by the gateway. Example use-case: A global security check for all incoming requests.
• Rule Level Plugins can be applied to specific rules based on the request path, method, or other conditions. Example use-case: Data masking only on a particular endpoint.
• Consumer Group Level Plugins defined for a specific group of consumers allow targeted policy enforcement for a subset of users or clients. Example use-case: Extra authentication checks for a premium consumer group.

​

Plugin Inheritance

Gateway Level (Global)
       ↓
Service Level (Service-specific)
       ↓
Route Level (Route-specific)
       ↓
Consumer Group Level (Group-specific)
       ↓
Consumer Level (Individual)

​

Execution Stages

• Pre-request Executes before the request is processed by the backend service. Use-case: Input validation, authentication, and early data masking.
• Post-request Executes after the request has been sent to the backend but before the response is generated. Use-case: Logging the request details or triggering asynchronous actions.
• Pre-response Executes after the backend has generated a response but before the response is sent back to the consumer. Use-case: Response data transformation, additional security checks, or further masking.
• Post-response Executes after the response is sent. This stage is generally used for logging, analytics, or cleanup actions. Use-case: Detailed auditing or updating metrics.

​

Execution Order: Sequential vs. Parallel

• Sequential Execution When parallel is set to false, plugins run one after the other.
• Priority: In sequential mode, the execution order is determined by the priority value. Lower priority numbers execute before higher ones. Use-case: When the order of execution is critical, such as masking data before logging.
• Parallel Execution When parallel is set to true, plugins execute concurrently, improving performance. Note: In parallel mode, the priority setting is ignored as execution order is not strictly sequential.

​

Plugin Configuration Examples

​

Global Gateway Level Example

{
    "name": "Gateway",
    "required_plugins": [
        {
            "name": "bedrock_guardrail",
            "enabled": true,
            "stage": "pre_request",
            "priority": 1,
            "parallel": true,
            "settings": {
                "guardrail_id": "12345",
                "version": "1",
                "credentials": {
                    "aws_access_key": "@@@",
                    "aws_secret_key": "@@@",
                    "aws_region": "eu-west-1"
                },
                "actions": {
                    "message": "%s"
                }
            }
        }
    ]
}

​

Rule Level Example

{
    "path": "/test",
    "service_id": "{{service_id}}",
    "methods": ["POST"],
    "strip_path": true,
    "active": true,
    "plugin_chain": [
        {
            "name": "data_masking",
            "enabled": true,
            "stage": "pre_request",
            "priority": 2,
            "settings": {
                "similarity_threshold": 0.8,
                "apply_all": true,
                "rules": [
                    {
                        "pattern": "secret_key",
                        "type": "keyword",
                        "mask_with": "[MASKED_KEY]",
                        "preserve_len": false,
                        "case_sensitive": false
                    },
                    {
                        "pattern": "(?i)password=\\S+",
                        "type": "regex",
                        "mask_with": "[MASKED_PASSWORD]",
                        "preserve_len": false
                    },
                    {
                        "pattern": "INT-\\d{6}",
                        "type": "regex",
                        "mask_with": "[MASKED_ID]",
                        "preserve_len": true
                    }
                ]
            }
        }
    ]
}

​

Consumer Group Level Example

{
    "name": "Consumer Group 1",
    "required_plugins": [
        {
            "name": "bedrock_guardrail",
            "enabled": true,
            "stage": "pre_request",
            "priority": 1,
            "settings": {
                "guardrail_id": "12345",
                "version": "1",
                "credentials": {
                    "aws_access_key": "@@@",
                    "aws_secret_key": "@@@",
                    "aws_region": "eu-west-1"
                },
                "actions": {
                    "message": "%s"
                }
            }
        }
    ]
}