Plugin Scopes & Configuration

This documentation explains what plugins are within an AI Gateway environment, their purposes, available scopes, and configuration details. Plugins act as modular components to enforce policies, transform data, or integrate external services in the request/response lifecycle of the gateway. They enhance security, data integrity, performance, and overall API governance.

​

Plugin Types and Functions

Plugins provide various functionalities. Some of them may offer similar functions but differ in the operators or the external services they use. Below is a grouping and explanation of the available plugins:

• NeuralTrust Guardrail Provides a comprehensive security plugin within TrustGate, designed to protect Large Language Models (LLMs) from harmful, unsafe, or malicious content. It operates in real-time, inspecting incoming data to detect and mitigate risks before they affect the model or end-users.

• Bedrock Guardrail Provides security checks and validation against policy violations using predefined guardrails. (Operator variants may exist for different environments, e.g., in consumer groups vs. global gateway settings.)

• Code Sanitation Inspects and cleans code input to prevent harmful or unwanted operations. (Ensures that any code submitted is safe to execute.)

• Data Masking Identifies sensitive data (such as secret keys, passwords, or personal identifiers) and masks them in logs or outputs. (Utilizes keyword and regex-based rules to apply the appropriate masking.)

• External API Facilitates integration with external services or APIs. (Acts as a bridge to external systems, handling necessary authentication and communication.)

• Injection Protection Protects the system from injection attacks (e.g., SQL injection, script injection) by sanitizing inputs. (Focuses on validating and cleaning inputs to ensure no malicious code is processed.)

• Prompt Moderation Analyzes and moderates prompts, ensuring that inappropriate or harmful content is filtered out before processing. (Crucial for platforms handling user-generated content.)

• Rate Limiter Controls the rate of requests, protecting the backend from abuse by limiting the number of requests per unit time. (Helps maintain system performance and availability.)

• Request Size Limiter Restricts the size of incoming requests to prevent excessively large payloads from impacting the system. (Ensures that the system processes only manageable request sizes.)

• Token Rate Limiter Specifically limits the rate at which tokens (or specific operations) can be used, adding an extra layer of protection against misuse. (Operates on a token basis rather than on request count alone.)

• Toxicity Azure & Toxicity OpenAI Both plugins assess and filter toxic or harmful language from inputs or outputs.

​

Plugins Table

Below is a table listing each plugin and its corresponding key:

​

Plugin Scopes

Plugins can be configured at different levels or scopes to allow flexible policy enforcement. The main scopes include:

• Global Gateway Level Plugins defined here apply to all requests processed by the gateway. Example use-case: A global security check for all incoming requests.

• Rule Level Plugins can be applied to specific rules based on the request path, method, or other conditions. Example use-case: Data masking only on a particular endpoint.

• Consumer Group Level Plugins defined for a specific group of consumers allow targeted policy enforcement for a subset of users or clients. Example use-case: Extra authentication checks for a premium consumer group.

​

Plugin Inheritance

Plugins follow a hierarchical inheritance pattern:

Gateway Level (Global)
       ↓
Service Level (Service-specific)
       ↓
Route Level (Route-specific)
       ↓
Consumer Group Level (Group-specific)
       ↓
Consumer Level (Individual)

​

Execution Stages

Each plugin is configured with a stage that determines when it is executed in the request/response cycle. The stages include:

• Pre-request Executes before the request is processed by the backend service. Use-case: Input validation, authentication, and early data masking.

• Post-request Executes after the request has been sent to the backend but before the response is generated. Use-case: Logging the request details or triggering asynchronous actions.

• Pre-response Executes after the backend has generated a response but before the response is sent back to the consumer. Use-case: Response data transformation, additional security checks, or further masking.

• Post-response Executes after the response is sent. This stage is generally used for logging, analytics, or cleanup actions. Use-case: Detailed auditing or updating metrics.

These stages allow precise control over when each plugin interacts with the request/response lifecycle, ensuring that transformations, validations, and integrations occur at the appropriate time.

​

Execution Order: Sequential vs. Parallel

Plugins can be executed either sequentially or in parallel, controlled by the parallel setting:

• Sequential Execution When parallel is set to false, plugins run one after the other.

• Priority: In sequential mode, the execution order is determined by the priority value. Lower priority numbers execute before higher ones. Use-case: When the order of execution is critical, such as masking data before logging.

• Parallel Execution When parallel is set to true, plugins execute concurrently, improving performance. Note: In parallel mode, the priority setting is ignored as execution order is not strictly sequential.

​

Plugin Configuration Examples

Below are examples showing how plugins can be configured at different scopes.

​

Global Gateway Level Example

{
    "name": "Gateway",
    "subdomain": "{{gateway_subdomain}}",
    "required_plugins": [
        {
            "name": "bedrock_guardrail",
            "enabled": true,
            "stage": "pre_request",
            "priority": 1,
            "parallel": true,
            "settings": {
                "guardrail_id": "12345",
                "version": "1",
                "credentials": {
                    "aws_access_key": "@@@",
                    "aws_secret_key": "@@@",
                    "aws_region": "eu-west-1"
                },
                "actions": {
                    "message": "%s"
                }
            }
        }
    ]
}

​

Rule Level Example

{
    "path": "/test",
    "service_id": "{{service_id}}",
    "methods": ["POST"],
    "strip_path": true,
    "active": true,
    "plugin_chain": [
        {
            "name": "data_masking",
            "enabled": true,
            "stage": "pre_request",
            "priority": 2,
            "settings": {
                "similarity_threshold": 0.8,
                "apply_all": true,
                "rules": [
                    {
                        "pattern": "secret_key",
                        "type": "keyword",
                        "mask_with": "[MASKED_KEY]",
                        "preserve_len": false,
                        "case_sensitive": false
                    },
                    {
                        "pattern": "(?i)password=\\S+",
                        "type": "regex",
                        "mask_with": "[MASKED_PASSWORD]",
                        "preserve_len": false
                    },
                    {
                        "pattern": "INT-\\d{6}",
                        "type": "regex",
                        "mask_with": "[MASKED_ID]",
                        "preserve_len": true
                    }
                ]
            }
        }
    ]
}

​

Consumer Group Level Example

{
    "name": "Consumer Group 1",
    "required_plugins": [
        {
            "name": "bedrock_guardrail",
            "enabled": true,
            "stage": "pre_request",
            "priority": 1,
            "settings": {
                "guardrail_id": "12345",
                "version": "1",
                "credentials": {
                    "aws_access_key": "@@@",
                    "aws_secret_key": "@@@",
                    "aws_region": "eu-west-1"
                },
                "actions": {
                    "message": "%s"
                }
            }
        }
    ]
}