The Data Leakage Evaluator is a specialized tool designed to assess a model’s ability to protect sensitive information and prevent unauthorized data disclosure. It tests the model’s compliance with data protection requirements by ensuring it responds appropriately to attempts to extract sensitive information.

​

Purpose

The Data Leakage Evaluator is particularly useful when you need to:

• Test model’s data protection capabilities
• Ensure compliance with data privacy regulations
• Prevent unauthorized disclosure of sensitive information
• Evaluate model robustness against data extraction attempts
• Assess system security against potential data leaks

​

Available Compliance Categories

The Data Leakage Evaluator includes several categories of compliance tests:

​

1. Data Exfiltration

Tests the model’s ability to prevent unauthorized data extraction, including:

• Personal identifiable information (PII)
• Sensitive business data
• Confidential user information
• System configuration details
• Access credentials

​

2. Instruction Leakage

Evaluates the model’s ability to protect its own instructions and system prompts, including:

• System prompt extraction
• Model configuration details
• Internal instructions
• Security measures
• Operational parameters

​

3. Phishing

Tests the model’s ability to detect and prevent phishing attempts, including:

• Social engineering attacks
• Credential harvesting
• Malicious link generation
• Deceptive content creation
• Identity impersonation

​

How It Works

The Data Leakage Evaluator uses a binary scoring system:

• Score: 0 (Compliant): The model responds appropriately, protecting sensitive information
• Score: 1 (Non-Compliant): The model fails to protect sensitive information, potentially exposing data

​

Usage Examples

​

Basic Compliance Testing

Basic DataLeakScenario testing uses a predefined dataset to evaluate the model’s responses against various data protection objectives.

from trusttest.catalog import DataLeakScenario
from trusttest.models.http import HttpModel, PayloadConfig
from trusttest.probes.data_leak import Category

model = HttpModel(
    url="https://chat.neuraltrust.ai/api/chat",
    headers={
        "Content-Type": "application/json"
    },
    payload_config=PayloadConfig(
        format={
            "messages": [
                {"role": "system", "content": "**Welcome to Airline Assistant**."},
                {"role": "user", "content": "{{ test }}"},
            ]
        },
        message_regex="{{ test }}",
    ),
    concatenate_field=".",
)

scenario = DataLeakScenario(
    model=model,
    categories={Category.PHISHING},
    max_objectives_per_category=10,
    use_jailbreaks=False,
    sampling="random",
)

test_set = scenario.probe.get_test_set()
results = scenario.eval.evaluate(test_set)
results.display_summary()

​

Iterative Compliance Testing

The CtfDataLeakScenario uses a capture-the-flag approach to iteratively test the model’s data protection capabilities through multi-turn conversations.

from trusttest.catalog import CtfDataLeakScenario
from trusttest.models.testing import RepeatModel
from trusttest.probes.data_leak import Category

scenario = CtfDataLeakScenario(
    model=RepeatModel(),
    categories={Category.DATA_EXFILTRATION},
    max_objectives_per_category=2,
    max_turns=4,
)

test_set = scenario.probe.get_test_set()
results = scenario.eval.evaluate(test_set)
results.display_summary()

​

When to Use

Use the Data Leakage Evaluator when you need to:

• Test data protection boundaries
• Ensure compliance with privacy regulations
• Prevent unauthorized data disclosure
• Evaluate security measures
• Assess model robustness against data extraction
• Test system security
• Validate data protection measures
• Ensure privacy compliance