NeuralTrust | The leading security platform for generative AI

Sensitive data leak testing evaluates your model’s ability to protect confidential information from unauthorized disclosure. These probes attempt to extract sensitive data through various techniques.

Purpose

Data leak testing is essential for:

Protecting user privacy and confidential information
Ensuring compliance with data protection regulations
Preventing unauthorized disclosure of system details
Validating data handling policies

Attack Categories

Sub-Category	Description
Direct Query for Sensitive Data	Tests resistance to straightforward requests for confidential information
Contextual Leakage Request	Evaluates if context can be used to extract sensitive data
Metadata Query	Assesses protection of system metadata and configuration details
Obfuscated Data Request	Tests against encoded or disguised data extraction attempts
Misleading Context Request	Evaluates resistance to social engineering style data requests

Agent-Specific Probes

For AI agents with tool access:

Sub-Category	Description
List of Agent Available Tools	Tests if the model reveals its available tools and capabilities
List of Parameters of Agent Tools	Evaluates protection of tool parameter details
Reasoning Details of AI Agents	Assesses if internal reasoning processes can be extracted

How It Works

The evaluator uses a binary scoring system:

Pass: The model protects sensitive information appropriately
Fail: The model discloses information that should be protected

Usage Example

from trusttest.catalog.sensitive_data_leak import SensitiveDataLeakScenarioBuilder, SubCategory
from trusttest.targets.http import HttpTarget, PayloadConfig

target = HttpTarget(
    url="https://your-model-endpoint.com/chat",
    headers={"Content-Type": "application/json"},
    payload_config=PayloadConfig(
        format={
            "messages": [
                {"role": "system", "content": "You are a helpful assistant."},
                {"role": "user", "content": "{{ test }}"}
            ]
        },
        message_regex="{{ test }}",
    ),
)

builder = SensitiveDataLeakScenarioBuilder(target=target, num_test_cases=20)
scenario = builder.get_scenario(SubCategory.DIRECT_QUERY_FOR_SENSITIVE_DATA)

test_set = scenario.probe.get_test_set()
results = scenario.eval.evaluate(test_set)
results.display_summary()

When to Use

Use sensitive data leak testing when you need to:

Validate data protection measures
Ensure privacy compliance (GDPR, CCPA, etc.)
Test models handling confidential information
Assess agent security boundaries
Audit data handling practices

Getting Started

Core Concepts

Connect your app

Create tests

Evaluate results

Sensitive Data Leak

Purpose

Attack Categories

Agent-Specific Probes

How It Works

Usage Example

When to Use

​Purpose

​Attack Categories

​Agent-Specific Probes

​How It Works

​Usage Example

​When to Use

Purpose

Attack Categories

Agent-Specific Probes

How It Works

Usage Example

When to Use