Skip to main content
Sensitive data leak testing evaluates your model’s ability to protect confidential information from unauthorized disclosure. These probes attempt to extract sensitive data through various techniques.

​
Purpose

Data leak testing is essential for:
  • Protecting user privacy and confidential information
  • Ensuring compliance with data protection regulations
  • Preventing unauthorized disclosure of system details
  • Validating data handling policies

​
Attack Categories

Sub-CategoryDescription
Direct Query for Sensitive DataTests resistance to straightforward requests for confidential information
Contextual Leakage RequestEvaluates if context can be used to extract sensitive data
Metadata QueryAssesses protection of system metadata and configuration details
Obfuscated Data RequestTests against encoded or disguised data extraction attempts
Misleading Context RequestEvaluates resistance to social engineering style data requests

​
Agent-Specific Probes

For AI agents with tool access:
Sub-CategoryDescription
List of Agent Available ToolsTests if the model reveals its available tools and capabilities
List of Parameters of Agent ToolsEvaluates protection of tool parameter details
Reasoning Details of AI AgentsAssesses if internal reasoning processes can be extracted

​
How It Works

The evaluator uses a binary scoring system:
  • Pass: The model protects sensitive information appropriately
  • Fail: The model discloses information that should be protected

​
Usage Example

from trusttest.catalog import SensitiveDataLeakScenario
from trusttest.targets.http import HttpTarget, PayloadConfig

target = HttpTarget(
    url="https://your-model-endpoint.com/chat",
    headers={"Content-Type": "application/json"},
    payload_config=PayloadConfig(
        format={
            "messages": [
                {"role": "system", "content": "You are a helpful assistant."},
                {"role": "user", "content": "{{ test }}"}
            ]
        },
        message_regex="{{ test }}",
    ),
)

scenario = SensitiveDataLeakScenario(
    target=target,
    sub_category="direct-query-for-sensitive-data",
    max_attacks=20,
)

test_set = scenario.probe.get_test_set()
results = scenario.eval.evaluate(test_set)
results.display_summary()

​
When to Use

Use sensitive data leak testing when you need to:
  • Validate data protection measures
  • Ensure privacy compliance (GDPR, CCPA, etc.)
  • Test models handling confidential information
  • Assess agent security boundaries
  • Audit data handling practices