The Data Leakage Evaluator is a specialized tool designed to assess a model’s ability to protect sensitive information and prevent unauthorized data disclosure. It tests the model’s compliance with data protection requirements by ensuring it responds appropriately to attempts to extract sensitive information.
The Data Leakage Evaluator is particularly useful when you need to:
The Data Leakage Evaluator includes several categories of compliance tests:
Tests the model’s ability to prevent unauthorized data extraction, including:
Evaluates the model’s ability to protect its own instructions and system prompts, including:
Tests the model’s ability to detect and prevent phishing attempts, including:
The Data Leakage Evaluator uses a binary scoring system:
Basic DataLeakScenario
testing uses a predefined dataset to evaluate the model’s responses against various data protection objectives.
The CtfDataLeakScenario
uses a capture-the-flag approach to iteratively test the model’s data protection capabilities through multi-turn conversations.
Use the Data Leakage Evaluator when you need to:
The Data Leakage Evaluator is a specialized tool designed to assess a model’s ability to protect sensitive information and prevent unauthorized data disclosure. It tests the model’s compliance with data protection requirements by ensuring it responds appropriately to attempts to extract sensitive information.
The Data Leakage Evaluator is particularly useful when you need to:
The Data Leakage Evaluator includes several categories of compliance tests:
Tests the model’s ability to prevent unauthorized data extraction, including:
Evaluates the model’s ability to protect its own instructions and system prompts, including:
Tests the model’s ability to detect and prevent phishing attempts, including:
The Data Leakage Evaluator uses a binary scoring system:
Basic DataLeakScenario
testing uses a predefined dataset to evaluate the model’s responses against various data protection objectives.
The CtfDataLeakScenario
uses a capture-the-flag approach to iteratively test the model’s data protection capabilities through multi-turn conversations.
Use the Data Leakage Evaluator when you need to: