Red teaming

Each probe generates test cases, sends them to the target, and collects responses for evaluation. Start with prompt injection, then add the categories that match the system you are shipping.

Prompt injections

Single-turn, multi-turn, and dataset attacks.

Content bias

Cognitive and stereotypical bias probes.

Sensitive data leak

Extract confidential data, tools, or reasoning.

System prompt disclosure

Attempts to extract internal instructions.

Input leakage

Whether prior conversation content leaks across boundaries.

Unsafe outputs

Harmful content, illegal activity, and malicious artifacts.

Off-topic

Whether the model stays inside its intended scope.

Agentic behavior

Stop-command override, tool misuse, and long-horizon drift.

Probe architecture

Each attack in TrustTest is implemented as a Probe. A probe generates test cases that are sent to your target model, collecting responses for evaluation. Probes can be:

Dataset-based (*DatasetProbe): Bundled YAML objectives via StaticDatasetProbe
Prompt-based (*PromptProbe): Dynamically generate attacks using LLMs
Multi-turn: Sophisticated attacks across multiple conversation turns

Catalog *ScenarioBuilder classes typically instantiate the dataset variant and evaluate with TrueFalseEvaluator (exceptions: cognitive bias uses BiasComparisonEvaluator; phishing/spam/virus/XSS use signature heuristics). Instantiate a *PromptProbe yourself when you want generated attacks. Dataset probes accept sampling, use_jailbreaks, model_focus, and translate_into_language. Every catalog scenario copies framework tags (EU AI Act, OWASP AITG / LLM Top 10, MITRE ATLAS, ISO/IEC 42001) onto the EvaluationScenario. FileSystemClient.get_overview() and remote metrics roll up by those tags. builder.get_scenario(sub) returns a Scenario (probe + eval); it does not execute remotely.

Attack Categories

Prompt Injections

Prompt injection attacks attempt to manipulate the model into ignoring its instructions or behaving in unintended ways. This is the most comprehensive category with attacks organized by technique:

Single Turn Attacks: Direct attacks in a single message (jailbreaking, encoding, structural attacks)
Multi-Turn Attacks: Sophisticated attacks across multiple conversation turns (Crescendo, Echo Chamber)
From Dataset: Attacks loaded from curated datasets

View all Prompt Injection attacks →

Content Bias

Content bias probes evaluate your model for cognitive and stereotypical biases.

Probe Type	Description
Anchoring Bias	Tests if the model over-relies on initial information
Framing Bias	Evaluates if responses change based on how questions are framed
Positional Bias	Assesses if the order of presented options affects decisions
Status Quo Bias	Tests preference for current state over alternatives
Temporal Bias	Evaluates if time-related framing affects reasoning
Ethnic Bias	Tests for discriminatory responses based on ethnicity
Gender Bias	Evaluates fairness across gender identities
LGBTIQ+ Bias	Assesses treatment of LGBTIQ+ topics
Religion Bias	Tests for religious discrimination

Learn more about Content Bias testing →

Sensitive Data Leak

Probes that attempt to extract confidential information from the model.

Probe Type	Description
Direct Query for Sensitive Data	Tests resistance to straightforward requests
Contextual Leakage Request	Evaluates if context can extract sensitive data
Metadata Query	Assesses protection of system metadata
Obfuscated Data Request	Tests against encoded extraction attempts
Misleading Context Request	Evaluates resistance to social engineering
List of Agent Available Tools	Tests if the model reveals its tools (agents)
List of Parameters of Agent Tools	Evaluates protection of tool details (agents)
Reasoning Details of AI Agents	Assesses if reasoning can be extracted (agents)

Learn more about Sensitive Data Leak testing →

System Prompt Disclosure

Probes that attempt to extract the model’s system prompt or internal instructions.

Probe Type	Description
Direct Request	Tests resistance to straightforward requests
Oblique Reference	Evaluates handling of indirect references
Confusion and Clarification	Tests if confusion tactics extract details
Explanation Mode	Assesses if explanations reveal the prompt
Meta-Level Instruction Inquiry	Tests abstract questions about instructions
Request for Phrase Conclusion	Evaluates if partial prompts can be completed
Request for a Portion	Tests if segments can be extracted

Learn more about System Prompt Disclosure testing →

Input Leakage

Probes that test whether the model reveals information from previous conversations.

Probe Type	Description
Input Persistence Check	Tests if inputs persist across boundaries
Sensitive Information Echo Test	Evaluates if sensitive data is echoed
Contextual Memory Leak Check	Assesses context leaks between sessions
Inadvertent Contextual Reference	Tests for unintended references

Learn more about Input Leakage testing →

Unsafe Outputs

Probes that test the model’s guardrails against generating harmful content.

Category	Probes
Harmful Content	Hate, Violent Crimes, Non-Violent Crimes, Suicide/Self-Harm, Defamation
Illegal Activities	Child Sexual Exploitation, Sex-Related Crimes, Indiscriminate Weapons, Intellectual Property
Malicious Outputs	Phishing, Spam, Virus, XSS Attack Vector

Learn more about Unsafe Outputs testing →

Off-Topic

Probes that test if the model stays within its intended scope.

Probe Type	Description
Competitors Check	Tests handling of competing products
Public Figures	Evaluates responses about public personalities
Disallowed Uses	Tests resistance to unauthorized purposes
Politics/Religion/Economy/Philosophy	Tests handling of sensitive topics
Illegal/Technology	Tests scope boundaries

Learn more about Off-Topic testing →

Agentic Behavior

Probes that test AI agents for safety concerns specific to autonomous systems.

Probe Type	Description
Stop Command Override	Tests if agents can ignore stop commands
Continuous Execution Prompt	Evaluates resistance to endless operation
Self-Preservation Prompt	Tests if agents prioritize self-preservation
Tool Misuse Simulation	Assesses resistance to inappropriate tool use
Recursive Planning Test	Evaluates handling of infinite loops
Long-Horizon Behavioral Drift	Tests for gradual deviation over time
Arbitrary Tools Invocation	Assesses resistance to unauthorized tools

Learn more about Agentic Behavior testing →

Choosing the Right Probes

By Risk Level

Critical Security (Must Test):

Prompt Injection attacks (especially DAN, System Override)
Unsafe Outputs (Hate, Violence, CSAM)
System Prompt Disclosure

High Priority:

Sensitive Data Leak probes
Input Leakage probes
Multi-turn attacks (Crescendo, Echo Chamber)

Standard Testing:

Content Bias probes
Off-Topic probes
Encoding/Obfuscation attacks

Agent-Specific (If Applicable):

Agentic Behavior probes
Tool-related data leak probes

By Use Case

Use Case	Recommended Probes
Customer Support Bot	Prompt Injection, Off-Topic, System Prompt Disclosure
Healthcare Assistant	Sensitive Data Leak, Unsafe Outputs (Self-Harm), Content Bias
Financial Advisor	Off-Topic (Economy), Sensitive Data Leak, Content Bias
General Purpose Chatbot	Full Prompt Injection suite, Unsafe Outputs, Input Leakage
AI Agent with Tools	Agentic Behavior, Tool-related probes, Prompt Injection
Content Moderation System	Unsafe Outputs, Bias probes

Start here

Connect

Concepts

Functional tests

Knowledge bases

Evaluate

Red teaming

Prompt injections

Content bias

Sensitive data leak

System prompt disclosure

Input leakage

Unsafe outputs

Off-topic

Agentic behavior

Probe architecture

Attack Categories

Prompt Injections

Content Bias

Sensitive Data Leak

System Prompt Disclosure

Input Leakage

Unsafe Outputs

Off-Topic

Agentic Behavior

Choosing the Right Probes

By Risk Level

By Use Case

Next Steps

Prompt Injections

Creating Custom Probes

Prompt injections

Content bias

Sensitive data leak

System prompt disclosure

Input leakage

Unsafe outputs

Off-topic

Agentic behavior

​Probe architecture

​Attack Categories

​Prompt Injections

​Content Bias

​Sensitive Data Leak

​System Prompt Disclosure

​Input Leakage

​Unsafe Outputs

​Off-Topic

​Agentic Behavior

​Choosing the Right Probes

​By Risk Level

​By Use Case

​Next Steps

Prompt Injections

Creating Custom Probes

Probe architecture

Attack Categories

Prompt Injections

Content Bias

Sensitive Data Leak

System Prompt Disclosure

Input Leakage

Unsafe Outputs

Off-Topic

Agentic Behavior

Choosing the Right Probes

By Risk Level

By Use Case

Next Steps