Off-Topics

Off-topic testing evaluates your model’s ability to stay within its intended scope and appropriately handle requests outside its domain. This ensures models maintain focus and avoid engaging with content they shouldn’t address.

Purpose

Off-topic testing is essential for:

Ensuring models stay within their intended scope
Preventing engagement with sensitive or unauthorized topics
Maintaining brand consistency
Avoiding liability from inappropriate responses

Content Categories

Scope Boundaries

Sub-Category	Description
Competitors Check	Tests handling of requests about competing products or services
Public Figures	Evaluates responses about celebrities, politicians, and public personalities
Disallowed Uses	Tests resistance to using the model for unauthorized purposes

Sensitive Topics

Sub-Category	Description
Politics Disallowed Content	Tests handling of political discussions when restricted
Religion Disallowed Content	Evaluates responses to religious topics when restricted
Economy Disallowed Content	Tests handling of economic/financial advice when restricted
Philosophy Disallowed Content	Evaluates philosophical discussions when restricted
Illegal Disallowed Content	Tests handling of requests about illegal activities
Technology Disallowed Content	Evaluates technical discussions outside model scope

Dataset vs Prompt probes

The catalog builder instantiates *DatasetProbe classes and evaluates with TrueFalseEvaluator. Each subcategory also has a *PromptProbe (for example CompetitorsPromptProbe). Dataset probes accept sampling, use_jailbreaks, max_jailbreak_per_objective, and translate_into_language. The builder maps language → translate_into_language and num_test_cases → max_objectives_per_category. See StaticDatasetProbe. Catalog scenarios attach EU AI Act Art. 15, OWASP AITG-APP-05 / LLM05, MITRE ATLAS AML.T0054, and ISO/IEC 42001 tags.

How It Works

The evaluator uses a binary scoring system:

Pass: The model appropriately stays within scope or redirects
Fail: The model engages with off-topic content it should avoid

Usage Example

from trusttest.catalog.off_topic import OffTopicScenarioBuilder, SubCategory
from trusttest.targets.http import HttpTarget, PayloadConfig

target = HttpTarget(
    url="https://your-model-endpoint.com/chat",
    headers={"Content-Type": "application/json"},
    payload_config=PayloadConfig(
        format={
            "messages": [
                {"role": "system", "content": "You are a customer service assistant for an airline."},
                {"role": "user", "content": "{{ test }}"}
            ]
        },
        message_regex="{{ test }}",
    ),
)

builder = OffTopicScenarioBuilder(target=target, num_test_cases=20)
scenario = builder.get_scenario(SubCategory.COMPETITORS_CHECK)

test_set = scenario.probe.get_test_set()
results = scenario.eval.evaluate(test_set)
results.display_summary()

When to Use

Use off-topic testing when you need to:

Define and enforce content boundaries
Ensure models stay on-brand
Prevent engagement with sensitive topics
Test domain-specific assistants
Validate content policy compliance

Start here

Connect

Concepts

Red teaming

Functional tests

Knowledge bases

Evaluate

Purpose

Content Categories

Scope Boundaries

Sensitive Topics

Dataset vs Prompt probes

How It Works

Usage Example

When to Use

​Purpose

​Content Categories

​Scope Boundaries

​Sensitive Topics

​Dataset vs Prompt probes

​How It Works

​Usage Example

​When to Use

Purpose

Content Categories

Scope Boundaries

Sensitive Topics

Dataset vs Prompt probes

How It Works

Usage Example

When to Use