The Regex Evaluator is a specialized tool designed to validate text responses against regular expression patterns. It provides a binary evaluation where the response must match the specified regex pattern to pass.

​
Purpose

The Regex Evaluator is particularly useful when you need to:

  • Validate text formats and patterns
  • Check for specific text structures
  • Verify data formats (emails, phone numbers, etc.)
  • Ensure responses follow a particular pattern
  • Test for specific text content requirements

​
How It Works

The evaluator uses a binary scoring system based on regex pattern matching:

  • Score: 1: The response matches the specified regex pattern
  • Score: 0: The response does not match the specified regex pattern

The evaluation uses Python’s re.search() function to check if the response contains any substring that matches the provided regex pattern.

​
Usage Example

import asyncio

from trusttest.evaluation_contexts import ExpectedResponseContext
from trusttest.evaluators import RegexEvaluator


async def evaluate():
    # Create evaluator with an email pattern
    evaluator = RegexEvaluator(
        pattern=r'^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$'
    )
    result = await evaluator.evaluate(
        response="user@example.com",
        context=ExpectedResponseContext()
    )
    print(result)

if __name__ == "__main__":
    asyncio.run(evaluate())

The evaluator returns a tuple containing:

  • A binary score (0 or 1) indicating pattern match status
  • A list of explanations including:
    • Success message with the pattern if matched
    • Failure message with the pattern if not matched

​
When to Use

Use the Regex Evaluator when you need to:

  • Validate email addresses
  • Check phone number formats
  • Verify date formats
  • Ensure specific text patterns
  • Test URL formats
  • Validate structured data formats
  • Check for specific text content
  • Verify code snippets or commands