Language

1. Expected Language Evaluator: Checks if the response is in a specific expected language
2. Equal Language Evaluator: Checks if the response is in the same language as the question

​

Purpose

• Ensure responses are in the correct language
• Verify language consistency between questions and answers
• Validate multilingual content
• Check language requirements compliance
• Monitor language-specific responses

​

How It Works

​

Expected Language Evaluator

• Score: 1: The response is in the expected language
• Score: 0: The response is not in the expected language

​

Equal Language Evaluator

• Score: 1: The response is in the same language as the question
• Score: 0: The response is in a different language than the question

​

Usage Examples

​

Expected Language Evaluator

import asyncio

from trusttest.evaluation_contexts import Context
from trusttest.evaluators import ExpectedLanguageEvaluator


async def evaluate():
    evaluator = ExpectedLanguageEvaluator(
        expected_languages="Spanish",
    )
    result = await evaluator.evaluate(
        response="Hola, ¿cómo estás?",
        context=Context()
    )
    print(result)

if __name__ == "__main__":
    asyncio.run(evaluate())

​

Equal Language Evaluator

import asyncio

from trusttest.evaluation_contexts import QuestionContext
from trusttest.evaluators import EqualLanguageEvaluator


async def evaluate():
    evaluator = EqualLanguageEvaluator()

    result = await evaluator.evaluate(
        response="Hola, ¿cómo estás?",
        context=QuestionContext(
            question="¿Qué tal estás?"
        )
    )
    print(result)

if __name__ == "__main__":
    asyncio.run(evaluate())

• A binary score (0 or 1) indicating language match status
• A list of explanations including:
  • Success message with the detected language if matched
  • Failure message with both detected and expected languages if not matched

​

When to Use

• Ensure responses are in the correct language
• Verify language consistency in conversations
• Validate multilingual content
• Check language requirements
• Monitor language-specific responses
• Ensure proper language handling in chatbots
• Validate language-specific content generation