NeuralTrust | Platform for Agent Security.

AnswerRelevanceEvaluator is an LLM judge that scores how well the answer addresses the user’s question. It uses QuestionContext (no gold answer required). It is used in RAG examples and was missing from the judge inventory. A case fails when score < threshold. Default threshold is 3 on a 1–3 scale, so only a high-relevance score passes.

How It Works

Score	Meaning
1	Mostly unrelated, ignores or contradicts the question
2	Partial / mixed relevance
3	Directly answers the question

Usage Example

import asyncio

from trusttest.evaluation_contexts import QuestionContext
from trusttest.evaluators import AnswerRelevanceEvaluator


async def evaluate():
    evaluator = AnswerRelevanceEvaluator()
    result = await evaluator.evaluate(
        response="Vic is the capital of Osona, in Catalonia.",
        context=QuestionContext(question="What is the capital of Osona?"),
    )
    print(result)

if __name__ == "__main__":
    asyncio.run(evaluate())

Parameter	Type	Default	Description
`threshold`	`float`	`3`	Minimum passing score
`language`	`LanguageType`	`"English"`	Language of judge reasons
`llm_client`	`LLMClient`	From `set_config` evaluator	Optional override
`name` / `description`	`str`	Built-in	Display metadata

When to Use

RAG functional tests without a gold answer
Checking that retrieval-backed answers stay on topic
Combining with Correctness or Completeness when you also have ExpectedResponseContext

Getting Started

Core Concepts

Connect your app

Create tests

Evaluate results

Answer Relevance

How It Works

Usage Example

When to Use

​How It Works

​Usage Example

​When to Use

How It Works

Usage Example

When to Use