An LLM judge reads the response and the evaluation context, then returns pass or fail. Use it when a regex or n-gram score cannot capture the criteria. It costs an extra model call and can vary slightly between runs.
Correctness
Factual accuracy against an expected response.
Completeness
Coverage of the expected response.
Tone
Tone and style against the criteria you set.
True / False
Pass/fail from
ObjectiveContext. Default for most catalog threat probes.URL correctness
Whether URLs in the response are valid for the task.
Answer relevance
How well the answer addresses the question (
QuestionContext).RAG poisoning
Whether the answer follows poisoned retrieval context.
Custom
CustomEvaluatorExpected or CustomEvaluatorObjective. There is no CustomEvaluator class.Prefer an LLM judge over a heuristic when the criterion is semantic. Prefer a
heuristic when you need a cheap, deterministic check.