In TrustTest, test sets are collections of input-output pairs used to evaluate the performance and reliability of AI models. These test sets serve as benchmarks to systematically assess how well models handle various scenarios and requirements.

​
Key Areas

Predefined Test Sets TrustTest comes with a variety of built-in test sets designed for common evaluation scenarios:
  • Off-topic use cases.
  • Data leakage.
  • Responsibility.
Automatic Test Generation TrustTest provides tools to automatically generate test sets through:
  • Knowledge base.
  • LLM-assisted test creation.
Custom Test Sets Users can create their own test sets by:
  • Defining specific input-output pairs.
  • Importing existing datasets.

​
Why It Matters

  • Comprehensive Evaluation Test sets provide a structured way to evaluate models across different scenarios and requirements.
  • Reproducible Testing Predefined and automatically generated test sets ensure consistent evaluation across different runs and environments.
  • Efficient Testing Automated test generation saves time and resources while maintaining test quality and coverage.
  • Customizable Assessment The ability to create and modify test sets allows for tailored evaluation based on specific needs and requirements.
  • Quality Assurance Well-designed test sets help identify model weaknesses and areas for improvement, leading to better overall performance.