TrustTest is a comprehensive framework designed to rigorously evaluate and safeguard your AI models against security vulnerabilities, harmful behaviors, and unexpected outputs. Harness advanced red teaming techniques and comprehensive functional evaluations to build robust, secure AI systems.

​
What is TrustTest?

TrustTest functions as a specialized testing framework for evaluating and securing AI models and LLM workloads. While traditional testing frameworks focus on code functionality and performance, TrustTest takes on these responsibilities with a focus on AI-specific needs.

​
Key Features

  • Identify vulnerabilities before they reach production.
  • Evaluate model from all points of view with a versatile set of probes and evaluators.
  • Automatic test generation to evaluate model behavior in a wide range of scenarios.
  • Built-in State-of-the-art algorithmic Red Teaming attacks to test model robustness and safety.
  • Track, record, and analyze tests, runs, evaluators, and scenarios locally or via integrated platform.
  • Test any LLM with a unified interface, whether it’s your own model or a third-party API.

​
Why use TrustTest?

In today’s rapidly evolving AI landscape, ensuring the safety and reliability of LLM deployments is crucial. TrustTest offers several compelling benefits:
  1. Proactive Security: Catch potential vulnerabilities and safety issues before they impact your production environment.
  2. Continuous Testing: Automatically generate and evaluate tests to ensure your model remains secure and reliable over time.
  3. Comprehensive Testing: Access a wide range of pre-built probes and evaluators to test your models across diverse scenarios and edge cases.
  4. Flexibility: Test any LLM with a unified interface, whether it’s your own model or a third-party API.
  5. Structured Evaluation: Organize your testing process with a clear framework that separates test cases, evaluations, and scenarios.
  6. Traceability: Keep track of all your tests, evaluations, and results either locally or through the integrated NeuralTrust platform.
By using TrustTest, you can build more reliable and safer AI systems while maintaining a systematic approach to model evaluation and security testing.