Skip to main content

TrustTest is an AI red-teaming and evaluation toolkit. Point it at a model, agent, or HTTP endpoint. Generate or supply test cases, score them, and keep the results so you can rerun the same assessment after every change.

​
How it works

1

Connect a target

HTTP, an LLM provider, or a custom target.
2

Create tests

Built-in probes, your datasets, or a knowledge base.
3

Evaluate

Heuristics and LLM-as-judge, with a strategy you can rerun after each change.

Install

Python package, credentials, and optional extras.

Run an evaluation

A dummy target first, then connect a real model.
TrustTest flags a jailbreak attempt in an AI conversation

​
Two kinds of test

Red teaming

Jailbreaks, prompt injection, data leaks, unsafe outputs, and agent misuse.

Functional evaluation

Does the system still answer correctly, stay on-topic, and meet your criteria?
Red teaming does not replace functional evaluation. Run both against the same target when you need security evidence and product quality in one plan.

​
FAQ

After material changes to the model, system prompt, tools, retrieval data, or the surrounding application.
Yes. FileSystemClient writes JSON locally. NeuralTrustClient persists the same artifacts in the platform. See Connect to NeuralTrust.
The Python package supports HttpTarget and custom targets. The web UI currently requires an LLM exposed over REST. Multi-turn probes need a ConversationTarget; HttpTarget already provides that.

Concepts

Test cases, probes, evaluators, and scenarios.

Tutorials

Local models, RAG, compliance, and custom judges.