Content security detectors

Content-security detectors are the LLM-aware core of TrustGuard. They score prompts for jailbreaks, and prompts and model output for toxicity and off-topic content, and they screen documents and URLs for indirect prompt injection and PII. Create them in the console under Detectors, then attach them from a policy to decide what happens when they match. All of them are detection-only. The action — Monitor, Block — and the phase — Input, Output — are set on the policy rule that references the detector, not on the detector itself.

​

Protection Sensitivity

Most content-security detectors have exactly one setting, and it is the same control everywhere: three preset cards. You pick one; there is no number to type. Balanced is applied by default, and it is the right answer far more often than people expect.

​

What actually changes

The preset moves one thing: how sure the detector has to be before it says something. It does not change what the detector looks for, and it does not make it smarter — Strict does not catch a different class of attack, it catches the same class earlier and less certainly. That is why the trade runs in both directions:

• Permissive only reports what it is confident about. You will miss borderline attempts, and almost never argue with a finding.
• Strict reports things it is unsure about. You will catch more real attempts, and you will also flag someone quoting an article about prompt injection, a developer testing your agent, or a support ticket that happens to contain the word “ignore”.

​

Choosing one

Start on Balanced and leave it there until Activity gives you a reason to move. The reason should be evidence, not caution: Going straight to Strict “to be safe” is the most common mistake here, and it backfires in a specific way: the findings are not wrong so much as numerous, the team stops reading them, and the one that mattered scrolls past unnoticed. A detector nobody trusts is worse than one tuned slightly loose. The provider behind these detectors and its credentials are not part of this form. They are configured once for the workspace.

​

Create a detector

The same four steps for every detector on this page:

1. Open Detectors and click New detector — or open the Catalog tab and click Add detector on the card you want.
2. On Basics, give it a Name. Use one that says how it is tuned, not what it is: Jailbreak — strict beats Prompt Guard 2, because the type is already shown next to it.
3. On Configuration, set what that detector exposes (below).
4. Click Create detector, then add it to a policy rule with an action.

A detector on its own inspects nothing. Until a policy references it and that policy is attached to a collector, it is inert.

​

Prompt Guard

​

The risk

Someone talking the model out of its instructions. “Ignore your rules and…”, role-play framing that makes the rule-breaking hypothetical, instructions hidden in another language or spelled out character by character to slip past a filter looking for phrases. What makes this worth a detector rather than a keyword list is that the successful attempts rarely look like the famous ones. They are rephrased, translated, or buried, and the wording that works changes faster than any list you would maintain.

​

What it looks for

Attempts to override the instructions the agent was given. Direct ones — “ignore everything above” — but mostly not: the same intent framed as a story or a hypothetical, written in another language, spelled out character by character, encoded, or simply reworded until it no longer resembles anything a filter would recognise. It judges intent rather than matching phrases, which is the point. A keyword list catches the attempt you have already seen; the wording that works changes faster than you can maintain one.

​

Where it looks

Input only. Every user turn in the request, and assistant turns replayed in that request. Each segment is scored on its own and the highest score wins, so an injection in an earlier turn still blocks when the latest message is benign. System and developer instructions are not scored, including Responses instructions, Anthropic system, Gemini systemInstruction, and Bedrock system. An Output rule is skipped and has no effect. The console warns: “This detector does not run on Output; this rule has no effect.” It does not read tool definitions or tool results. An instruction that arrives inside a search result, a ticket, or a tool description is a different problem with a different detector — Indirect Prompt Injection.

​

Configure

Protection Sensitivity, and nothing else. Start on Balanced. Move to Strict when you see attempts getting through in Activity — not before. Strict on a public chatbot will also flag people quoting an article about prompt injection, or testing your agent out of curiosity, and a stream of those trains everyone to ignore the findings.

​

Toxicity Detection

​

The risk

Two different problems that happen to share a detector. On the way in, abuse of your service: someone using your agent to generate hateful or harmful material, or simply abusing it. That is a conduct question, and the useful response is usually to record it against the consumer rather than to block a single message. On the way out, what your agent is willing to say. A model that produces something hateful, graphic, or harmful in a customer-facing reply is a far more expensive problem than the same text arriving as input, and it is the direction people most often leave unguarded.

​

What it looks for

Toxic language — abusive, hateful, degrading or otherwise harmful. It answers one question: is this toxic. There is no breakdown by kind, so a finding tells you that a message crossed the line, not which flavour of harm it was. If you need to separate, say, hate speech from violent content, that is Custom Moderation, where you define those subjects yourself, not this detector.

​

Where it looks

Input: the message, before it reaches the model. Output: the reply, before it reaches the person. Reading the findings in Activity is therefore about who and how often rather than what kind — the useful pattern is a single consumer producing them repeatedly, or a rise after a prompt change.

​

Configure

Protection Sensitivity, and nothing else. Consider running it in both directions but with different policy actions: Monitor on input, where you mostly want the record, and Block on output, where the cost of one bad reply is real.

​

Custom Moderation

​

The risk

A subject specific to your business — a competitor’s product line, an internal project name, a regulated matter particular to your sector, a product you discontinued and do not want discussed. It answers one question per subject: is this the topic I described. That makes it scope control rather than safety — nothing here is an attack, it is your agent working correctly on something you never signed up to answer for.

​

What it looks for

Whether the text belongs to a subject you described — judged against the words in your definition, one answer per topic you defined. Each topic is judged on its own: the other topics on the detector never change its score.

​

Where it looks

Input: the message, before it reaches the model. Output: the reply, before it reaches the person — worth enabling, because scope is as easily broken by what your agent says as by what it is asked. Long text is scored in overlapping windows and each topic keeps its highest score, so a subject near the end of a long message is not missed. A plain-text message is read up to its first 10,000 characters; anything past that is not inspected. When a message carries structured JSON, each text field inside it is scored on its own, so a subject in one field is not diluted by the rest. A finding reports the topic that matched and its score. On the coding-agent and agent platform collectors it sees the user’s latest message only, so a subject that comes up purely inside tool activity is not what this detector is for.

​

Configure

Every topic here is yours — its name, and crucially its definition. One detector holds up to five of them. Each topic adds evaluation time, because every message is checked against each one separately. If you need more subjects, add another Custom Moderation detector. The five count everything on the detector, whether you picked it from the list below or wrote it yourself. You do not start from a blank page. Thirteen common subjects come ready as editable drafts — pick the ones that matter to you: Add the ones that apply to your agent and Customize them, or Add new topic and write your own. Either way what gets saved is your version — a preloaded subject you never edit is still yours to tighten later, and the wording you leave in it is the wording that gets scored. Each topic opens a form in two parts: Then set Protection Sensitivity. It applies to every topic you defined rather than per topic, so to watch one subject more closely than the rest, give it its own detector with a stricter setting.

​

Writing a good definition

Because each topic is judged on its own, a definition only has to say what belongs. It never has to rule out the other topics. Do

• Describe what you want caught — the content or requests that belong to the topic. A match is what gets flagged, so describe the subject to keep out, not the scope your agent should stay within.
• Write one plain sentence. A sentence works a little better than a bare list of keywords.
• Keep it short — under about 50 words, or 300 characters; the limit is 400. A longer definition doesn’t make matching less accurate, but it makes long messages slower to check.
• Write it in the language of your traffic when most messages are in one language. Messages in other languages still match, but they score lower, so fewer of them cross the threshold.

Don’t

• Don’t name a neighbouring topic to carve it out — ”…, not X”, “unlike X”, “rather than X”. It is the one wording choice we measured to make matching worse. If a neighbouring subject keeps matching, describe your topic more narrowly instead of listing what to exclude.
• Don’t paste example messages into the definition. On one topic this made short system messages match by mistake.

The same scope, worded both ways: Each bad version names neighbouring subjects to exclude them. The good versions only say what belongs. Built-in drafts are already tuned. Their wording was chosen by testing it against the detector, even where it doesn’t follow these rules. Keep it unless you need a different scope; the rules above are for topics you write or rescope. Personal data is a different detector. To catch identity numbers, bank accounts or contact details, use Data Loss Prevention. Custom Moderation judges what a message is about, not whether it contains a kind of data. After changing a definition, treat it as a new one: try it on sample messages in the policy’s Test tab, then watch it in Activity with the policy in Observe — short and automated messages included — before you enforce it. These recommendations were tested on English, Catalan, German, Spanish, French, Galician, Italian, Portuguese and Turkish.

​

URL Analyzer

​

The risk

An instruction hidden in a page the agent is asked to read. The link looks harmless — a documentation page, a ticket, a shared doc — and the payload is on the other side of it. Nobody in the conversation typed anything suspicious, which is exactly why a prompt-side detector never sees it. The same fetch also picks up personal data: a page the agent pulls into its context can carry someone’s details into a model that was never meant to see them.

​

What it looks for

Two things, in the text of the page it fetched: instructions aimed at the agent, and personal data. The URL itself is not what is being judged — a perfectly ordinary link to a page someone edited is the whole problem.

​

Where it looks

Links in the message, Input side only, at the moment the message is evaluated — before the model sees it. The page is fetched right then and scored as it reads at that instant, so a page that changes after you tested it is a different page. Two consequences worth planning around. Links inside an uploaded file are not covered here — that is Document Analyzer. And fetching happens at evaluation time, so a page that changes after you tested it is scored as it is now. Fetches run under an SSRF guard: HTTPS only, with internal, private and cloud-metadata addresses refused, so a link cannot be used to make TrustGuard reach into your network. A refused address is reported as ssrf_probe and is not fetched.

​

How many links it fetches

URL Analyzer fetches at most 10 unique URLs per request. Repeated copies of the same link are deduplicated first, so they do not count toward that cap, and a message that repeats one link never overflows. When a request carries more than 10 unique URLs, the surplus is never fetched or scored. The request is a finding with signal type url_limit_exceeded, and the detector’s action applies: blocked in Enforce, reported in Observe. A detection among the first 10 URLs keeps its own type. Precedence is jailbreak, then pii, then ssrf_probe, then url_domain_blocked, then url_limit_exceeded. Evidence on the finding always includes urls_total, urls_skipped, and url_limit_exceeded. With no overflow those are 0, 0, and false.

​

Configure

This detector does not carry its own sensitivity or entity list. Instead you point it at two detectors you have already created: If you have neither yet, the console offers Create a Prompt Guard detector and Create a Data Loss Prevention detector inline. Optionally restrict Domains with an allow list, a block list, or both. An empty allow list permits every domain that is not blocked. A domain the lists reject is reported as url_domain_blocked. The number of links fetched is fixed at 10 unique URLs per request. See How many links it fetches.

​

Document Analyzer

​

The risk

An instruction hidden inside a file someone uploads. A CV with white text on a white background telling the screening agent to recommend the candidate; an invoice carrying a line aimed at the assistant rather than the reader. This is content risk, not malware scanning. A file can be perfectly clean by every antivirus measure and still carry a hostile sentence — that sentence is what this detector is for. It does not tell you whether a file is safe to open.

​

What it looks for

The same two things as URL Analyzer, in the text it pulls out of the file: instructions aimed at the agent, and personal data. Not viruses, not file structure — what the document says.

​

Where it looks

Files attached to the message, Input side only, before the model reads them. It extracts the text first — PDFs, Office documents, plain text, and images through OCR — and screens what it extracts. What happens when that extraction produces nothing is in When a file cannot be read. OCR is the part that matters: an instruction rendered as a picture, or inside a scanned page, is invisible to anything reading the file as text. It is also why the same attack in a screenshot is caught here and nowhere else.

​

When a file cannot be read

Screening covers the text the extractor returns. When there is nothing to screen, the result depends on why. If the extractor isn’t available (no capacity or not configured), the file is allowed; if parsing the file fails or times out, it’s treated as uninspectable. A PDF that is blocked and contains nothing harmful could not be parsed: it is corrupt, truncated, or a format the extractor rejects, or parsing it timed out. An attachment that is too large to inspect is a separate finding, signal type guard_payload_too_large with reason payload_too_large.

​

Configure

The same linked-detector model as URL Analyzer: select an existing Prompt Guard detector for injection screening and an existing Data Loss Prevention detector for PII. The warning above applies here too. There is no sensitivity setting of its own. How strict this detector is comes from the Prompt Guard detector you linked, so the way to tighten document screening is to raise that detector’s Protection Sensitivity — which also tightens everything else pointing at it. Nothing else to set. OCR and injection screening are always on — there is no switch to forget, and no file-size or language list to tune.

​

When to use

• Prompt Guard is the baseline for any chat traffic. If you deploy one detector, deploy this one.
• Toxicity Detection on output when your agent talks to the public; on input when abuse of the service is the concern.
• Custom Moderation for scope control — keeping your agent off the subjects you describe for your business.
• URL Analyzer and Document Analyzer for RAG and agent flows that ingest links or files.
• Tool results are a different detector: Indirect Prompt Injection.

Start every one of them with the policy in Observe. Move to Enforce once Activity shows you are catching what you expected and not much else.