Skip to main content
Applies to LLM. One policy type doing two unrelated jobs; the console asks which before showing fields. Both run on the request only, before it reaches the model. Neither touches the response. The two jobs answer different problems, and almost nobody needs both at once. Read the one that describes yours.

​
The two jobs

​
Inject a system prompt into every request

Every application that calls a model carries its own system prompt, so the organisation’s rules — tone, what the assistant may not do, what it must disclose — live in a dozen codebases and drift apart. Injecting the prompt at the gateway puts one copy where every request passes, and changing it needs no release. Your platform team writes the prompt; the gateway injects it; callers send their requests unchanged and never see it. The parts that vary per request are {{placeholders}}, each mapped to a Header or a JWT claim — a field inside the bearer token the caller sends — that the gateway reads as the request arrives: a tenant id from X-Tenant-Id, a plan tier from a claim. A placeholder with no source cannot be saved.

​
Publish a prompt library clients invoke

A prompt is part of the product and changes more often than the client does. Publishing it from the gateway lets the people who own the prompt ship it without a deployment, and lets a client pin the version it was tested against. Prompts live on the gateway and change without a client release. A client references one by name in a plain user message — {template://support-greeting}, or {template://support-greeting@stable} to pin a label — and the gateway renders it in place of what the client sent. Placeholders are filled either by the calling application, which sends the values along with its request, or by a header or claim the gateway reads.
One policy can hold both, and then both run. The console edits one job at a time and saves both, so opening the panel on one never deletes the other — a callout says how many entries the job you cannot see is still carrying.When both are configured the library renders first, replacing the conversation, and the injected prompt is then merged into the rendered system prompt.

​
Configure the policy

Open Policies → Library → Prompt Template. It sits under Prompt Management.

​
What do you want to do?

Two radio cards, and the whole form follows from this choice:

​
Content, and the variables it creates

Both jobs are built from the same editor. Add template appends a card, and Content is the prompt body. You do not declare variables anywhere. Write {{tenant}} in Content and a row appears under it for that placeholder, asking where the value comes from. The rows are a consequence of what you typed rather than a second list to keep in step. Each row has a Source and, for the gateway-side sources, the header or claim name to read: A placeholder name must be letters, numbers, dots, hyphens or underscores. {{ tenant }} with spaces around the name is fine; a space inside the name is not, and the save is refused. In the injection job every placeholder must name a gateway-side source — there is no client to ask. In the library job Sent by the client is a legitimate answer, and the usual one.
A JWT claim is read without verifying the token’s signature. It is trustworthy only when the caller authenticates with that same bearer token. An application that authenticates with an API key, or with mTLS, can put anything in an Authorization header, and the claim you read is whatever it put there.Never make an authorisation decision on a claim unless bearer auth is the only way in. The console raises this inline the moment you select JWT claim, and it means it.
A variable none of your templates mentions any more shows no row, and the console does not delete it either. Something other than this form most likely put it there, and dropping a setting you cannot even see, on an unrelated save, would be configuration loss with nothing to show for it.

​
Injection: if a variable is missing

Under Advanced Settings, one select — If a variable is missing. It decides what happens when a header or claim an injected template needs is not on the request.
A new policy and an existing one do not open on the same option, and that is deliberate. A policy you create here pre-selects Skip this template — the caller cannot fix a header it was never asked to send. A policy that came from somewhere other than this form, with nothing stored for this setting, falls back to the gateway’s own default instead: Reject request.Opening an existing policy here does not flip it: a policy that already has templates and no stored value is shown as Reject request, so saving an unrelated change cannot quietly turn a fail-closed policy into a fail-open one. Look at the select before you save, either way.
A skipped template looks like success to everyone except the event. The caller gets 200 and a perfectly ordinary answer; the model simply never received your prompt, and nothing in the reply says so.If the prompt carries a rule you rely on, Skip this template is the wrong choice, and traffic will never tell you. Alert on the skipped decision instead.

​
Prompt library: templates, versions and labels

Each card is one named template. A template written in the console carries one version, labelled latest, and Default label is pre-filled with latest so {template://name} resolves with no label at all. That is the whole versioning story the console tells. A template that already carries several versions keeps every one: only the first is editable, and a callout says how many more there are and that they are preserved. Content may be a plain string, which the gateway sends as a single user message, or a list of messages written as JSON, which becomes the conversation verbatim:
Content that starts with [ but is not a valid list of messages is refused on save.

​
What the caller sends

The reference goes in the content of a message — the gateway scans message content and nothing else. The values for a template’s placeholders travel with the same request, sent by the calling application; how they are put there is a question for whoever builds it, not something you set in the console.
In an OpenAI-shaped request the application’s own system prompt is a message too, and a reference in it is found. The gateway reads the content of every message without caring what role it carries. A {template://…} an application left in its own system prompt therefore resolves like any other — and rendering it replaces the whole message list, system prompt included. Formats that keep the system prompt in a separate field — Anthropic’s system, Gemini’s systemInstruction, Responses’ instructions, Bedrock’s system — keep it outside the conversation, and it is never scanned.Worth checking before you publish a library: a reference sitting in an application’s system prompt for its own reasons is not inert.
Those values never reach the model. The gateway takes them off the request before it goes upstream — in Observe too, and even when the policy has no library configured at all. No provider ever sees them.
Rendering a named template replaces the entire message list. The conversation history the client sent is discarded before the model sees anything — not only the message carrying the reference, all of them. This fits single-turn calls. The event reports how many turns were dropped, because the caller cannot see it happen: the answer just gets worse.A multi-turn client that resends the same reference every turn is not an error — repeating one reference is explicitly allowed — but each turn still arrives at the model as a fresh, historyless conversation.

​
Prompt library: Advanced Settings

A template reference is required either way, whatever created the policy. A policy you create here has Require template reference on and refuses a request that names no template. A policy that came from somewhere other than this form, with nothing stored for this setting, refuses those requests too: on the gateway side, nothing stored means the reference is required. Open such a policy here and the toggle reads On, which is what it has been doing all along.There is no path on which an untemplated request quietly passes because nobody set this.
In the library job, a value the gateway resolved wins. A placeholder you sourced from a header or a JWT claim is filled from that header or claim, and a value the caller sends under the same name does not displace it. A caller cannot scope a prompt to someone else’s tenant by guessing what you called the placeholder.Which source you choose is therefore the decision that matters, and it is a real one. A gateway-resolved placeholder is exactly as trustworthy as the header or the claim behind it — read the warning about JWT claims above before you lean on one. A placeholder you leave for the caller to fill is, by definition, the caller’s to choose: do not put anything in it that would be an authorisation decision.The injection job never reads anything the caller sent at all. A placeholder there resolves from the header or the claim and from nothing else.
Values are escaped before they are substituted into a template written as a message list, so a caller cannot break out of the message its value is placed in and append one of its own. A value that tries lands inside the message it belongs to, ugly and inert.

​
Settings the form does not show

A policy can carry more settings than this form draws. The gateway honours them, and the panel neither shows them nor destroys them, so a policy that has them keeps them through every edit you make here. What you cannot do is set them yourself: none of the rows below has a control in the form, so changing one is something to ask whoever manages your gateway for.

​
Mode and scope

Enforce and Observe. Read Modes and failures before you rely on Observe: it is narrower than it sounds, and for this policy it is also close to useless as a dry run, since nothing is injected and nothing is rendered. There is no leg to choose. The policy runs on the request on its way to the model, and never on the answer coming back. Scope works as it does for every policy — Requests from, for applications and groups. See Policies overview.

​
Supported request formats

The policy edits each chat format in the field its model reads, so the same prompt reaches the model whichever route the application calls. A role a format cannot carry is forwarded without that injection, and the event records it as not applied. Observe never blocks on it. A named template is found in each format’s own conversation field. On Anthropic, Gemini, Responses and Bedrock its system text is merged into the request’s system prompt; on OpenAI-compatible formats the rendered messages replace the conversation. Requests that are not chat — embeddings, files, images, audio, rerank — pass through untouched. The exception is a policy with Require template reference on: such a request names no template, so it is refused with 400 in Enforce. A chat request the policy cannot edit safely is refused in Enforce. When templates are configured and the body is one the policy cannot rewrite without risking a wrong result — an unusual spelling of a key, a duplicated key, a system field it cannot read, malformed messages — the gateway answers 400 rather than forward the request without the prompt. In Observe the request goes through unchanged.

​
How the decision is made

1

The request is read, and the caller's own values are taken off it

The request has to be readable as a structured body. Any values the caller sent for placeholders are taken off it here, before anything else happens and regardless of mode, so they never reach the model.
2

The library runs first, if one is configured

Message contents are scanned for {template://…}. No reference, with Require template reference off, and the library does nothing. A reference resolves to a template and a label, its required variables are checked, and the rendered result replaces the whole message list.
3

Then the injected prompts are applied, in the order listed

Each template is rendered from its variables and merged into the system prompt — the application’s own system prompt if the request carries one, otherwise a new one. Where the system prompt lives depends on the request format; see Supported request formats.Merge appends after a blank line, and the application’s own system prompt comes first.
4

The rewritten request goes upstream

If nothing changed, and the caller sent no values of its own, the request is forwarded byte for byte. Anything the policy does not model is preserved either way.
Every rejection says which of these it was, so a colleague reporting one tells you where to look:

​
Modes and failures

Observe does not mean “never refuses”. Two paths refuse the request in Observe exactly as they do in Enforce, because they happen before the mode is ever read:
  • a request the gateway cannot read, including an empty one;
  • a stored configuration it cannot parse.
Both reach the caller as 502, with nothing in the response naming which policy did it. The second matters more than it sounds: a policy saved through the console cannot reach that state, but one that came from somewhere else can, and it then fails every request in its scope in either mode.
Observe is not a dry run for this policy. It records what it would have done and forwards the original, which for a rewriting policy means the model answers without the prompt, the library reference is never resolved, and the caller’s message reaches the model with the literal text {template://support-greeting} still in it. Useful for counting how often a template would fire; useless for seeing what it does.

​
Limits

  • Request only. There is no response leg, and no leg to choose.
  • Chat formats only. The policy edits the chat formats listed in Supported request formats. Anything that is not a chat request passes through untouched.
  • A template reference is found in the content of any message, whatever role that message carries. In an OpenAI-shaped request the application’s own system prompt is a message, so a reference left there is found and rendered like any other. Only a separate system field, on the formats that have one, is never scanned; a reference there is forwarded to the model as literal text.
  • On OpenAI-compatible and Anthropic requests, a reference inside structured content is not found. When a message’s content is sent as a list of parts rather than as plain text, the gateway does not look inside it. Nothing is rendered, nothing refuses the request, and the model receives the literal {template://…} text. The event does record that the request carried a reference it did not read, so it can be alerted on. Gemini, Responses and Bedrock content is always a list of parts, and their text parts are read.
  • One template per request. Two different references are a 400; the same reference repeated across turns is fine.
  • Rendering discards the conversation. See the warning above.
  • Merging is the only thing this form can choose. A prompt you create here is added to the application’s own system prompt, and no control offers anything else. That is not the same as normalising: a policy already set to replace that prompt keeps replacing it, and saving from the console will not change it back.
  • A template you build here validates no client input. Required Variables has no editor, so type, allowed-value and length checks are out of reach from the console, and a value the caller sends that matches no placeholder is ignored rather than refused.
  • Labels are effectively fixed at latest in the console. Real versioning — several labelled versions, a client pinning one — is beyond what the form can edit. It preserves what is already there and declines to change it.
  • Mustache is the only template engine implemented. A policy naming the other one is refused on save.
  • LLM traffic only. The policy does not apply to MCP (Model Context Protocol) tool calls.

​
Verify the policy

Start in the Playground, and expect to leave it after the first step. The Playground sends a model and a message history, and nothing else — no header of your own, no bearer token of your own, no values of your own. It can prove the path works, and it can prove an injected template that has no placeholders. It cannot prove anything that depends on a value the gateway reads off the request, because the Playground never sends one.Everything after step one therefore has to come from whatever already calls this gateway: your own application, sending the header, the token or the values the template needs. Arranging that is your developers’ side of the work, and this page does not cover it — what it can tell you is exactly what to look for once such a request arrives.
1

Control — the path works at all

Open the Playground, pick the application the policy applies to, and send:
Expect a normal reply. If this fails, the problem is the application, the model or the provider — not the policy. Fix it before going on.If your injected template has no placeholders, this is also the only check you need for it: ask the model what its instructions are, here, and read the answer.
2

Injection — the prompt reaches the model

A template that reads a header needs a request that carries one, so this is where you leave the Playground: have your application send a request carrying the header your template reads, and ask the model what its instructions are.Expect the event to record injected.Read the reply, not just the status. The model must answer as though it had been given the prompt — naming the tenant, if that is what your template interpolates. That is what proves the injection happened rather than merely being recorded.
3

The missing header — you get the behaviour you chose

Have the same request sent again, this time without that header.With Reject request, expect 400 and a refusal saying the variable could not be resolved. With Skip this template, expect 200, an answer that does not reflect the prompt, and the event recording skipped. Check which one you get: this is the setting a new policy and an existing one do not agree about.
4

Prompt library — the template renders

The caller’s own values are something the Playground has no way to send, so this one comes from your application too: a request whose message carries the reference, supplying whatever the template’s placeholders need.Expect the event to record rendered and to name the resolved template. The reply must answer the template’s question, not the literal reference text. If the model says something about a template:// address, the reference was forwarded instead of resolved — check the mode, then, on an OpenAI-compatible or Anthropic request, check that the message’s content was sent as plain text rather than as a list of parts.
5

Prompt library — history really is discarded

Have that request sent again with two earlier turns of conversation before the reference.Expect rendered again, and the event to report the discarded turns. Then ask the model about something from the turns you sent: it will not know. That is the point of the check — this is the behaviour most likely to surprise an application team after it ships.
Every decision lands on the request itself, in Observe mode too. Open Activity, open the request, and read its Policies tab — the Prompt Template row is there with the decision it reached. The event schema lists the full set of fields a decision records.

​
Troubleshooting