Skip to main content
Applies to LLM and MCP — the Model Context Protocol your agents call tools over. An agent that fails to parse a tool result tends to call the same tool again, and again. Some tools are cheap to repeat; a code executor, a search billed per call or a tool that sends messages is not. This caps how many times a given tool may actually run in a window, however many requests it took. Counts actual tool executions, not HTTP requests, and caps them per tool pattern over one or more windows — execute_code* at 60 a minute and 500 an hour. The first rule whose pattern matches wins. That choice — executions rather than requests — is what makes the policy worth having and also what makes it partial. The gateway can only count an execution it sees. Read Limits before you rely on this for anything expensive.

​
Configure the policy

Open Policies → Library → Per-Tool Rate Limiter.

​
Rules

Add rule appends a rule card. Rules are evaluated in order and the first whose tool pattern matches wins — a later rule for the same tool is dead. Put the specific patterns above the catch-all. A rule with several windows enforces all of them: 60 a minute and 500 an hour means a burst is stopped by the first and a slow grind by the second.
The windows are fixed, not sliding. A window opens when the tool’s counter is first charged and closes exactly its duration later, at which point the counter disappears and the whole allowance returns at once.With 10 per hour, if the counter opened at 10:00 it closes at 11:00. Ten executions at 10:59 exhaust it — and ten more at 11:01 are all allowed, because by then the window is a new one. Twenty executions in two minutes, inside a policy that reads as ten an hour. Size the window for the burst you can tolerate, not the average you want.

​
Behavior when exceeded

Three choices. On MCP only the first exists. Default behaviour at the bottom of the form applies to every rule left on Use default behavior. It starts at Reject.
Inject error silently becomes Strip tool on a streaming request. The rewrite it wants to make happens on the response, and a streamed response cannot be rewritten — tokens are already leaving. So on a streaming request the gateway falls back to withdrawing the tool from the request instead. The effect is similar and the agent’s experience is not: it gets a model that never proposes the tool, rather than an explanation of why its call was refused.
A policy attached to an MCP application is refused at save time unless every behaviour is Reject. On MCP the gateway is the caller: the client asked for one tool by name and there is no conversation to edit, so Inject error and Strip tool have nothing to act on. Rather than accept the setting and quietly enforce something else, the association is rejected and the error names the rules to change.

​
Scope

This form has no scope of its own. The policy’s own scope is the control — the Requests from and Resources you set alongside the rules. Resources decides which MCP servers and tools the policy runs on at all; Requests from decides how the counters are partitioned. That choice is not cosmetic. One shared counter is an organisation-level ceiling on a tool nobody can opt out of; one counter per application keeps a single misbehaving agent’s problem to itself. Decide which of the two you want before you set the numbers, because the same limit means something different under each — and One execution, counted twice, below, is a case where it changes the answer outright.

​
Mode

This policy has only one mode, and the console shows no mode selector for it. It always enforces. There is no Observe, so there is no way to find out what it would do before it does it. Roll it out with a generous limit, watch what it does on the Policies tab of the requests in Activity, and tighten from there — that is the nearest thing to a dry run this policy offers.

​
How the decision is made

The counter is charged when the gateway observes an execution, and it is checked on the next request. The two are separate moments, and which request is which depends on whether the traffic is chat or MCP.
1

On LLM, an execution is a tool result coming back

The gateway reads the incoming conversation, finds the most recent turn in which the assistant proposed tool calls, and looks at the messages after it. Each result answering one of those proposals is one execution, and charges every window of the matching rule.Each execution is charged once. The gateway recognises a result it has already counted — an agent retrying, or re-sending its history — and ignores the repeat for as long as the rule’s longest window.
2

On MCP, an execution is the tool call itself

The gateway is the caller, so it knows the call really ran. Every MCP tool call that reaches the upstream charges the windows, whether or not it was tied to the model’s own tool call.
3

The tools the request declares are checked against their counters

On LLM, only the tools the request declares are checked. On MCP, the single tool the call names.
4

A tool at or above any of its windows triggers its behaviour

Reject refuses the whole request. Strip tool removes that tool from the request and lets the rest through. Inject error lets the request reach the model and removes the tool call from the answer instead.

​
One execution, counted twice

A tool an agent runs through MCP shows up twice at the gateway: once as the MCP call that executes it, and again as the result the agent reports on its next LLM turn. When Requests from is All traffic both hit the same counter, so one execution is charged twice. Narrowing Requests from to Applications is the fix you can make from the console, and the one to reach for: there the two sides keep separate counters and the problem does not arise at all. The other fix is not a console setting — it is a change inside the agent, so this part is for whoever maintains it rather than for you. The agent can tell the gateway that the MCP call and the tool result are the same execution by sending the identifier the model gave that tool call alongside the MCP call, in the call’s metadata under the key ai.neuraltrust/toolCallId. The identifier has to match the model’s exactly. One longer than 128 characters, or carrying anything but letters, digits, hyphens and underscores, is quietly ignored — a malformed one costs you the pairing, not the call. Only the LLM side can be left uncharged this way, and only by an identifier the MCP side already claimed. The MCP call itself always charges. Reusing one identifier across several real MCP calls buys no free executions: each of those really ran, and each is charged.

​
What the caller receives

Reject on LLM traffic is 429 with these headers, where <scope> is global when Requests from is All traffic and consumer when it names applications or groups: and a refusal naming the tool that ran out of allowance. On MCP traffic it is an error carried inside a 200 response, under the code -32004, with the same headers. The upstream tool is never invoked. Strip tool and Inject error both return 200. Nothing tells the caller a policy acted, apart from the record in Activity — and, for Inject error, the assistant message the gateway wrote.

​
Modes and failures

There is only Enforce. The column is here to be read against the other policies, not because there is a choice.
A counter-store failure allows the request on both the LLM leg and the MCP leg. The event records decision failed_open. There is no Observe mode, and no setting that refuses traffic because Redis is down. A client that has already disconnected is a different case: that error is not recorded as a store outage, and on the LLM leg the proxy can still answer 502.

​
Limits

  • No Observe mode. You cannot dry-run this policy. Start loose and tighten.
  • On LLM, only declared tools are enforced. The check reads the tools the request declares. A client that stops declaring a tool — or never declares one, because the model is being asked to answer in prose — is not checked at all, however exhausted that tool’s counter is.
  • On LLM, only reported executions are counted. The gateway learns a tool ran by seeing its result come back in a later request. A tool the agent executes and never reports through the gateway is free.
  • Only the latest turn’s results are counted. Results from earlier turns in the same conversation are skipped, so a history replayed after the window reopened is not re-counted — and a straggling result reported several turns late is not counted at all.
  • The windows are fixed, not sliding. See the note above.
  • Inject error cannot act on a streaming response, and silently becomes Strip tool.
  • On MCP only Reject exists, and a policy configured otherwise cannot be attached to an MCP application.
  • Reading a counter and charging it are separate steps, so a burst of simultaneous executions can overshoot a window slightly.
  • The counters live in the gateway’s shared store. If it is cleared, every window reopens.

​
Verify the policy

Set a rule to 1 per minute on a tool you can trigger on demand, then send these in this order. The counting model means you need three turns, not two, and the middle one is the one people forget.
1

Control — the path works at all

In the Playground, pick the application the policy applies to, and send a prompt that makes the model propose your tool. Expect a normal reply carrying the tool call. If this fails, the problem is the application or the model, not the policy. Fix it before going on.
2

Report an execution — the counter is charged

The Playground cannot charge the counter. The gateway only learns a tool ran when it sees the result reported back, and a result is something an agent sends, not something you type into a prompt box.So this middle turn has to come from whatever already calls this gateway: your agent running the tool it was just asked for, and reporting the result the way it normally does. Expect nothing to be refused on that turn — it is the turn that charges the counter. It is also the one people forget, which is why the check below so often seems never to fire.
3

The limit acts

Back in the Playground, send the first prompt again.With Reject, expect 429 and a refusal naming your tool. To confirm the request never reached the model, open Activity, click the request, open its Policies tab and read Aggregated Latency: the Provider row should be empty.With Strip tool, expect 200 and a reply that behaves as though the tool did not exist; the tool is gone from what the model received.With Inject error, expect 200, no tool call in the answer, and an assistant message naming the rate-limited tool.
4

If you run both chat and MCP — one execution, one charge

Set the rule to 2 per minute. Have your agent run the tool once over MCP, sending the model’s tool-call identifier with it as described above, and then report that same execution on its next LLM turn. A second MCP call must still be allowed: a budget of two owes one more execution.If that second call is refused, the two sides are not being recognised as one execution — the identifier is not arriving, or does not match the model’s exactly. That is agent-side, and not something the console can show you.
Every decision this policy takes is recorded. To read one, open Activity, click the request, and open its Policies tab: the policies chain lists this policy and the decision it made. The event schema is the reference for everything a decision records.

​
Troubleshooting