Skip to main content
Smart routing is a load balancing strategy that classifies each prompt’s complexity and sends the request to the matching tier (registry + model). Use it when you want cost-efficient defaults for easy traffic and stronger models only when the prompt needs them. In the console it is the first option under Strategy → Load balancing.

​
Complexity labels

Every tier uses one of three fixed labels. Operators never set numeric thresholds — only these labels appear in the UI: Each label can be used at most once per consumer. You need at least two tiers (for example Simple + Hard, or Simple + Medium + Hard).

​
When to use it

​
How it works

  1. The client calls the consumer (typically with "model": "auto" — see Model resolution).
  2. TrustGate classifies the prompt into a complexity band: Simple, Medium, or Hard.
  3. It selects the tier configured for that label.
  4. The request is forwarded to that tier’s registry and model.
  5. If the pick fails and fallback is configured, TrustGate can retry another path.
Example configuration: A straightforward prompt lands on Simple; a demanding multi-step prompt lands on Hard. Tiers that share a registry with different models are supported and common.

​
Configure in the console

  1. Open Consumers → select a consumer → Routing.
  2. Set Routing mode to Static.
  3. Open Strategy → Load balancing → Smart routing.
  4. Open Complexity tiers.
  5. Add at least two tiers (the UI blocks save with fewer).
  6. For each tier set:
    • Complexity — Simple, Medium, or Hard (each label once).
    • Registry — upstream provider connection.
    • Model — model used when that band is selected.
  7. Save. Open Connect — snippets use "model": "auto".

​
Rules the UI enforces

You can point several tiers at the same registry with different models — that is supported and common (one OpenAI connection, mini vs full model per band).

​
Clients and auto

With smart routing enabled, applications should send:
The consumer Connect tab and Playground already use Auto for this strategy. Do not hard-code a single model id unless you intentionally want to bypass complexity selection (and the model is allowed on the consumer).

​
Tips

  • Put the cheapest capable model on Simple so most easy traffic stays inexpensive.
  • Use Medium for the default production model when you need three bands.
  • Reserve Hard for models that justify the cost on difficult prompts.
  • Pair with LLM Budget if you need hard spend ceilings on top of routing.
  • Watch Analytics (Cost / LLM) after rollout to confirm tier mix matches expectations.
  • Use Fallback if an upstream outage should retry another registry instead of failing the request.
  • Load balancing — other algorithms (round-robin, weighted, least-connections, random).
  • Model resolution — auto, short names, and model filters on the consumer.
  • Consumers — Routing tab overview.