Key facts
| Router model | model="plugsky-fusion" routes each request (live) |
| Strategies | cost_saver, balanced (default), max_quality and custom rules |
| Routing inputs | Token count, tool use, prompt heuristics and workspace strategy |
| Scope | Per workspace, per API key or per request |
| Overrides | A specific model name bypasses routing for regulated flows |
| Decision log | Chosen model, strategy and matching rule recorded per request |
| Residency | Pin inference and logs to one region; VPC, on-prem and air-gapped available |
| Roadmap | Classifier routing /v1/plugsky/route (model=auto) is coming soon |
TL;DR
- Map each workload to a strategy instead of arguing model choice per team.
- Keep deterministic overrides on regulated and audit-critical endpoints.
- Log every routing decision for internal audit and regulator questions.
- Route high-volume triage to cheap tiers and reserve frontier models narrowly.
- Pin inference to a region; keep the router inside the API gateway.
How it works, step by step
- Inventory workloads and classify them: triage, extraction, analyst assistance, regulatory drafting.
- Assign strategies per workload — cost_saver for volume, balanced by default, max_quality for high-stakes output.
- Add custom rules only for reliable markers, such as long documents or policy keywords.
- Pin fixed models on endpoints where reproducibility or regulation requires it.
- Export routing and key events to your SIEM and review escalation distributions monthly.
- Pilot with one business line, compare against the current setup, then widen.
Try it yourself
Open the AI workload router simulator →
Mapping banking workloads to strategies
Routing works best when it mirrors how the bank already triages work. Alert summaries, transaction categorisation and document extraction are high-volume and low-risk: cost_saver or a cheap-first chain is appropriate. Analyst assistance and customer-response drafting sit in the middle at balanced. Regulatory reporting, policy interpretation and anything a committee will read belong on max_quality.
The router inspects input tokens, expected output size, tool use and prompt heuristics before choosing, so a short prompt that looks routine does not accidentally reach a frontier model, and a long, tool-heavy request escalates even when the caller did not ask.
Determinism where regulation demands it
Some banking processes must be repeatable. When an endpoint falls under model risk management, pin the model: sending an explicit model name bypasses routing entirely, so the same input reaches the same model every time.
- Scope: strategies can be attached per workspace, per API key or per request, letting regulated desks run deterministic while the rest of the bank optimises.
- Residency: pin inference to me-central-1, sa-central-1, eu-west-1, eu-central-1, us-east-1, us-west-2 or ap-southeast-1.
- Isolation: project-scoped keys keep environments and business lines separate.
Evidence, cost and capacity
Routing decisions are evidence. Each request records the chosen model, the strategy and the rule that fired, alongside tokens, latency, status, key ID, project and region. That trail is exportable to Splunk, Sentinel, QRadar or Chronicle, so audit and operations work from the same data.
On cost, self-serve plans are flat with unlimited fair-use usage and no per-token charges, so the economics of routing are about quality and capacity rather than invoice arithmetic. Start with one business line, review the routing distribution after a month, and adjust thresholds with real data before standardising bank-wide.
Honest comparison
| Concern | Plugsky routing | Manual model selection | One model for all |
|---|---|---|---|
| Decision point | Per request, inside the gateway | Engineer per call | Fixed at build time |
| Policy controls | Strategies and ordered custom rules | Code reviews | None |
| Determinism | Explicit model names bypass routing | Manual pinning | Always deterministic |
| Evidence | Model, strategy and rule logged | Partial | Single model logs |
| Cost shape | Flat plans, fair-use RPM | Varies by tier chosen | Premium on every call |
Frequently asked questions
How does routing choose a model?
The router inspects input token count, expected output size, whether tools are present, prompt heuristics and the strategy set for the workspace, then selects a model from your tier.
Can we force a specific model?
Yes. Send an explicit model name and routing is bypassed, which is how we recommend handling regulated or reproducibility-critical endpoints.
How do we prove which model ran?
Every request logs the chosen model, strategy and matching rule, plus token, latency, status, key, project and region metadata, exportable to your SIEM.
Does routing add network hops?
No. The router runs inside the API gateway, so requests do not take an extra hop through a separate service.
How is data kept in-country?
Pin the workspace to a region, or set the region per request. Data never leaves the pinned region, and VPC, on-prem and air-gapped deployments are available.
What happens if a model is unavailable?
Strategies fall back or escalate by policy, and deterministic endpoints can pin a model with a defined failure path. Track availability as you would any dependency.
How does a bank start?
Pick one business line, run it on balanced with a couple of rules for a month, review the decision log, then standardise the strategies that proved themselves.