Key facts
| Router model | model="plugsky-fusion" with strategies and ordered rules (live) |
| Throughput | Fair-use request limits scale by tier, up to 10K+ RPM on Enterprise |
| Rule inputs | Token count, tool use, prompt heuristics and workspace strategy |
| Scoping | Strategies per workspace, per key or per request — pool by queue |
| Overrides | Fixed models bypass routing for regulated flows |
| Residency | Pin GCC, EU, US or APAC; on-prem and air-gapped available |
| Audit | Model, strategy, rule and request metadata logged; exportable to SIEM |
| Roadmap | Classifier routing /v1/plugsky/route (model=auto) is coming soon |
TL;DR
- Split traffic into pools: self-service, care, assurance, retention.
- Escalate hard cases automatically instead of over-provisioning every pool.
- Add rules for language, queue and business-hours behaviour.
- Pin each pool to the jurisdiction its data requires.
- Load-test with real peak traffic before trusting any threshold.
How it works, step by step
- Define traffic pools and assign each a strategy and latency expectation.
- Separate keys per pool so behaviour and telemetry stay attributable.
- Add rules where markers are reliable — language, queue, tool use, prompt length.
- Load-test peak-hour volume and confirm fair-use request rates cover it.
- Pin workspaces to required regions and export routing logs to your SIEM.
- Review escalation and fallback rates monthly, then tune thresholds with quality data.
Try it yourself
Traffic pools and their strategies
Telco volumes are too large and too varied for one policy. Split them into pools: IVR and self-service answers, live care chat, assurance triage and retention offers. Give each pool a key with its own strategy — cost_saver for self-service, balanced for care chat, max_quality or a pinned strong model for retention conversations where revenue is at stake.
The router then does the fine-grained work within a pool: short questions stay cheap, tool-heavy or long requests escalate. Aggregate behaviour stays predictable because each pool has an owner and a budget.
Rules for language, queue and hours
Routing rules can encode operating reality. Business-hours traffic may justify a stronger default; overnight self-service can be cheaper. Language or originating-queue markers can send a market's traffic to models that perform well for it. Tool use is a strong signal — a request that touches billing or provisioning should not run on the smallest model.
- Order matters: rules evaluate first match wins, so specific conditions go first.
- Escape hatch: pin a model for flows that regulators or auditors examine.
- No extra hop: the router runs inside the API gateway, so routing itself does not add a network leg.
Latency, residency and oversight at scale
At peak, latency is a capacity problem and a routing problem at once. Use parallel tool calls to reduce round trips, measure with the latency tester, and capacity-plan against fair-use request rates up to 10K+ RPM on Enterprise. Fan-out strategies can add overhead, so keep interactive pools on lean strategies and reserve heavier chains for asynchronous work.
Residency is per pool: pin workspaces to the jurisdiction their data requires, or run on-prem and air-gapped where data cannot leave the estate. Every request logs the model, strategy, rule, tokens, latency, status, key ID, project and region, and those events stream to your SIEM so oversight keeps pace with volume.
Honest comparison
| Concern | Plugsky routing | Single model at scale | Vendor telco suite |
|---|---|---|---|
| Pool control | Strategy per pool, key or request | Global default only | Vendor-defined |
| Escalation | Automatic on reliable markers | Manual | Limited |
| Throughput | Fair-use tiers to 10K+ RPM Enterprise | Provider caps | Contract-dependent |
| Residency | Regional pinning, on-prem, air-gapped | Provider regions | Vendor regions |
| Evidence | Model, strategy and rule per request | Basic logs | Suite logs |
Frequently asked questions
Can routing handle peak telco volume?
Enterprise tiers support fair-use request limits up to 10K+ RPM. Load-test with real peak traffic, and use parallel tool calls to reduce model round trips.
How do we keep subscriber data in-country?
Pin each pool or request to a region — GCC, EU, US or APAC — or deploy on-prem and air-gapped where data cannot leave your estate.
Can we route by language?
Rules can key on prompt heuristics including language, plus queue or key context, so each market gets a model that performs well for it.
How do we control quality at volume?
Pools escalate by rule, and the decision log shows escalation rates per pool and model so thresholds can be tuned with evidence.
Can regulated flows avoid routing?
Yes. Pin an explicit model so the same flow takes the same path every time.
What is logged?
The chosen model, strategy and matching rule plus token, latency, status, key, project and region metadata, exportable to Splunk, Sentinel, QRadar or Chronicle.
How should we pilot?
Run one pool — self-service or care chat — for a month, compare escalation rate and satisfaction against the current setup, then expand pool by pool.