Key facts
| Router model | model="plugsky-fusion" applies a strategy per request (live) |
| Strategy scoping | Set per workspace, per API key or per request |
| Escalation | Default chain micro to pro to max, escalating on hard prompts |
| Observability | Model, strategy, rule, tokens, latency, key and region logged per request |
| Overrides | Fixed model names bypass routing for deterministic or latency-critical endpoints |
| Pricing | Flat self-serve plans; no per-token charges, fair-use RPM per tier |
| Residency | Pin per workspace or per request for customer contracts |
| Roadmap | Classifier routing /v1/plugsky/route (model=auto) is coming soon |
TL;DR
- Give each endpoint its own strategy: interactive, background, premium.
- Pin fixed models where a latency budget or determinism rules out fan-out.
- Use one flat plan instead of forecasting per-token cost per feature.
- Watch the model-choice logs after every strategy change.
- Degrade to cheaper tiers gracefully instead of failing under load.
How it works, step by step
- Classify product surfaces by budget: interactive chat, background enrichment, premium analysis, internal tooling.
- Assign a strategy to each — cost_saver or balanced for interactive, max_quality for background where quality leads.
- Opt in with model="plugsky-fusion" and load-test each surface for latency impact.
- Pin fixed models for any endpoint with a hard latency or determinism requirement.
- Instrument the routing logs into your dashboards, including tokens and latency per model.
- Roll rule changes gradually, watching error rates and the distribution of chosen models.
Try it yourself
A latency budget per product surface
Users notice latency before they notice model choice. Fusion lets you treat quality as a per-surface setting: interactive chat defaults to cost_saver or balanced, where speed matters most, while background jobs such as nightly enrichment or report generation can afford max_quality and longer chains.
Fan-out and escalation add some overhead versus a single fixed model, so measure per endpoint rather than assuming. Where a surface has a hard budget — autocomplete, inline suggestions — pin a fast model explicitly and remove routing from the critical path.
Cost predictability at scale
Per-token billing makes every new AI feature a finance conversation. Plugsky self-serve plans are flat monthly with unlimited fair-use usage and no per-token charges, so a feature that suddenly goes viral changes your request rate, not your cost per user in an unpredictable way.
Token counts still return in every response, so you can attribute usage internally and negotiate enterprise terms with real data. The only rate dimension to plan around is fair-use requests per minute for your tier, which is a capacity question — the kind of problem capacity planning already solves.
Observability and safe rollouts
Treat routing rules like product code. Each request logs the chosen model, the strategy and the rule that fired, alongside latency, tokens and status, so you can measure the impact of a rule change the same way you measure a deploy. If p95 latency rises after a change, the log shows whether escalation is the cause.
- Gradual rollout: apply a new strategy to one key or service first, then widen it.
- Fallback: the router escalates or falls back by policy, and fixed overrides remain for endpoints that must not vary.
- Roadmap: classifier routing with model=auto is coming soon, which will add another lever to the same log.
Honest comparison
| Concern | Plugsky Model Fusion | Per-token AI gateway | One fixed model |
|---|---|---|---|
| Per-surface control | Strategy per workspace, key or request | Rules with metered cost | No control |
| Latency | Tunable via cheap-first or fixed override | Varies | Predictable |
| Cost shape | Flat monthly, fair-use RPM | Per token, variable | Per token, variable |
| Observability | Model, strategy and rule per request | Gateway logs | Single-model logs |
| Determinism | Explicit model names bypass routing | Varies | Always deterministic |
Frequently asked questions
Does fusion hurt interactive latency?
It can add some overhead through fan-out and escalation. Use cost_saver or balanced for interactive surfaces, pin a fixed model where a hard budget exists, and verify with the latency tester.
Can different features use different strategies?
Yes. Strategy can be set per workspace, per API key or per request, so each product surface behaves independently on the same account.
How do we keep costs predictable?
Self-serve plans are flat with unlimited fair-use usage and no per-token charges. Token telemetry still returns per response for internal attribution.
What happens during traffic spikes?
The plan's fair-use request rate applies; routing keeps model cost flat while your capacity planning handles the request rate.
Can we still pin a specific model?
Yes. Any explicit model name bypasses routing, which is how we recommend handling latency-critical or deterministic endpoints.
How do we debug a quality change?
Read the decision log: it shows the chosen model, strategy and matching rule per request, so regressions can be traced to a specific routing change.
Is per-tenant residency supported?
Yes, pin a workspace to a region, or set the region per request for customers with specific contractual requirements.