Key facts
| Router model | model="plugsky-fusion" applies strategy per request (live) |
| Scoping | Per workspace, per API key or per request |
| Defaults | cost_saver for interactive, balanced for general, max_quality for background |
| Overrides | Explicit model names bypass routing for deterministic endpoints |
| Observability | Model, strategy, rule, tokens, latency, key and region logged per request |
| Pricing | Flat self-serve plans; no per-token charges, fair-use RPM per tier |
| Residency | Pin per workspace or per request for customer contracts |
| Roadmap | Classifier routing /v1/plugsky/route (model=auto) is coming soon |
TL;DR
- Give every product surface a strategy and a latency budget.
- Canary routing changes on one key before a fleet-wide rollout.
- Watch escalation rates; they are your quality-versus-latency signal.
- Keep deterministic endpoints on fixed models.
- Use flat pricing to keep unit economics stable as traffic grows.
How it works, step by step
- List product surfaces and assign each a quality target and latency budget.
- Apply strategies per endpoint via separate API keys for each service.
- Pin fixed models for surfaces where determinism or hard latency limits apply.
- Instrument routing logs into your dashboards, tracking model distribution and p95 latency.
- Roll out rule changes through a canary key, then widen after a monitoring window.
- Review monthly: escalation rate, quality signals and residency coverage per tenant.
Try it yourself
Open the model availability tracker →
Strategy per endpoint, not per company
A SaaS product is a set of very different AI surfaces. Inline suggestions need speed above all; chat assistants need balance; nightly enrichment wants quality and has hours to spare. Routing by endpoint turns those differences into configuration instead of a company-wide compromise.
Give each service its own API key with a strategy attached. Interactive keys run cost_saver or balanced; batch keys run max_quality; anything with a hard latency contract pins a fast model and opts out of routing entirely. The keys also give you clean per-service telemetry.
Canary changes like any deploy
Routing rules change behaviour, so they deserve the same rollout discipline as code. Apply a rule to one canary key first, watch the decision log for the distribution it produces, and compare quality and latency before widening. Because an explicit model name bypasses routing, rollback for an affected endpoint is a string change with immediate effect.
- Availability: track model availability as a dependency and know your degradation story per surface.
- Rate limits: the fair-use request rate for your plan tier is the capacity dimension; model cost stays flat.
- Buckets: keep an evaluation set per surface so canary comparisons are statistically meaningful.
Monitoring what the router chose
You cannot manage what you cannot see, and routing is invisible by default. Each request logs the chosen model, the strategy and the rule that fired, alongside tokens, latency, status, key ID, project and region. Pipe those events into your dashboards and alert on distribution drift — a sudden rise in frontier usage usually means a rule is matching more than intended.
That observability also answers customer questions: which region answered, whether a request touched a specific model, and how a tenant's traffic is shaped. For customers with residency requirements, pin workspace or per-request regions and feed the same logs into your compliance reporting.
Honest comparison
| Concern | Plugsky routing | Gateway with metered billing | Fixed model per service |
|---|---|---|---|
| Per-endpoint policy | Strategy per key or request | Rules with token cost | One model forever |
| Latency control | Balanced or pinned fast models | Varies | Predictable but fixed |
| Change rollout | Canary key plus instant override | Custom process | Deploy to change |
| Observability | Model, strategy and rule per request | Gateway logs | Single-model metrics |
| Cost shape | Flat plans, fair-use RPM | Per token | Per token |
Frequently asked questions
How do we give different features different models?
Attach a strategy to a per-service API key, or set the strategy per request. Each endpoint then behaves independently on the same account.
Will routing affect our latency SLOs?
Balance and cost_saver strategies are designed for interactive traffic, but fan-out can add overhead. Measure per surface and pin fast models where a hard budget exists.
How do we roll out a rule change safely?
Canary it on one key, watch the decision log and latency metrics for a monitoring window, then widen it. Rollback is an explicit model name on affected endpoints.
Can we see which model a tenant used?
Every request log includes the chosen model, strategy and rule, plus key, project and region, so per-tenant reconstruction is possible.
What happens if a model becomes unavailable?
Fallback and escalation follow the strategy. Track availability as a dependency and define behaviour per surface — degrade, queue or retry.
Does this change our pricing?
Self-serve plans are flat with unlimited fair-use usage and no per-token charges; token counts return for internal attribution. See the live pricing page for tiers.
Is classifier routing available?
Not yet — POST /v1/plugsky/route with model=auto is coming soon. Strategies and ordered rules are live today.