Key facts
| Primary lever | Route routine work to small models; escalate only hard cases |
| Signals | Task type, input length, user tier and validation result |
| Policy location | Configuration with version history, not scattered in code |
| Waste to remove | Duplicate questions, oversized prompts, unbounded retries |
| Catalogue | 30+ models behind one OpenAI-compatible endpoint |
| Pricing model | Flat monthly self-serve plans with fair-use usage |
| Free plan | plugsky-micro and plugsky-lite, no card required |
| Product status | Live |
TL;DR
- Find the waste before you route: duplicates, prompt bloat and retries.
- Send routine traffic to the free or smallest tier that passes.
- Escalate on a validator failure, not on a hunch.
- Keep routing policy in configuration and version it.
- Track cost per successful task so savings are real, not theoretical.
How it works, step by step
- Baseline current usage: request mix, average input size, retry rate and duplicate prompts.
- Remove the easy waste first — cache repeat questions and trim system prompts.
- Define a validator per workload so escalation has an objective trigger.
- Route each workload to the cheapest tier that passes its evaluation set.
- Cap escalation attempts and log the escalation rate.
- Review the policy monthly and after every model-card change.
Try it yourself
Open the AI workload router simulator →
Find the waste before you route
Routing amplifies whatever you already send. If a third of your traffic is duplicate questions or prompts padded with unused context, fix that first — it is cheaper than any routing rule. Instrument request mix, input length and retry counts, then look at cost per successful task rather than cost per request.
Three quick wins usually appear in the data: repeated questions that should be cached, system prompts that grew without review, and retries that fire on deterministic errors a validation step would have caught.
Routing patterns that cut cost
- Static rules: route by task type and input length. Simple, explainable, easy to audit.
- Validation escalation: answer on the cheap tier, escalate only when a schema, test or citation check fails.
- Classifier: when rules cannot separate easy from hard, train or prompt a small model to classify and route.
- Tenant tiers: give premium plans a stronger default and free plans the free models.
Keep the escalation evidence compact — send the failure, not the full transcript — and prefer small models such as plugsky-lite for the first hop. The model routing guide covers implementation.
Measure and maintain
Track four numbers per workload: requests, escalation rate, cost per successful task and quality score. If escalation climbs, either the validator got stricter or the cheap tier regressed — both are actionable. If quality drops while escalation stays flat, the validator is the problem.
Self-serve plans are flat monthly with fair-use usage rather than per-token billing, so map volume to plan limits on the live pricing page instead of estimating token spend. Re-run the evaluation set whenever a model card changes, and treat routing thresholds as production configuration that needs review, not set-and-forget values.
Honest comparison
| Workload | Route to | Why | Guardrail |
|---|---|---|---|
| FAQ and macro answers | Free tiers (micro, lite) | High volume, low risk | Validator plus sampled human review |
| Classification and tagging | Free tiers | Deterministic output shape | JSON schema check |
| Long-document analysis | Long-context tier | Needs the full window | Input-length rule |
| Code edits | Fast coder tier | Cost per accepted change | Repository test suite |
| High-stakes decisions | Top tier | Quality before cost | Human review |
Frequently asked questions
What is model routing?
It is choosing which model handles a request at runtime — for example sending routine questions to a small model and escalating hard ones to a stronger tier, based on rules or a classifier.
How much does routing cut costs?
It depends on your escalation rate. If most requests pass on a cheap tier, blended cost stays close to that tier; measure cost per successful task on your own workload rather than quoting a generic percentage.
What should trigger an escalation?
An objective validator failure: a JSON schema check, a failed test, a missing citation or a rubric score below threshold. Hunches and retries are not routing policies.
Where should routing rules live?
In versioned configuration, so tuning is a reviewable change rather than a code release, and so the policy can be tested independently of application logic.
Can I route on the free plan?
Yes — plugsky-micro and plugsky-lite are free with no card required and make an ideal first hop. Use the 14-day full-access trial to test escalation tiers.
How do I avoid quality regressions?
Keep a per-workload evaluation set and run it on every model or threshold change. Track quality alongside escalation rate so a stricter validator cannot hide a real regression.
How is pricing structured?
Self-serve plans are flat monthly with fair-use usage rather than per-token billing. See the live pricing page for current plans and limits.