Key facts
| Goal | Match each workload to the cheapest model that passes its quality checks |
| Portfolio tiers | Free, workhorse and top tiers — one default plus one escalation per workload |
| Catalogue | 30+ models behind one OpenAI-compatible endpoint |
| Evaluation | One shared eval set per workload, re-run when model cards change |
| Routing | Keep model choice in configuration, not scattered through code |
| Governance | Ownership, cost visibility, residency and fallback policy per workload |
| Free plan | plugsky-micro and plugsky-lite, no card required |
| Product status | Live |
TL;DR
- Start with a workload inventory, not a model wish list.
- Give every workload a default model and one escalation target.
- Keep routing in configuration so policy changes do not need releases.
- Run shared evals before and after every model change.
- Add governance early: owner, budget, residency and fallback per workload.
How it works, step by step
- Inventory workloads: task type, volume, latency budget, quality bar and compliance constraints.
- For each workload, pick the cheapest default model that passes an evaluation set.
- Add exactly one escalation model for failures, and cap escalation attempts.
- Move model names and thresholds into configuration with a version history.
- Automate the evaluation set in CI so regressions are caught before release.
- Assign an owner and a cost view to every workload, and review quarterly.
Try it yourself
Open the AI workload router simulator →
Start with a workload inventory
A multi-model strategy fails when it starts from model names. Start from work: chat, classification, extraction, summarisation, code edits, retrieval and visual triage. For each workload record volume, latency budget, quality bar and compliance constraints. That table, not a leaderboard, tells you which tiers you need.
Two workloads that look similar can need different models — a support macro and a contract summary both generate text, but only one can tolerate an occasional miss. Write the acceptance test before you write the routing rule.
Design the portfolio
Keep it deliberately small: a default model and one escalation model per workload is usually enough. Defaults should sit on the cheapest tier that passes — often plugsky-lite or plugsky-micro for high-volume tasks — while escalation covers the tail. A shared top tier can serve several workloads as their escalation target.
- Do not add a model without a workload that needs it.
- Version the policy and record why each default was chosen.
- Prefer signals you already have: input length, task type, tenant tier, validator result.
Operate and govern the portfolio
A strategy only survives if it has owners and feedback. Give every workload a named owner, a cost dashboard and an evaluation set that runs automatically. When a model card changes, re-run the evaluation before the change reaches production; when an incident occurs, the fallback path should be defined in advance.
Governance also covers where data goes. Residency, retention and audit requirements belong in the workload table, alongside the model choice. See the AI governance framework and model routing for the operational detail.
Honest comparison
| Dimension | Single-model policy | Tiered multi-model | Bespoke multi-vendor |
|---|---|---|---|
| Quality ceiling | One model for every task | Strong where it matters | Highest, hardest to manage |
| Cost predictability | Simple to forecast | Tunable per workload | Varies by vendor contract |
| Operational effort | Lowest | Moderate | Highest |
| Vendor risk | Concentrated | Reduced by one API | Spread but complex |
| Governance | One path to audit | Policy per workload | Multiple paths to reconcile |
| Time to ship | Fastest | Fast with planning | Slowest |
Frequently asked questions
What is a multi-model AI strategy?
It is a plan that assigns each workload a default model and an escalation model, with routing rules, evaluation sets and governance defined per workload instead of defaulting everything to one model.
How many models should a strategy include?
Most teams need two or three tiers: a cheap default, a stronger escalation target and occasionally a specialist such as a long-context or vision tier. Plugsky exposes 30+ models, but a portfolio only needs the ones your workloads require.
Where should routing logic live?
In configuration with a version history, not spread through application code. That way a policy change is a config review, not a release.
How do I know a cheaper model is good enough?
Define an evaluation set with acceptance criteria for the workload, then run every candidate against it. The cheapest model that consistently passes becomes the default.
How do we stop costs creeping up?
Track cost per successful task per workload, cap escalation attempts, cache repeated questions and remove oversized prompts. Review the portfolio quarterly and after model changes.
Can we build this on the free plan?
Yes — plugsky-micro and plugsky-lite are free with no card required, which is enough to establish defaults and evaluation sets. Use the 14-day full-access trial to test escalation tiers.
How is pricing structured?
Self-serve plans are flat monthly with fair-use usage rather than per-token billing. See the live pricing page for current plans and limits.