Key facts
| Small LLM sweet spot | Classification, extraction, tagging, routing and simple chat |
| Frontier sweet spot | Multi-step reasoning, synthesis and high-stakes judgement |
| Decision inputs | Failure cost, request volume, latency budget and input complexity |
| Typical pattern | Small default plus a frontier escalation path |
| Catalogue | 30+ models behind one OpenAI-compatible endpoint |
| Free starting point | plugsky-micro and plugsky-lite, no card required |
| Trial | 14-day full-access trial for the top tiers |
| Product status | Live |
TL;DR
- Narrow, high-volume, latency-sensitive work belongs on a small model.
- Multi-step reasoning and high-stakes judgement belong on a frontier model.
- The real question is failure cost, not model size.
- Combine both: small default, validator, frontier escalation.
- Measure pass rate — most teams overestimate how often they need the big model.
How it works, step by step
- List candidate tasks and answer three questions: how hard, how often, what does failure cost.
- Run the small model on a labelled evaluation set and record its pass rate.
- Add a deterministic validator — schema, test, citation or rubric — that catches failures.
- Route only validator failures to the frontier tier, with a capped retry.
- Measure blended cost per successful task and the escalation rate.
- Widen the small model's scope as its pass rate proves out; narrow it where failures cluster.
Try it yourself
The task-fit test
Three questions separate small-model work from frontier work. First: how structured is the output? JSON extraction and closed-label classification are easy to validate, which makes them safe for small models. Second: how many steps of reasoning are required? A single extraction step is very different from reconciling five sources. Third: what happens when the answer is wrong? Low-stakes mistakes can tolerate a small model; customer-facing or financial ones usually cannot.
If the task is narrow, validatable and cheap to be wrong about, start small. If it needs synthesis, planning or judgement, start at a stronger tier.
Where small models win
Small models win on volume and responsiveness. Classification at scale, tagging, routing, simple question answering and generating tool arguments are all workloads where a small model plus validation beats a frontier model on cost and latency. The free tiers make this testable at zero cost: plugsky-lite for speed and plugsky-micro when a little reasoning helps.
The catch is that small models fail differently — they agree confidently with bad premises. Build validators for that: schema checks, allowed-value lists, citation verification and simple consistency rules catch most of it before a user does.
Combining both without waste
The practical pattern is a hybrid: a small default with a validator, and a frontier escalation path for failures. It keeps the median request cheap while protecting the tail. Two rules make it work: the validator must be objective, and the escalation evidence must be compact — send the failing case and the requirements, not the whole transcript.
Then measure. If the escalation rate is low, the small model is doing its job; if it is high, the task or prompt is wrong, not the model. See model routing for implementing the policy and the frontier guide for choosing the escalation tier.
Honest comparison
| Task | Small LLM | Frontier model | Why |
|---|---|---|---|
| Intent classification | Best fit | Overkill | Narrow label space |
| Multi-step analysis | Weak | Best fit | Reasoning depth |
| Single-document summary | Good | Best for synthesis across sources | Context and judgement |
| Code autocomplete | Best fit | Good but slower and costlier | Latency per keystroke |
| High-stakes decisions | Only with strong review | Best fit | Failure cost |
| Cost at volume | Lowest | Highest | Volume multiplies the difference |
Frequently asked questions
When is a small LLM good enough?
When the task is narrow and validatable — classification, extraction, tagging, routing, simple chat — and a mistake is cheap to catch or recover from. Test with a labelled set rather than assuming.
When do I need a frontier model?
For multi-step reasoning, synthesis across documents and high-stakes judgement where a wrong answer is expensive. It is also the right escalation target when a small model fails your validator.
How do I measure which one I need?
Run the small model on a labelled evaluation set, measure its pass rate, then route failures to the frontier tier and track the escalation rate. That number decides the split.
Can I use a small model for free?
Yes — the free plan includes plugsky-micro and plugsky-lite with no card required, which is enough to measure pass rates before paying for anything.
Does a small model hallucinate more?
It fails differently: smaller models tend to agree confidently with flawed premises. Deterministic validators and clear refusal instructions reduce the risk substantially.
Should I ever run both on every request?
Ensembling both on every request is expensive. Reserve a second model for verification of high-stakes outputs, or for escalation after validation failure.
Is there a trial for the frontier tier?
Yes — a 14-day full-access trial lets you evaluate the top tiers against your escalation workload. See the live pricing page for details.
How is pricing structured?
Self-serve plans are flat monthly with fair-use usage rather than per-token billing. Check the live pricing page for current plans.