How does Plugsky model routing pick the right model automatically?

Key facts

Routing endpoint	POST /v1/plugsky/route with model="auto" — documented as coming soon
Route hints	cost, quality or latency, plus a max-cost-per-million guard
Live alternative	model="plugsky-fusion" runs a configured chain today
Savings signal	Docs describe 60-80% savings on production traffic for fusion chains
Observability	The response model field names the model that answered
Fallback	Unhealthy upstreams are routed around automatically
Escape hatch	Pin an explicit model name when determinism matters
Product status	Coming soon (classifier endpoint); Fusion routing is live

TL;DR

• Routing trades a little determinism for cost and latency control.
• The classifier endpoint is coming soon; Fusion chains route today.
• Cost hints only work if you define quality floors first.
• Log the answering model or routing becomes invisible.
• Keep explicit model pinning for regulated or deterministic workloads.

How it works, step by step

1. Define a quality floor for each workload using your own eval set.
2. Baseline cost and latency per model on a representative traffic sample.
3. Start with a Fusion cost-saver chain in staging and log the answering model.
4. Compare quality and percentile latency against the single-model baseline.
5. Segment by task type and keep explicit model names where routing hurts.
6. Adopt the classifier route endpoint when it becomes available, using route hints and cost guards.
7. Re-evaluate the routing table whenever the model catalogue changes.

Try it yourself

Open the LLM cost calculator →

What routing can and cannot decide

A router can only optimise against something measurable. Good signals: prompt length, presence of code or structured data, request class from your own taxonomy, user tier, and explicit cost or latency budgets. Weak signals: intuition about difficulty, or a single global threshold that treats every workload the same. The honest framing is that routing is a classifier problem with a quality floor — if you cannot evaluate answers, you cannot safely route away from your strongest model.

The routing options in Plugsky

Two mechanisms appear in the docs, and their status differs:

• Model Fusion (live): set model="plugsky-fusion" and the platform runs your configured sequential, parallel or cost-saver chain. This is the production-safe option today.
• Classifier routing (coming soon): POST /v1/plugsky/route defaults to model="auto" and accepts route_hint of cost, quality or latency with a maximum-cost guard. Model auto is not accepted by chat completions directly.
• Explicit pinning (always available): name the model in the request when output must be reproducible or regulated.

Measuring the trade-off honestly

Run routing as an A/B against a pinned baseline on the same prompts. Track three curves: quality score from your eval set, p50 and p95 latency, and the distribution of models actually selected. A router that saves cost but shifts traffic away from your quality floor is a regression, and a router that escalates nearly everything is pure overhead. Segment results by task class — extraction, classification, summarisation, reasoning — because aggregate numbers hide the workload where routing fails.

Production guardrails

• Cap the maximum cost per request so a chatty prompt cannot select a premium model by accident.
• Log the answering model, tokens, latency and route decision for every request.
• Keep a deterministic fallback path for compliance-sensitive flows.
• Alert when routing distribution drifts after a model or prompt change.
• Version your routing rules like code — review them, and roll back on regressions.

Routing is an optimisation on top of a working baseline, not a first step. Ship with one model, build the eval harness, then let routing earn its place. See /docs for the current status of the routing endpoint.

Honest comparison

Capability	Plugsky routing	Manual model selection	In-house classifier
Cost optimisation	Fusion cost-saver live; classifier coming soon	Human decision per workload	You train and maintain it
Route hints	cost, quality, latency with cost guard (docs)	None	Custom
Observability	Answering model in every response	Your logs	Custom
Failure handling	Automatic upstream failover	Manual	You implement it
Determinism	Intentional trade-off	Total	Total
Ops burden	Low	Manual reviews	High

Frequently asked questions

Is classifier-based routing available today?

No. The POST /v1/plugsky/route endpoint with model="auto" and route_hint values is documented as coming soon. Model Fusion chains route per request today.

What route hints are planned?

cost, quality and latency, plus a maximum-cost guard per request. Check /docs for the final field names and status before integrating.

Can I use model="auto" on chat completions?

No. The docs state auto is handled by the routing endpoint, not by /v1/chat/completions. Use plugsky-fusion or an explicit model name on chat completions.

How much does routing save?

The docs describe 60-80% savings on production traffic for fusion chains. Treat that as a site-stated figure and measure against your own eval set and traffic mix.

What is the biggest routing risk?

Quality regressions on edge cases the cheap path mishandles. Define a quality floor and test every routing change against it before production.

How do I keep routing observable?

Log the response model field and per-request logs (model, latency, tokens, status). If you cannot see which model answered, you cannot debug the router.

When should I pin a model instead?

When output must be reproducible, when regulators need a fixed model version, or when your eval shows the cheapest path fails your quality floor.

Does routing work with tools and JSON mode?

Routing changes which model answers, so verify that the selected models support the capabilities your request uses, such as function calling or structured outputs.