How do you build AI routing with AI agents?

Key facts

Router endpoint	POST /v1/chat/completions with a small JSON schema for complexity, intent and language
Target models	30+ models behind one endpoint; aliases include plugsky-micro, plugsky-lite, plugsky-pro and plugsky-frontier
Escalation	Cascade pattern: cheap attempt first, one escalation on low confidence or validation failure
Function calling	Live on supported models; use tools to call specialist agents and deterministic services
Pricing model	Flat monthly self-serve plans with unlimited fair-use usage
Free tier	Free plan with 2 free AI models, no card required
Observability	Per-route logging plus usage analytics and scoped keys
Roadmap	Assistants and responses endpoints are coming soon

TL;DR

• Classify first, then dispatch: routing decisions belong in a cheap, explicit step.
• Keep a small set of routes — three to five tiers beats a sprawling model map.
• Escalate once on low confidence; loops hide quality problems.
• Run the router in shadow mode before it controls production traffic.
• One OpenAI-compatible endpoint means a route change is a model alias change.

How it works, step by step

1. Collect real requests and label the cheapest model tier that would satisfy each one.
2. Define three to five tiers (for example: micro, routine, reasoning, frontier) and the model alias for each.
3. Write a router prompt or rules that output a tier, a confidence score and a short reason.
4. Dispatch to the chosen alias and validate the response against a per-tier acceptance check.
5. Escalate once to the next tier when confidence is low or validation fails, then stop.
6. Log route, confidence, escalation and outcome; review weekly for drift and misroutes.

Original data

Try it yourself

Open the AI workload router simulator →

Routing patterns that survive production

Four patterns cover most needs, ordered by effort:

• Static rules: map known call sites, tenants or features to aliases. Zero latency, easy to audit, breaks on unseen traffic.
• Semantic router: embed the query with plugsky-embed and match it against labeled examples per route. Fast and cheap, needs a curated example set.
• LLM classifier: ask plugsky-micro for a tier, confidence and reason using a strict JSON schema. Flexible, adds one cheap call.
• Cascade: attempt the cheap tier, validate, and escalate once when confidence or schema checks fail.

Most teams should combine static rules for known traffic with an LLM classifier for the long tail, then wrap both in a cascade.

Building the router step

The router is a normal function-calling or JSON-mode call. Keep the schema tiny: a tier enum, a confidence number and a reason string. Store the reason for audits.

• Include routing hints in the schema: language, whether the request needs fresh data, and whether it touches regulated data.
• Reserve plugsky-micro for routing itself — a frontier model deciding which model to use defeats the purpose.
• Make dispatch data-driven: tier to alias to timeout to output cap in one config object.
• Use function calling when the target is a specialist agent with its own tools, not just a different model.
• Bound the cascade: one escalation, one retry, then a clear failure response.

Evaluating a router

Treat the router as a model with its own metrics:

• Routing accuracy: share of requests sent to the tier your labeled set says is appropriate.
• Escalation rate: how often the cascade had to upgrade, by route.
• Quality by tier: pass rate per tier on a frozen evaluation set, so a cheap tier is never silently degraded.
• Overhead: added latency and cost of the routing call itself.
• Drift: weekly change in route distribution and classifier confidence.

Start in shadow mode: log what the router would choose while a fixed model serves traffic. Only switch control after accuracy is stable for several days.

Limitations and failure modes

Routers fail in predictable ways:

• User text can manipulate the router, so never let request content override tier caps for sensitive actions.
• Feature parity differs across models: tool support, context length and JSON reliability vary, so validate per tier rather than assuming equivalence.
• Routing adds latency on the critical path; for interactive chat, keep the classifier small or run it in parallel with retrieval.
• Distribution shift degrades classifiers quietly. Re-label examples monthly and watch confidence histograms.
• Assistants and responses endpoints are coming soon; do not design routing around managed agent state today.

Honest comparison

Capability	Plugsky router agent	Static if/else routing	One frontier model for all
Decision quality	Classifier plus confidence and escalation	Deterministic but blind to content	No decision needed
Model choice	30+ aliases behind one endpoint	Whatever you hard-code	Single vendor model
Escalation	Built-in cascade on validation failure	Manual	Not applicable
Cost shape	Cheap first, strong only when needed	Depends on the rules	Highest per request
Evaluation	Route accuracy and per-tier quality	Route coverage only	Single quality metric

Frequently asked questions

What is AI routing with agents?

It is an agent step that classifies a request and dispatches it to the cheapest model or specialist agent that can satisfy it, with a defined escalation path.

How many routes should I start with?

Three to five tiers. More routes multiply evaluation work and rarely improve outcomes before you have labeled data.

Which model should run the router itself?

A cheap one such as plugsky-micro. Routing is a small classification task, and using a frontier model for it removes the savings.

How do I stop bad routes in production?

Validate every tier's output against an acceptance check, cap escalations at one, and alert on spikes in escalation rate or validation failure.

Do I need embeddings for routing?

Only for semantic routing. Rules and a JSON-mode classifier are enough to start; add embeddings when you have a labeled example set.

Does routing work across languages?

Yes if the classifier and evaluation set include each language you serve. Include language in the router schema so you can audit per-language quality.

Can I try this on the free plan?

Yes. plugsky-micro and plugsky-lite are available on the free plan with no card, and a 14-day full-access trial opens the paid models for evaluation.