How do developers route model requests with Plugsky?

Key facts

Setup	Set model="plugsky-fusion" — same endpoint, no new SDK
Strategies	cost_saver, balanced (default), max_quality and custom rules
Rule order	Custom rules evaluate in order; first match wins
Scope	Per workspace, per API key or per request
Example rules	Keyword to a coding model; long prompts to long-context; free-plan traffic to free models
Overrides	Explicit model names bypass routing
Logs	Chosen model, strategy and rule recorded per request
Roadmap	POST /v1/plugsky/route with model=auto is coming soon

TL;DR

• Routing is configuration, not infrastructure — one model string opts you in.
• Write custom rules with clear markers and order them most-specific first.
• Set strategies per key so dev, staging and prod behave differently.
• Use fixed model names for tests that must be reproducible.
• Read the decision logs instead of guessing why a model changed.

How it works, step by step

1. Switch staging to model="plugsky-fusion" and keep your prompts and evaluation set unchanged.
2. Start with balanced; measure latency and quality per endpoint before adding rules.
3. Add one custom rule at a time, placing the most specific conditions first.
4. Give each service or environment its own key with a strategy that matches its budget.
5. Pin explicit models for deterministic tests and latency-critical calls.
6. Review request logs after each change and export them to your observability stack.

Try it yourself

Open the AI model selector →

Rules you can express

Custom routing rules map a condition to a model, and the reference implementation gives useful patterns: prompts containing a keyword such as refactor go to a coding model, inputs above roughly 4,000 tokens go to a long-context or frontier model, traffic from free-plan users goes to free models, and business-hours traffic can take a stronger default.

Order matters because rules evaluate in order and the first match wins. Put the most specific conditions at the top, keep each rule explainable in one sentence, and prefer markers the router can detect reliably — token count, tool presence, workspace or key — over clever heuristics that drift.

Strategy per service and environment

The strategy can be set per workspace, per API key or per request, which is the feature that makes routing practical. Development and CI keys run cost_saver so test suites stay affordable. A batch enrichment service can run max_quality overnight where latency does not matter. Interactive endpoints run balanced or pin a fast model outright.

• One account, many behaviours: rotate keys per service rather than sharing one strategy everywhere.
• Escape hatch: any explicit model name bypasses routing, so rollback is a string change.
• No extra hop: the router runs inside the API gateway, not as a separate proxy.

Debugging and improving routing

Treat routing as observable code. Each request log records the chosen model, the strategy and the rule that fired, so when quality shifts you can tell whether a rule changed behaviour or a model did. Compare distributions week over week: a sudden jump in frontier usage usually means a rule is matching more than intended.

When you need new behaviour, add a rule in staging, replay a sample of real traffic, and inspect the decision log. Classifier-based selection through POST /v1/plugsky/route with model=auto is coming soon; until then, explicit strategies and custom rules cover most production needs.

Honest comparison

Concern	Plugsky routing	DIY router service	Static model config
Where it runs	Inside the API gateway, no extra hop	Separate service you operate	Client side
Rule engine	Ordered rules, first match wins	Custom code	None
Scope	Per workspace, key or request	Whatever you build	Global constant
Observability	Model, strategy and rule per request	You instrument it	Model name only
Maintenance	Configuration	Code, deploys and tests	Manual changes

Frequently asked questions

Where do I configure rules?

Strategies and custom rules are configured in the dashboard per workspace, API key or request; requests opt in with model="plugsky-fusion".

In what order do rules run?

In the order you define them, and the first match wins, so specific conditions should come before general ones.

Can I bypass routing?

Yes. Sending a specific model name bypasses the router entirely, which is useful for deterministic tests and latency-critical calls.

Does routing add a network hop?

No. The router runs inside the API gateway, so latency impact comes from fan-out and escalation behaviour, not an extra proxy.

How do I know which model answered?

Each request's log records the chosen model, the strategy and the matching rule, plus standard token, latency and status metadata.

Do frameworks still work?

Yes. The endpoint stays OpenAI-compatible, so the OpenAI SDKs, LangChain, LlamaIndex and the Vercel AI SDK work unchanged.

What is the classifier endpoint?

POST /v1/plugsky/route with model=auto is a coming-soon endpoint that will select models by classifier; today's strategies and rules are live.