Key facts
| Setup | Opt in with model="plugsky-fusion" — no new SDK or endpoint |
| Fan-out | Parallel fan-out with best-result, vote and merge selection |
| Escalation | Default chain runs micro to pro to max and escalates on hard prompts |
| Strategies | cost_saver, balanced (default), max_quality and custom rules |
| Overrides | Any specific model name bypasses routing for deterministic tests |
| Logs | Chosen model, strategy and rule recorded per request |
| Compatibility | OpenAI SDKs, LangChain, LlamaIndex and the Vercel AI SDK work as-is |
| Roadmap | POST /v1/plugsky/route with model=auto is coming soon |
TL;DR
- Turn fusion on with one model string — keep your existing SDK code.
- Use it in staging to learn which tier your prompts actually need.
- Pin a fixed model where tests must be deterministic.
- Set strategies per environment: cost_saver in dev, balanced or max_quality in prod.
- Read the decision logs to turn routing into a measurable choice.
How it works, step by step
- Establish a baseline with one fixed model on your evaluation set, capturing quality and latency.
- Switch staging to model="plugsky-fusion" and rerun the same set without changing prompts.
- Compare per-case results against the baseline and inspect the recorded model choice for each request.
- Adopt a strategy: cost_saver for development and batch, balanced for general production, max_quality for demanding paths.
- Add custom rules only where you can name the marker — long inputs, tool use, language, or a keyword in the prompt.
- Keep fixed model names in latency-critical or deterministic tests, and monitor logs after each rule change.
Try it yourself
Open the multi-agent workflow generator →
Fusion as an evaluation harness
The hardest question in a multi-model stack is which tier each prompt really needs. Fusion answers it empirically. Run your evaluation set through model: "plugsky-fusion", then read the request logs to see which model was selected and which strategy produced it. Cases routed to cheap tiers are cases you can stop paying frontier prices for.
Because the endpoint stays OpenAI-compatible, the switch is one string. Your streaming handlers, retry logic and framework integrations — LangChain, LlamaIndex, the Vercel AI SDK — do not change. You can flip staging to fusion on Monday and compare against last week's baseline by Friday.
Choosing a strategy per environment
Strategies are the control surface. cost_saver picks the cheapest model that can handle the call; balanced (the default) uses mid-tier for most calls, cheap for trivial prompts and strong for hard ones; max_quality always takes the strongest model in your tier; custom evaluates your rules in order, first match wins.
- Dev and CI: cost_saver keeps test suites affordable.
- Production defaults: balanced for general traffic.
- High-stakes endpoints: max_quality or a fixed model.
Strategy can be set per workspace, per API key or per request, so a batch job and an interactive endpoint can share one account without sharing behaviour.
Reading logs and shipping rules
Every request records the chosen model, the strategy and the rule that fired. That turns routing from a black box into a tunable system: if a rule fires too often or too rarely, you can see it in the logs and adjust.
Version your rule changes and roll them out the way you ship code — one change at a time, with the evaluation set behind it. Keep a fixed model on any endpoint where determinism is a requirement, and remember that automatic classifier routing via /v1/plugsky/route is coming soon, which will extend the same decision log to model selection by classifier.
Honest comparison
| Concern | Plugsky Model Fusion | DIY ensemble | One fixed mid-tier model |
|---|---|---|---|
| Setup effort | One model string | Per-provider clients and merges | One provider |
| Model choice | Router picks across 30+ models | You wire each model | Whatever you chose |
| Escalation | Automatic on hard prompts | Manual retry logic | None |
| Observability | Model, strategy and rule logs | Custom instrumentation | Single model in logs |
| Cost shape | Flat plans, fair-use RPM | Per-token from each vendor | Per-token from one vendor |
Frequently asked questions
How do I enable fusion?
Set model="plugsky-fusion" in the existing chat completions call. No new endpoint, SDK or framework is required.
Will fusion slow my requests down?
Fan-out and escalation can add latency compared with a single fixed model. Use cost_saver or balanced, or pin a fixed model for latency-critical paths, and measure on your own workload.
Can I pin specific models?
Yes. Any explicit model name bypasses routing, which is useful for deterministic tests and regulated endpoints.
Does streaming still work?
Yes. Streaming is supported on chat completions, including requests routed through fusion.
How do I see what happened?
Each request's log entry records the chosen model, strategy and matching rule, alongside the usual token, latency and status metadata.
Do frameworks need changes?
No. The API stays OpenAI-compatible, so the OpenAI SDKs, LangChain, LlamaIndex and the Vercel AI SDK keep working.
What is coming next?
Classifier routing via POST /v1/plugsky/route with model=auto is coming soon; today's strategies and custom rules are live.