Key facts
| Router model | plugsky-fusion escalates per request across tiers (live) |
| Strategies | cost_saver, balanced, max_quality and custom rules per key |
| Models | 30+ models; cheap tiers suit FAQs, small talk and intent detection |
| Streaming | Live via server-sent events on chat completions |
| Pricing | Flat monthly self-serve plans with no per-token charges on self-serve |
| Free tier | plugsky-micro and plugsky-lite on the free plan, no card required |
| Usage controls | Scoped keys and usage analytics per workspace or key |
| Roadmap | Classifier routing (/v1/plugsky/route, model=auto) is coming soon |
TL;DR
- Most chatbot turns are easy: answer them on the cheapest capable tier.
- Trim history by token budget and summarize instead of resending everything.
- Cap reply length per intent; open-ended generation is where spend explodes.
- Escalate on complexity signals rather than on every message.
- Measure cost per resolved conversation, not cost per token.
How it works, step by step
- Tag every request with conversation id, intent and feature to find the expensive paths.
- Classify each incoming turn cheaply and select a tier from that classification.
- Set cost_saver or balanced as the default policy per chatbot key.
- Trim history to a fixed token budget with system prompt and recent turns first, summaries after.
- Cap max_tokens per intent and stop generation early when a complete answer exists.
- Escalate to stronger tiers when intent is complex, tools fail or the user repeats a question.
- Review cost per resolved conversation monthly and re-tune thresholds.
Try it yourself
Open the LLM token calculator →
The three cost drivers in chat
Conversation history is the first driver: every turn resends context, so a chat that starts cheap gets expensive by message twenty. Model choice is the second: routing greeting-and-FAQ traffic to a frontier model is pure waste. Retries and regeneration are the third: a confusing prompt or a flaky tool loop burns tokens without producing answers.
All three are addressable without hurting the experience. Trim history by token budget, keep a rolling summary of older turns, and let the router choose the tier per turn. The user sees the same conversation; the model behind it changes only when the task does.
Routing chat turns by intent
A cheap classification call up front pays for itself immediately. Detect whether the turn is small talk, an FAQ, a data lookup, a multi-step task or a complaint, then select the tier and the tools that intent needs.
- Cheap tier: greetings, FAQs, status checks, intent confirmation.
- Mid tier: summarising a conversation, drafting a structured reply.
- Strong tier: troubleshooting, reasoning over multiple records, sensitive handling.
- Pinned paths: compliance or high-value conversations that must never downgrade.
Fusion strategies cover the default; custom rules pin exceptions. Classifier routing with model=auto is coming soon, and until then per-key policies and intent checks do the work.
Budgets, caching and measurement
Set a token budget per turn and per conversation, and enforce it in code rather than trusting prompts. Cap completion length by intent, reuse stable system prompts, and avoid resending retrieved documents that have not changed. Where the API supports prefix caching, structure prompts so the stable part stays first.
Measure cost per resolved conversation alongside resolution rate, escalation rate and retries. A cheap tier that forces users to repeat themselves is not cheap; a strong tier answering a FAQ is not quality. Flat self-serve plans make the bill predictable while you tune — see the live pricing page — and the free plan with plugsky-micro and plugsky-lite is enough to build the instrumentation before scaling.
Honest comparison
| Cost lever | Routed chatbot on Plugsky | Single strong model | Single cheap model |
|---|---|---|---|
| FAQs and small talk | Cheap tiers by default | Frontier price per turn | Cheap but error-prone |
| Complex turns | Automatic escalation | Native strength | Frequently wrong |
| History growth | Token budget with summaries | Grows every turn | Grows every turn |
| Reply length | Capped per intent | Often verbose | Often truncated |
| Visibility | Per-request tier logs | Uniform, hard to optimise | Uniform, low headroom |
Frequently asked questions
How much can routing save on chatbots?
The saving depends on your intent mix, but the principle is stable: FAQ and small-talk traffic that runs on cheap tiers removes the largest avoidable expense. Measure cost per resolved conversation before and after.
Will cheap models make the bot sound worse?
Not for routine turns, because greetings and FAQs need fluency, not deep reasoning. Escalate complex or sensitive conversations to stronger tiers, and evaluate tone per intent before widening.
How do I keep long conversations affordable?
Trim to a token budget, summarize older turns, and send retrieved context only when it changed. Never resend the entire transcript by default.
Should I stream replies?
Streaming improves perceived latency but does not change token cost. It is still the right default for chat because time to first token dominates how responsive the bot feels.
How do I stop runaway loops?
Cap tool calls and regeneration attempts per turn, add timeouts, and fail into a human handoff rather than retrying indefinitely.
What are fusion strategies?
cost_saver, balanced and max_quality presets, with custom rules for pinned paths. Set them per chatbot key so production and testing do not share a policy.
How does flat pricing affect chatbot costs?
Self-serve plans are flat monthly with no per-token charges, so retries and longer prompts do not change the bill the way they do on metered APIs. See the live pricing page for plans.
Can I start for free?
Yes. plugsky-micro and plugsky-lite are on the free plan with no card, and the 14-day full-access trial lets you test stronger tiers on real conversations.