Key facts
| Routing | Send classification, extraction and formatting to small models |
| Context trimming | Fewer resent tokens cut input cost and latency together |
| Caching | Stable prompt prefixes and reused embeddings avoid repeat work |
| Output caps | Max-token limits prevent expensive rambling completions |
| Batching | Offline jobs can run in batches outside peak hours |
| Evaluation | Eval sets prove a cheaper configuration keeps quality |
| Flat-rate option | Self-serve Plugsky plans are flat monthly with unlimited fair-use usage and no per-token billing |
| Product status | Chat, streaming, JSON mode, function calling, embeddings and RAG are live |
TL;DR
- Cheaper calls come from smaller models, less context and capped output.
- Validate with evals — cost cuts without measurement are quality regressions waiting to happen.
- Caching and embedding reuse remove work instead of degrading it.
- Route by task difficulty, not by habit.
- Flat-rate plans remove the per-token incentive to under-serve users.
How it works, step by step
- Build a representative evaluation set with expected answers before changing anything.
- Baseline cost per task and quality score for the current configuration.
- Route easy requests to small models and re-run the eval.
- Trim conversation history and retrieved chunks, then measure both cost and quality.
- Enable prefix caching and reuse embeddings for unchanged documents.
- Set max-token limits and structured output where free text is unnecessary.
- Keep only changes that hold quality within your threshold, and monitor weekly.
Try it yourself
Change the workload, not the standard
Most AI bills are set by three choices: which model runs, how much context travels with each request, and how long completions run. Optimising those three does not require degrading answers — it requires matching resources to the difficulty of each request.
The discipline that keeps quality intact is measurement. Build an evaluation set of real inputs with expected outputs, then treat every cost change as an experiment that must pass the same bar as a product change.
The highest-impact levers
- Model routing. A small model handles classification, routing, tagging and formatting; reserve frontier calls for reasoning-heavy work.
- Context trimming. Summarise old turns, retrieve fewer but better chunks, and drop boilerplate from prompts.
- Caching. Stable system prompts and unchanged documents should not be recomputed or re-embedded.
- Output control. Set max tokens, ask for JSON when structure is enough, and stop generation when the answer is complete.
- Batching. Move backfills, re-indexing and eval runs off the interactive path.
Measure with evals, not vibes
An eval set of a few hundred representative cases catches regressions that spot checks miss. Score each configuration on task success, format correctness and any domain-specific rubric, then compare cost per task at the same quality threshold.
Track three numbers over time: cost per task, quality score and p95 latency. A change that lowers cost while holding quality and latency is a win; a change that lowers cost and quality is a product decision, not an optimisation.
When flat-rate changes the calculus
On per-token pricing, every extra call, retry and agent step is a line item, which pushes teams to under-provision context and avoid useful techniques like self-checks or multi-pass generation. A flat plan with unlimited fair-use usage removes that marginal cost: extra iterations and richer context do not add charges on self-serve plans.
That flips the optimisation target from token shaving to quality and latency. You still trim waste — it improves speed — but you stop trading answer quality for a smaller bill. Use the live pricing page to compare flat plans against your measured token spend.
Honest comparison
| Optimisation | Impact on cost | Impact on quality | How to validate |
|---|---|---|---|
| Model routing | High on metered plans | Neutral if routing is correct | Eval set per route |
| Context trimming | High for chat and RAG | Can regress if overdone | A/B on answer accuracy |
| Prefix caching | Medium, repeat workloads | None — identical prompt | Cache hit rate |
| Output caps | Medium for verbose tasks | Risk of truncation | Truncation rate metric |
| Embedding reuse | Medium for re-indexing | None if documents unchanged | Index freshness check |
| Flat-rate plan | Removes marginal token cost | Enables richer context and retries | Quality score at fixed cost |
Frequently asked questions
What is the fastest way to reduce AI API cost?
Route the easiest requests to a smaller model and trim resent context. Those two changes usually move the bill most, and both can be validated quickly with an evaluation set.
Will cheaper models hurt answer quality?
Only if they run tasks they cannot handle. Use routing so hard reasoning stays on stronger models while classification, extraction and formatting move to small ones.
How do I prove quality did not drop?
Keep an evaluation set of representative inputs with expected outputs, score every configuration against it, and require the same pass rate before shipping a cost change.
Does prompt caching reduce quality?
No. Caching reuses identical prefix computation, so answers are unchanged. It lowers cost and latency for stable system prompts and repeated context.
Are there per-token charges on Plugsky?
No. Self-serve plans are flat monthly with unlimited fair-use usage, and there are no per-token charges or overage fees. See the live pricing page for current plans.
Should I cap completion length?
Yes for structured or short-answer tasks, with monitoring for truncation. Uncapped generation is one of the most common sources of unnecessary output-token spend.
How does flat-rate pricing change optimisation?
It removes the marginal cost of extra calls, so you can keep richer context and use multi-pass techniques without watching a meter. Optimisation then targets latency and quality rather than token count.
How often should I re-run cost reviews?
Monthly, plus after every model or prompt change. Token profiles drift, and a route that was correctly sized last quarter may be mispriced now.