Key facts
| Input vs output | Output tokens are typically priced higher than input tokens |
| Tokenizer variance | The same text can produce different token counts across model families |
| Context resend | Multi-turn and RAG workloads resend context on every request |
| Caching | Prompt caching reduces cost when a stable prefix repeats |
| Retries | Failed and retried calls still consume budget on per-token plans |
| Flat alternative | Plugsky self-serve plans are flat monthly with unlimited fair-use usage and no per-token billing |
| Model tiers | Cheaper small models handle classification and routing before frontier calls |
| Product status | Chat, streaming, JSON mode, function calling, embeddings and RAG are live |
TL;DR
- Cost per million tokens is not one number — input and output are weighted differently.
- Resent context and retries are the two costs teams forget to measure.
- Tokenizers differ, so compare cost per task, not cost per character.
- Cheap models for routing and classification cut spend before it starts.
- Flat-rate pricing replaces the per-million maths with a fixed monthly plan.
How it works, step by step
- Instrument your application to log input and output token counts per request.
- Separate workload classes: chat, RAG, classification, extraction and agents.
- Measure average tokens per request and requests per user per month for each class.
- Multiply by the live rates for the models you actually call, then add retry overhead.
- Run the same workload through a smaller model and compare quality with an eval set.
- Compute the monthly total and compare it with the flat plan that covers your usage.
- Re-measure after every prompt or model change — token profiles drift quickly.
Try it yourself
Open the LLM token calculator →
The four variables that set your per-million price
Input weighting. Most providers price input and output differently, and output is usually the expensive side. Tokenizer. The same paragraph can be more tokens on one model family than another, so a lower headline rate can still cost more per task. Context resend. A RAG call that sends a long retrieved passage pays for that passage on every turn. Retries. Timeouts and 429 retries that reach the model are billed work on per-token plans.
How to calculate your own cost per million tokens
Do not estimate from character counts. Log real usage, then apply this method:
- For each workload, record average input tokens and average output tokens per request.
- Multiply by monthly request volume to get monthly input and output tokens.
- Read the current input and output rates from the live pricing page and apply them separately.
- Add retry and re-run overhead as a percentage you measured, not a guess.
- Divide total monthly cost by total tokens to get your blended cost per million.
The token calculator runs the same arithmetic interactively. If the total is dominated by stable, predictable traffic, compare it against a flat-rate plan where usage is unlimited under fair use.
Where teams overspend without noticing
- Full history on every turn. Chat apps resend the whole conversation; trimming or summarising old turns cuts input tokens immediately.
- Oversized retrieved context. Sending ten chunks when three answer the question is a silent multiplier.
- Frontier models for easy calls. Routing classification, formatting and extraction to small models changes the blended rate.
- Uncapped output. A missing max-token setting lets a model ramble at output rates.
- Re-embedding unchanged documents. Incremental indexing avoids paying twice for the same text.
When flat-rate beats per-million maths
Per-token pricing is efficient for low, spiky volume: you pay for exactly what you use. It becomes awkward when volume is steady and forecasting matters, because a prompt change, a new feature or a retry storm moves the bill. Flat monthly plans with unlimited fair-use usage convert that variable line into a fixed one, and there is no per-token charge or overage fee on self-serve plans.
Keep the measurement discipline either way: tokens per task is the metric that tells you whether a quality improvement is worth its cost.
Honest comparison
| Cost factor | Plugsky flat-rate | Per-token pricing | Self-hosted inference |
|---|---|---|---|
| Billing unit | Flat monthly plan | Input and output tokens | GPU hours plus ops |
| Output-token premium | None on self-serve plans | Usually priced higher than input | Depends on your hardware |
| Context resend | Covered by the plan under fair use | Billed on every call | Consumes GPU time |
| Retries | No per-token charge on self-serve | Billed when they reach the model | Consumes GPU time |
| Forecastability | Fixed monthly line item | Varies with traffic and prompts | Capex plus utilisation risk |
| Optimisation lever | Route models for quality and latency | Compress prompts and context | Batch and quantise |
Frequently asked questions
Why are output tokens more expensive than input tokens?
Generating text is sequential and compute-heavy, while prompts can be processed in parallel. Most providers therefore weight output tokens higher, which is why uncapped completions inflate bills.
Does streaming change the cost per million tokens?
No. Streaming changes how tokens are delivered, not how many are generated or how they are billed. It improves perceived latency while the token count stays the same.
How do I count tokens accurately?
Use the tokenizer for the model you call rather than estimating from words or characters. Log token counts returned in API responses so your measurements match what is actually billed.
Do retries cost money?
On per-token plans, yes — a retried call that reaches the model is billed again. On Plugsky self-serve plans there is no per-token billing, so retries consume rate-limit budget rather than adding charges.
What is a blended cost per million tokens?
It is your total monthly spend divided by total tokens across all models and workloads. It is the number to compare against flat-rate plans because it reflects your real model mix.
Is flat-rate always cheaper?
No. Flat-rate wins when usage is steady and predictable. Per-token can be cheaper for low, spiky or experimental workloads where a monthly plan would sit idle. Compare with your own measured volumes.
How much does Plugsky cost?
Prices change, so check the live pricing page for current plans; the free plan includes two models with no card.
Do embeddings count toward inference cost?
Yes. Embedding calls consume tokens and should be measured separately, especially when re-indexing. Incremental indexing and cached embeddings keep that line small.