How do you forecast AI infrastructure costs for a SaaS product?

Key facts

Driver tree	Users to sessions to requests to tokens to model mix
Scenarios	Base, growth and spike cases with explicit assumptions
Sensitivity	Output tokens, context length and retries move totals most
Embedding refresh	Re-indexing documents is a recurring, often forgotten cost
Infra beyond inference	Storage, vector database, monitoring and egress sit alongside tokens
Flat-rate layer	Self-serve Plugsky plans are flat monthly with unlimited fair-use usage and no per-token billing
Free plan	2 free models (plugsky-micro and plugsky-lite), no card required
Product status	Chat, streaming, function calling, embeddings and RAG are live

TL;DR

• Forecast from a driver tree, not a single usage number.
• Run base, growth and spike scenarios with written assumptions.
• Output tokens, context length and retries dominate sensitivity.
• Embedding refreshes and monitoring are recurring costs, not one-offs.
• A flat-rate plan turns the token layer into a fixed line item.

How it works, step by step

1. Map the driver tree: users, sessions per user, requests per session, tokens per request.
2. Measure each ratio from production telemetry rather than assumptions.
3. Apply current published rates for the model mix each feature uses.
4. Add storage, vector database, monitoring, evaluation runs and egress.
5. Build base, growth and spike scenarios and document the assumptions in each.
6. Run a sensitivity check on the three variables that move the total most.
7. Review monthly against actuals and update ratios when prompts or features change.

Try it yourself

Open the LLM cost calculator →

Build the driver tree first

Every credible forecast reduces to a chain of ratios you can measure: monthly active users, sessions per user, requests per session, tokens per request, and the model mix those requests hit. Multiply the chain and apply rates. If any ratio is a guess, instrument it — a wrong ratio at the top of the tree distorts every scenario below.

The tree also tells you where to intervene. If requests per session is the dominant driver, product design matters more than prompt tuning.

Scenario planning that finance can use

• Base. Current usage ratios held flat with modest user growth.
• Growth. User growth plus a feature that increases sessions per user.
• Spike. A launch or seasonal peak with retries and support load included.

For each scenario, state the assumptions in one line and show the resulting monthly cost range. A forecast with a range and named assumptions survives contact with reality far better than a single point estimate.

The costs teams forget to forecast

Inference tokens are rarely the whole picture. Add:

1. Embedding refreshes — re-indexing changed documents on a schedule.
2. Evaluation runs — standing quality checks across model versions.
3. Vector storage and database — grows with documents and tenants.
4. Monitoring and tracing — observability for latency, errors and cost.
5. Egress and integration — moving data between services and regions.

These behave like infrastructure: predictable, recurring and independent of token volume.

Making the forecast stable with flat-rate pricing

The hardest line to forecast is the token layer, because prompts, retries and feature launches move it without warning. Flat monthly plans with unlimited fair-use usage replace that line with a fixed cost, so the forecast becomes user growth times plan economics plus infrastructure.

Self-serve plans have no per-token charges or overage fees, which removes the true-up risk that makes finance nervous. Compare the flat plan against your base and spike token estimates on the live pricing page, and revisit when sustained usage approaches fair-use limits.

Honest comparison

Forecast component	Plugsky flat-rate	Per-token API	Self-hosted inference
Inference cost	Fixed monthly plan	Tokens x rates x growth	GPU capex plus utilisation
Scenario sensitivity	Low — cost independent of prompts	High — output and context drive totals	Medium — hardware sizing
Overage risk	None on self-serve plans	True-ups above budget	Capacity shortfalls
Embeddings	Covered under fair use	Metered separately	Shares GPU capacity
Monitoring and evals	Usage analytics included	Provider-dependent	You build it
Forecast effort	Plan plus user growth	Full driver tree required	Hardware demand model

Frequently asked questions

What is the best unit for an AI cost forecast?

Cost per active user per month. It combines the driver tree into one number that finance can compare against revenue and that engineering can decompose when it changes.

Which variable should I stress-test first?

Output tokens and context length, because both scale with usage patterns rather than user count. Retry overhead is the third, especially for agent workloads.

How do I forecast retries?

Instrument retry rates in production and apply the observed percentage to the token estimate. Retries are measurable, not a rounding error, and agent loops amplify them.

Are embeddings part of infrastructure cost?

Yes. Embedding generation and re-indexing recur whenever documents change, and vector storage grows with tenants. Treat them as infrastructure lines, not one-off setup costs.

How does flat-rate pricing help forecasting?

It replaces the variable token layer with a fixed monthly plan, so the forecast becomes user growth plus infrastructure. Self-serve plans have no per-token charges or overage fees.

How often should the forecast be updated?

Monthly against actuals, and immediately after a model migration, pricing change or major feature launch. Ratios drift as prompts and user behaviour evolve.

Should the forecast include evaluation runs?

Yes. Evals are a standing quality cost, especially if you test multiple model versions. Include them in infrastructure rather than hiding them in engineering time.

What if actuals exceed the forecast?

Compare ratios against assumptions to find the drifted one, then adjust routing, context or plan. A forecast that is reviewed monthly is a control system, not a document.