Key facts
| RPM | Requests per minute allowed for a key or project |
| TPM | Tokens per minute, weighted by prompt plus completion size |
| Concurrency | Number of requests that may be in flight simultaneously |
| Throttle signal | HTTP 429 with Retry-After guidance when a limit is exceeded |
| Backoff | Official SDKs retry 429s with exponential backoff and jitter |
| Pricing model | Self-serve plans are flat monthly with unlimited fair-use usage; no per-token billing or overage fees |
| Free plan | 2 free AI models (plugsky-micro and plugsky-lite), no card required |
| Product status | Rate limits, retries and usage analytics are live |
TL;DR
- RPM, TPM and concurrency are separate limits — you hit whichever comes first.
- Capacity planning starts with measured peak traffic, not average traffic.
- Keep client concurrency below the cap so retries still have headroom.
- Treat 429s as normal backpressure: backoff, jitter, capped retries, then shed load.
- Flat-rate plans still have fair-use limits; per-token billing is what they remove.
How it works, step by step
- Measure peak requests per minute and peak concurrent requests from your logs.
- Convert token volume to TPM using average prompt plus completion tokens per call.
- Set a client-side concurrency limit below the plan cap to leave retry headroom.
- Handle 429 responses with exponential backoff, jitter and a hard retry ceiling.
- Load-test at 1.5 to 2 times expected peak and watch p95 latency and 429 rate.
- Queue or shed non-critical work first: batch jobs, evals and backfills can wait.
- Review usage analytics weekly and raise limits or move plans before launches.
Original data
Try it yourself
Open the rate limit calculator →
RPM, TPM and concurrency are three different limits
Requests per minute caps how often you can call. Tokens per minute caps how much text those calls may carry, weighting prompts and completions together. Concurrency caps how many calls can be open at once. A workload with short prompts may pass RPM while failing TPM; a workload with long streaming responses may pass both while sitting at its concurrency ceiling.
Read the response headers and your usage dashboard to see which dimension you are actually approaching. Optimising the wrong one wastes engineering time.
Why flat-rate plans still have fair-use limits
Flat monthly pricing removes per-token billing, not physics. GPUs serve a finite number of concurrent sequences, so every provider enforces limits somewhere. Plugsky's self-serve plans use unlimited fair-use usage: normal production traffic runs without token meters or overage fees, while sustained patterns that would starve other tenants get rate-limited rather than billed.
- No per-token billing on self-serve plans.
- No overage fees — the failure mode is backpressure, not a surprise invoice.
- Enterprise plans add committed capacity and negotiated limits.
A capacity planning method you can run yourself
Use your own numbers instead of vendor estimates:
- Peak RPM = highest requests in any minute over the last 30 days, times a safety factor.
- Peak TPM = peak RPM x (average prompt tokens + average completion tokens).
- Peak concurrency = peak RPM x average request duration in seconds / 60.
- Headroom = plan limit / peak, target at least 1.5 before a launch.
Run the arithmetic in the rate limit calculator, then validate with a load test that replays production-like prompts. If headroom is thin, raise concurrency limits or move to an enterprise plan rather than hoping traffic stays average.
Handling 429s without losing work
A 429 means the request never ran, so retrying is safe for idempotent inference calls. Use exponential backoff with jitter so many workers do not retry in lockstep, cap retries at three to five attempts, and route exhausted requests to a queue for later. Reserve concurrency for user-facing paths and let batch jobs absorb throttling.
Alert on 429 rate as a first-class signal. A rising 429 rate with flat traffic usually means a retry storm or a runaway loop, not genuine growth.
Honest comparison
| Capability | Plugsky | Typical per-token API | Self-hosted stack |
|---|---|---|---|
| Cost model | Flat monthly, unlimited fair use on self-serve | Per-token with overage | GPU capex plus ops |
| Throttle behaviour | Fair-use RPM, TPM and concurrency limits | Hard per-token quotas and spend caps | Your own queueing |
| Burst headroom | Plan-dependent, enterprise adds committed capacity | Pay for higher tier limits | Bounded by your GPUs |
| Retry handling | SDKs back off with jitter on 429 | Manual retry logic | Manual retry logic |
| Observability | Usage analytics and limit headers | Varies by provider | You build it |
| Planning effort | Measure peaks and size a plan | Forecast token spend | Forecast hardware and utilisation |
Frequently asked questions
What is the difference between RPM and concurrency?
RPM caps how many requests you may send per minute; concurrency caps how many may be in flight at the same moment. A long-running stream can exhaust concurrency while staying well under RPM.
Does Plugsky charge per token?
No. Self-serve plans are flat monthly with unlimited fair-use usage, and there are no per-token charges or overage fees on those plans. See the live pricing page for current plan details.
What should I do when I get a 429?
Retry with exponential backoff and jitter, cap the number of attempts, and queue or shed non-critical work. Plugsky SDKs already implement backoff, so most applications only need a retry ceiling.
How do I size a plan for a new product?
Estimate peak RPM and TPM from a load test or pilot cohort, apply a 1.5x safety factor, and compare against plan limits. The rate limit calculator walks through the arithmetic.
Are unlimited plans really unlimited?
They are unlimited for normal production traffic under fair use. Rate limits protect shared capacity, and sustained patterns that would degrade other tenants are throttled rather than billed.
Can I get higher limits?
Yes. Enterprise plans support committed capacity and negotiated limits, including VPC, on-prem and air-gapped deployments. Talk to the team when your peak headroom drops below 1.5x.
Do retries cost extra?
On self-serve plans there is no per-token billing, so retried requests do not add line-item charges. They do consume rate-limit budget, which is why capped retries and backoff matter.
Where can I see current usage and limits?
Usage analytics in the dashboard show requests, tokens and errors per key, and responses include rate-limit headers. The status page reports live component health.