Self-hosting break-even calculator — API vs on-premise costs

The real cost of self-hosting

Self-hosting an LLM involves more than buying a GPU. Here is the full cost breakdown:

Cost category	Example	Monthly (amortized)
GPU hardware	RTX 4090 $3K / 3yr	$83
Power & cooling	450W GPU + system	$100
Internet & colo	Home or colocation	$50
Labor (ML ops)	0.1 FTE at $240K/yr	$2,000
Software & licenses	CUDA, container registry, monitoring	$100
Total monthly (RTX 4090)		$2,333

Break-even formula

The break-even point is where:

Tokens/month = (Hardware_amortization + Power + Labor + Software) ÷ API_cost_per_token

For a typical setup (RTX 4090, $2,333/mo total, $5/MTok API):

Break-even = $2,333 ÷ $0.000005 = ~467 million tokens/month

Below this volume, paying for API access is cheaper than self-hosting. Above it, self-hosting makes financial sense (but you still need to manage it).

Self-host vs cloud GPU vs flat-rate API

Option	50M tokens/mo	500M tokens/mo	5B tokens/mo	Ops effort
API per-token	$250	$2,500	$25,000	None
Self-host (buy GPU)	$2,333	$2,333	$4,666 (2x GPUs)	High
Cloud GPU rent	$1,080	$1,080	$2,160 (2x GPUs)	Medium
Plugsky (flat)	$99	$299	Contact us	None

When self-hosting makes sense

Self-hosting is the right call when:

• High volume, predictable usage — 500M+ tokens/month with steady demand
• You already have GPUs — repurposing existing hardware changes the math
• Compliance mandates on-prem — data can never leave your network
• You have ML engineering headcount — idle talent that can manage the stack

Plugsky: the middle ground

Most teams fall between API and self-hosting. Plugsky is designed for that gap:

• Same OpenAI-compatible API as API providers — no code changes
• Flat pricing from $99/mo — more predictable than per-token, cheaper than GPU rental
• No ops — we manage the GPUs, scaling, patching, and monitoring
• 30+ models — including 70B-class and MoE architectures

Frequently asked questions

When does self-hosting LLMs become cheaper than API?

Typically between 100-500M tokens/month depending on the model. A 7B model with Q4 quantization on an RTX 4090 ($3K hardware) breaks even with GPT-4-class API at ~150M tokens/month. Larger models need more volume to justify the GPU investment.

What are the hidden costs of self-hosting?

Beyond GPU hardware ($3K-$30K), self-hosting includes: power ($50-200/mo per GPU), cooling, internet bandwidth, labor (ML engineer time for setup and maintenance), software licensing, model storage, and opportunity cost of time spent on infrastructure instead of product.

Can I lease a GPU instead of buying?

Yes. Cloud GPU rental (Vast.ai, RunPod, Lambda, AWS) costs $0.50-2.00/hour per GPU. At $1/hr, a single GPU costs ~$720/mo. This shifts CAPEX to OPEX and is better for variable workloads, but often more expensive than flat-rate API at low volumes.

Is Plugsky cheaper than self-hosting?

For most teams under 1B tokens/month, yes. Plugsky's flat $99/mo tier is cheaper than a single cloud GPU rental ($720+/mo). At higher volumes, Plugsky enterprise pricing is designed to undercut self-hosting TCO (hardware + labor + power).

What token volume justifies buying a GPU?

A rough rule: 24GB GPU (RTX 4090, $3K) → break-even at ~100-200M tokens/month vs API. 80GB GPU (A100, $15K) → break-even at ~500M-1B tokens/month. Below these volumes, API or flat-rate services are more cost-effective.

Last updated Jul 2026. Costs are estimates and vary by region, provider, and usage pattern.