VRAM calculator — how much GPU memory for LLMs?

How VRAM calculation works

The formula for estimating VRAM is straightforward:

VRAM = (params × bytes_per_param) + (ctx_len × num_layers × hidden_size × 2 × 2) + overhead

For most models, a good rule of thumb is:

• FP16: 2 bytes per parameter → 7B model = ~14 GB for weights
• Q8: 1 byte per parameter → 7B = ~7 GB
• Q4: 0.5 bytes per parameter → 7B = ~3.5 GB
• Q2: 0.25 bytes per parameter → 7B = ~1.75 GB

KV cache and context overhead

The KV cache stores attention keys and values for every token in the context. It grows linearly with context length and can exceed the model weights at very long contexts:

• 4K context: ~1 GB KV cache (7B model)
• 32K context: ~8 GB KV cache
• 128K context: ~32 GB KV cache

For multi-user serving, multiply the KV cache by the number of concurrent users. Production deployments with 10+ concurrent users often spend more VRAM on KV cache than on model weights.

VRAM by quantization format

Format	Bits	7B VRAM	70B VRAM	Quality
FP32	32	28 GB	280 GB	Reference
FP16	16	14 GB	140 GB	Near-lossless
GGUF Q8	8	7 GB	70 GB	Excellent
GPTQ/AWQ 4-bit	4	4.5 GB	42 GB	Good
GGUF Q2	2	2.5 GB	22 GB	Noticeable loss

Not enough VRAM? Use Plugsky instead

If your GPU doesn't have enough memory for the model you want, you have three options:

1. Quantize more aggressively — trade quality for memory (e.g., Q4 → Q2)
2. CPU offloading — split layers between GPU and system RAM (slower)
3. Use Plugsky cloud inference — run any model on cloud GPUs with no hardware requirements

Plugsky gives you access to 30+ models including 70B-class and MoE architectures. No VRAM limits, no GPU purchase, no power bills. Flat pricing from $99/mo.

Frequently asked questions

How much VRAM do I need to run a 7B model?

A 7B model in FP16 requires about 14GB VRAM for the weights alone. With Q4 quantization, this drops to ~4.5GB. Adding KV cache for 4K context brings it to ~5GB total. A 8GB GPU can run a 7B model at Q4 with moderate context.

What is the formula for VRAM calculation?

VRAM = (model_parameters × bytes_per_param) + (context_length × num_layers × hidden_size × 2 × 2) + overhead. bytes_per_param: FP16=2, FP32=4, Q8=1, Q4=0.5, Q2=0.25. Overhead is typically 10-20%.

Can I run a 70B model on a single GPU?

Not at FP16 (requires 140GB). With Q4 quantization, a 70B model needs ~40GB, which fits on an A100 80GB or dual RTX 4090s. For Q2, ~20GB fits on a single 24GB GPU but quality loss is significant.

How does context length affect VRAM?

KV cache scales linearly with context length. A 7B model with 4K context uses ~1GB of KV cache. At 32K context it uses ~8GB. At 128K context it uses ~32GB — often more than the model weights themselves.

What if I don't have enough VRAM?

You can: (1) use more aggressive quantization, (2) offload layers to CPU (slower), (3) use Plugsky cloud inference instead. Plugsky runs models on cloud GPUs with no local VRAM requirement — flat pricing from $99/mo.

Last updated Jul 2026. VRAM estimates are approximate and vary by implementation and model architecture.