Quantization calculator — compare GGUF, GPTQ, AWQ, FP8 formats

What each quantization format means

GGUF (GPT-Generated Unified Format)

Created by the llama.cpp project, GGUF is the most portable format. It supports 10+ quantization levels from Q2 through Q8, including "importance" variants (Q4_K_M, Q5_K_M) that preserve more quality on important weights. Recommended for CPU inference, Apple Silicon, and edge devices.

GPTQ (Post-Training Quantization)

GPTQ uses a one-shot calibration process to quantize weights to 4 or 8 bits. It requires a small calibration dataset and optimizes for GPU execution. GPTQ models load faster than GGUF on GPU because the format is designed for GPU memory layout. Best for NVIDIA GPU inference.

AWQ (Activation-Aware Weight Quantization)

AWQ is a newer GPU format that identifies "salient" weights (those with large activations) and protects them during quantization. This produces better quality than GPTQ at the same bit width (4-bit AWQ ≈ 4.5-bit GPTQ in practice). AWQ also supports in-flight quantization scale reordering for speed.

FP8 (8-bit Float)

FP8 is a hardware-native format on H100/H200 GPUs using the Transformer Engine. Unlike GGUF/GPTQ/AWQ (which are integer quantizations), FP8 uses floating-point representation with shared exponents. No calibration needed. Highest speed on compatible hardware.

Model size by quantization level

Precision	7B	13B	30B	70B	180B
FP16	14 GB	26 GB	60 GB	140 GB	360 GB
FP8	7 GB	13 GB	30 GB	70 GB	180 GB
Q8	7 GB	13 GB	30 GB	70 GB	180 GB
Q6	5.5 GB	10 GB	23 GB	53 GB	135 GB
Q4	4 GB	7.5 GB	17 GB	40 GB	100 GB
Q3	3 GB	5.5 GB	13 GB	30 GB	75 GB
Q2	2.5 GB	4.5 GB	10 GB	22 GB	55 GB

Quality vs speed: choosing the right format

The tradeoff is always the same: more bits = better quality but larger memory and slower inference. Here's a decision framework:

• FP16 — maximum quality, suitable for production when hardware permits. Used by Plugsky cloud.
• FP8 / Q8 — near-lossless compression. Use when hardware supports it (H100 for FP8, any GPU for Q8).
• Q4 (GPTQ/AWQ/GGUF) — the sweet spot for most users. 4× memory reduction with 95%+ quality retention.
• Q2 — for edge deployment where memory is critical. Quality loss is noticeable.

Which format for your hardware

Hardware	VRAM	Max model	Recommended format
Raspberry Pi / phone	1-4 GB	1-3B	GGUF Q2_K
MacBook Air / laptop	8 GB	7B	GGUF Q4_K_M (Metal)
RTX 3060 / 4060	12 GB	13B	GPTQ/AWQ 4-bit
RTX 4090	24 GB	30B	GPTQ/AWQ 4-bit or Q8
A100 80GB	80 GB	70B	FP16 or FP8
H100	80 GB	70B	FP8 (Transformer Engine)

When quality matters more than cost: Plugsky

Quantization always loses information. Even the best Q4 format drops some benchmark performance. When quality is your priority — for code generation, medical analysis, legal contracts, or research — full precision matters.

Plugsky runs all models at FP16 on datacenter A100 and H100 GPUs. No quantization, no calibration errors, no format conversions. You get the maximum quality the model can deliver, with no local VRAM limits and flat pricing.

Frequently asked questions

What is the difference between GGUF, GPTQ, and AWQ?

GGUF is a CPU-friendly format designed for llama.cpp, supporting many quantization levels (Q2-Q8). GPTQ is a GPU-optimized format that uses calibration data for weight quantization. AWQ is a newer GPU format that activates based on weight magnitude for better quality at the same bit width.

Which quantization format is fastest?

AWQ is typically fastest on GPUs due to its activation-aware design. GPTQ is close behind. GGUF Q4-Q5 offers good speed on both CPU and GPU. FP8 is fastest on H100/H200 GPUs with native hardware support.

How much quality loss comes with quantization?

Q8 is near-lossless (typically <0.5% perplexity increase). Q4 loses ~1-3% on benchmarks. Q2 loses ~5-10% quality and is noticeable in reasoning tasks. For production use, Q4 is the sweet spot for quality vs size.

Which quantization should I use for my hardware?

CPU only: GGUF Q4-Q5 (llama.cpp). Apple Silicon: GGUF Q4-Metal. 8GB GPU: Q4 GPTQ/AWQ. 16GB GPU: Q4 or Q8. 24GB+ GPU: Q8 or FP16. H100: FP8 with native transformer engine. 4GB or less: GGUF Q2 with CPU offloading.

Why does Plugsky use full precision?

Plugsky cloud models run at full FP16 precision on datacenter GPUs. This means no quality loss from quantization, no calibration errors, and no format compatibility issues. You get maximum quality from every model — use Plugsky when quality matters more than cost.

Last updated Jul 2026. Quality benchmarks are approximate and vary by model architecture and task.