Local LLM on MacBook — Run AI on Apple Silicon (M1-M4)

Why MacBooks are great for local AI

Apple Silicon Macs have three unique advantages for running local LLMs:

1. Unified memory = VRAM. On a PC, you have separate system RAM (typically 32-64 GB) and GPU VRAM (8-24 GB). On a Mac, the same pool of memory is shared between CPU and GPU. A Mac with 64 GB unified memory can load a 50 GB model. A PC with 64 GB RAM + 24 GB VRAM can only load a ~22 GB model (limited by the 24 GB VRAM).
2. Metal GPU acceleration. Apple's Metal API provides fast GPU inference through llama.cpp and Ollama. Performance is competitive with mid-range NVIDIA GPUs.
3. Power efficiency. Running a 7B model on an M3 MacBook Pro draws ~15-25W. A desktop RTX 4090 draws 300-450W for similar work. You can run AI on battery for hours.

The trade-off vs. a desktop PC is absolute top speed. An RTX 4090 generates tokens faster than any Mac. But for most users, a MacBook Pro with sufficient memory offers the best balance of model capability, speed, and portability.

Best models for each Mac memory tier

Here is exactly which models you can run at each unified memory configuration. These numbers assume Q4_K_M quantization and 8K context.

Key insight: a MacBook Pro with 64 GB unified memory runs the same models as a dual-RTX 3090 PC setup (48 GB), but in a laptop form factor. This is the Mac's killer advantage for local AI.

Use our VRAM calculator to check specific model + quantization combinations.

Setting up Ollama on Mac

Ollama is the easiest way to run local LLMs on a Mac. It automatically uses Metal GPU acceleration and downloads optimally quantized models.

# Install Ollama
curl -fsSL https://ollama.com/install.sh | sh

# Download and run a 7B model
ollama run qwen2.5:7b

# For more context (16K instead of default 2K)
ollama run qwen2.5:7b --context-size 16384

# Run a 14B model (needs 24 GB+ unified memory)
ollama run qwen2.5:14b

Ollama automatically detects Apple Silicon and uses the Metal backend. You can verify acceleration by checking the log output — it should mention ggml_metal and show GPU layers being offloaded.

For advanced users, llama.cpp offers more control:

# Clone and build llama.cpp with Metal support
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
make clean && LLAMA_METAL=1 make -j

# Download a GGUF model and run it
./main -m qwen2.5-7b-q4_k_m.gguf -n 512 -ngl 99

The -ngl 99 flag offloads all layers to the GPU. Lower numbers offload fewer layers and use more CPU + system RAM.

Performance benchmarks

Real-world token generation speeds for a 7B Q4_K_M model on different Macs using Ollama with Metal acceleration:

For larger models, speeds decrease proportionally. A 70B Q4 model on an M4 Max 64 GB runs at ~15-25 tok/s — slower than 7B but still usable for reading and reasoning.

Thermal considerations

Running LLMs on a MacBook generates heat. Key facts:

• MacBook Air (fanless) will throttle after 2-5 minutes of continuous inference. The M1/M2/M3 Air is fine for occasional queries but not sustained batch processing.
• MacBook Pro (with fans) maintains peak performance indefinitely. The fans spin up during sustained use (you will hear them at ~3500-5000 RPM) but the chip stays cool.
• Mac Mini / Mac Studio have excellent thermal performance due to the larger chassis and active cooling.
• Battery impact: Running a 7B model continuously drains a MacBook Pro battery in 3-5 hours. Plug in for extended sessions.

If you plan to do serious local AI work, get a MacBook Pro (not Air) or a Mac Mini/Studio. The fanless Air design is fine for experimentation but not for daily driver use.

eGPU is not supported on Apple Silicon

A common question: can you add an external GPU to a MacBook for faster inference? No. Apple removed eGPU support starting with the M1 series. Thunderbolt eGPU enclosures that worked with Intel Macs do not work with Apple Silicon.

This means your Mac's unified memory is the only pool available. If you bought an 8 GB MacBook Air, you cannot upgrade later — you are capped at small 1-3B models. Choose your memory tier carefully at purchase time.

For users who need more capability later, consider a cloud fallback. Plugsky gives you access to 30+ models with no hardware requirements.

Tips and best practices

• Buy as much memory as you can afford at purchase time. You cannot add more later. 16 GB is the minimum for useful work. 24-36 GB is the sweet spot. 64 GB+ is future-proof.
• Use Q4_K_M quantization. It offers the best quality-to-size ratio for most models. Q8 is better for quality but needs 2x the memory. Q2 fits more models but the quality loss is noticeable.
• Close other apps when running large models. macOS uses system memory too. A 64 GB Mac with 20 Chrome tabs open only has ~50 GB available for your model.
• Use the Ollama API. Ollama exposes http://localhost:11434/v1 with an OpenAI-compatible API. Point any OpenAI client at it and enjoy local inference with your favourite frontend.
• Monitor your swap. If your model does not fit in unified memory, macOS swaps to disk. Performance drops to 1-2 tok/s. Check Activity Monitor for "memory pressure" — if it is in the red zone, use a smaller model.

Frequently asked questions