Hardware · 22 cards and machines
Which GPU to run an LLM?
Consumer cards, pro workstations, datacenter and unified-memory machines (Mac, DGX Spark, AMD mini-PC): VRAM, bandwidth, price per GB and the biggest model they load.
Unified memory€11,500
Mac Studio M3 Ultra 512 GB512GB unified
- Bandwidth
- 819 GB/s
- Power
- 270 W
- € per usable GB
- €30
- Llama 8B Q4
- ~91 tok/s
Biggest model on its own (Q4): Llama 3.1 405B
Unified memory€2,300
Mini-PC Ryzen AI Max+ 395 128 GB128GB unified
- Bandwidth
- 256 GB/s
- Power
- 140 W
- € per usable GB
- €21
- Llama 8B Q4
- ~28 tok/s
Biggest model on its own (Q4): Mistral Large 123B
Unified memory€4,000
NVIDIA DGX Spark 128 GB128GB unified
- Bandwidth
- 273 GB/s
- Power
- 240 W
- € per usable GB
- €37
- Llama 8B Q4
- ~30 tok/s
Biggest model on its own (Q4): Mistral Large 123B
Unified memory€4,400
Mac Studio M4 Max 128 GB128GB unified
- Bandwidth
- 546 GB/s
- Power
- 150 W
- € per usable GB
- €46
- Llama 8B Q4
- ~61 tok/s
Biggest model on its own (Q4): Mistral Large 123B
No card found.