Hardware · 22 cards and machines
Which GPU to run an LLM?
Consumer cards, pro workstations, datacenter and unified-memory machines (Mac, DGX Spark, AMD mini-PC): VRAM, bandwidth, price per GB and the biggest model they load.
Pro / workstation€9,000
RTX PRO 6000 Blackwell 96 GB96GB
- Bandwidth
- 1,792 GB/s
- Power
- 600 W
- € per usable GB
- €103
- Llama 8B Q4
- ~199 tok/s
Biggest model on its own (Q4): Mistral Large 123B
Pro / workstation€7,000
RTX 6000 Ada 48 GB48GB
- Bandwidth
- 960 GB/s
- Power
- 300 W
- € per usable GB
- €165
- Llama 8B Q4
- ~107 tok/s
Biggest model on its own (Q4): Llama 3.3 70B
No card found.