Hardware · 22 cards and machines

Which GPU to run an LLM?

Consumer cards, pro workstations, datacenter and unified-memory machines (Mac, DGX Spark, AMD mini-PC): VRAM, bandwidth, price per GB and the biggest model they load.

All22 Consumer11 Pro / workstation2 Datacenter5 Unified memory4

Datacenter€42,000

B200 192 GB

192GB

Bandwidth: 8,000 GB/s
Power: 1000 W
€ per usable GB: €250
Llama 8B Q4: ~889 tok/s

Biggest model on its own (Q4): Qwen3 235B-A22B

Datacenter€33,000

H200 141 GB

141GB

Bandwidth: 4,800 GB/s
Power: 700 W
€ per usable GB: €277
Llama 8B Q4: ~533 tok/s

Biggest model on its own (Q4): Mistral Large 123B

Datacenter€11,000

A100 80 GB

80GB

Bandwidth: 2,039 GB/s
Power: 300 W
€ per usable GB: €213
Llama 8B Q4: ~227 tok/s

Biggest model on its own (Q4): gpt-oss-120b

Datacenter€28,000

H100 80 GB

80GB

Bandwidth: 3,350 GB/s
Power: 700 W
€ per usable GB: €425
Llama 8B Q4: ~372 tok/s

Biggest model on its own (Q4): gpt-oss-120b

Datacenter€7,500

L40S 48 GB

48GB

Bandwidth: 864 GB/s
Power: 350 W
€ per usable GB: €281
Llama 8B Q4: ~96 tok/s

Biggest model on its own (Q4): Llama 3.3 70B