Hardware · 22 cards and machines

Which GPU to run an LLM?

Consumer cards, pro workstations, datacenter and unified-memory machines (Mac, DGX Spark, AMD mini-PC): VRAM, bandwidth, price per GB and the biggest model they load.

All22 Consumer11 Pro / workstation2 Datacenter5 Unified memory4

Unified memory€11,500

Mac Studio M3 Ultra 512 GB

Bandwidth: 819 GB/s
Power: 270 W
€ per usable GB: €30
Llama 8B Q4: ~91 tok/s

Biggest model on its own (Q4): Llama 3.1 405B

Datacenter€42,000

B200 192 GB

Bandwidth: 8,000 GB/s
Power: 1000 W
€ per usable GB: €250
Llama 8B Q4: ~889 tok/s

Biggest model on its own (Q4): Qwen3 235B-A22B

Datacenter€33,000

H200 141 GB

Bandwidth: 4,800 GB/s
Power: 700 W
€ per usable GB: €277
Llama 8B Q4: ~533 tok/s

Biggest model on its own (Q4): Mistral Large 123B

Unified memory€2,300

Mini-PC Ryzen AI Max+ 395 128 GB

Bandwidth: 256 GB/s
Power: 140 W
€ per usable GB: €21
Llama 8B Q4: ~28 tok/s

Biggest model on its own (Q4): Mistral Large 123B

Unified memory€4,000

NVIDIA DGX Spark 128 GB

Bandwidth: 273 GB/s
Power: 240 W
€ per usable GB: €37
Llama 8B Q4: ~30 tok/s

Biggest model on its own (Q4): Mistral Large 123B

Unified memory€4,400

Mac Studio M4 Max 128 GB

Bandwidth: 546 GB/s
Power: 150 W
€ per usable GB: €46
Llama 8B Q4: ~61 tok/s

Biggest model on its own (Q4): Mistral Large 123B

Pro / workstation€9,000

RTX PRO 6000 Blackwell 96 GB

Bandwidth: 1,792 GB/s
Power: 600 W
€ per usable GB: €103
Llama 8B Q4: ~199 tok/s

Biggest model on its own (Q4): Mistral Large 123B

Datacenter€11,000

Bandwidth: 2,039 GB/s
Power: 300 W
€ per usable GB: €213
Llama 8B Q4: ~227 tok/s

Biggest model on its own (Q4): gpt-oss-120b

Datacenter€28,000

Bandwidth: 3,350 GB/s
Power: 700 W
€ per usable GB: €425
Llama 8B Q4: ~372 tok/s

Biggest model on its own (Q4): gpt-oss-120b

Pro / workstation€7,000

RTX 6000 Ada 48 GB

Bandwidth: 960 GB/s
Power: 300 W
€ per usable GB: €165
Llama 8B Q4: ~107 tok/s

Biggest model on its own (Q4): Llama 3.3 70B

Datacenter€7,500

Bandwidth: 864 GB/s
Power: 350 W
€ per usable GB: €281
Llama 8B Q4: ~96 tok/s

Biggest model on its own (Q4): Llama 3.3 70B

Consumer€2,600

RTX 5090 32 GB

Bandwidth: 1,792 GB/s
Power: 575 W
€ per usable GB: €109
Llama 8B Q4: ~199 tok/s

Biggest model on its own (Q4): Mixtral 8x7B

RTX 3090 24 GB (used)

Bandwidth: 936 GB/s
Power: 350 W
€ per usable GB: €69
Llama 8B Q4: ~104 tok/s

Biggest model on its own (Q4): DeepSeek R1 Distill 32B

Consumer€2,000

RTX 4090 24 GB

Bandwidth: 1,008 GB/s
Power: 450 W
€ per usable GB: €121
Llama 8B Q4: ~112 tok/s

Biggest model on its own (Q4): DeepSeek R1 Distill 32B

Radeon RX 7900 XTX 24 GB

Bandwidth: 960 GB/s
Power: 355 W
€ per usable GB: €77
Llama 8B Q4: ~107 tok/s

Biggest model on its own (Q4): DeepSeek R1 Distill 32B

RTX 4060 Ti 16 GB

Bandwidth: 288 GB/s
Power: 165 W
€ per usable GB: €84
Llama 8B Q4: ~32 tok/s

Biggest model on its own (Q4): gpt-oss-20b

RTX 4070 Ti SUPER 16 GB

Bandwidth: 672 GB/s
Power: 285 W
€ per usable GB: €109
Llama 8B Q4: ~75 tok/s

Biggest model on its own (Q4): gpt-oss-20b

Consumer€1,050

RTX 4080 SUPER 16 GB

Bandwidth: 736 GB/s
Power: 320 W
€ per usable GB: €122
Llama 8B Q4: ~82 tok/s

Biggest model on its own (Q4): gpt-oss-20b

Consumer€1,200

RTX 5080 16 GB

Bandwidth: 960 GB/s
Power: 360 W
€ per usable GB: €131
Llama 8B Q4: ~107 tok/s

Biggest model on its own (Q4): gpt-oss-20b

Radeon RX 7800 XT 16 GB

Bandwidth: 624 GB/s
Power: 263 W
€ per usable GB: €88
Llama 8B Q4: ~69 tok/s

Biggest model on its own (Q4): gpt-oss-20b

RTX 3060 12 GB

Bandwidth: 360 GB/s
Power: 170 W
€ per usable GB: €98
Llama 8B Q4: ~40 tok/s

Biggest model on its own (Q4): Qwen3 14B

RTX 4070 12 GB

Bandwidth: 504 GB/s
Power: 200 W
€ per usable GB: €121
Llama 8B Q4: ~56 tok/s

Biggest model on its own (Q4): Qwen3 14B