Hardware · 22 cards and machines
Which GPU to run an LLM?
Consumer cards, pro workstations, datacenter and unified-memory machines (Mac, DGX Spark, AMD mini-PC): VRAM, bandwidth, price per GB and the biggest model they load.
Consumer€2,600
RTX 5090 32 GB32GB
- Bandwidth
- 1,792 GB/s
- Power
- 575 W
- € per usable GB
- €109
- Llama 8B Q4
- ~199 tok/s
Biggest model on its own (Q4): Mixtral 8x7B
Consumer€750
RTX 3090 24 GB (used)24GB
- Bandwidth
- 936 GB/s
- Power
- 350 W
- € per usable GB
- €69
- Llama 8B Q4
- ~104 tok/s
Biggest model on its own (Q4): DeepSeek R1 Distill 32B
Consumer€2,000
RTX 4090 24 GB24GB
- Bandwidth
- 1,008 GB/s
- Power
- 450 W
- € per usable GB
- €121
- Llama 8B Q4
- ~112 tok/s
Biggest model on its own (Q4): DeepSeek R1 Distill 32B
Consumer€950
Radeon RX 7900 XTX 24 GB24GB
- Bandwidth
- 960 GB/s
- Power
- 355 W
- € per usable GB
- €77
- Llama 8B Q4
- ~107 tok/s
Biggest model on its own (Q4): DeepSeek R1 Distill 32B
Consumer€450
RTX 4060 Ti 16 GB16GB
- Bandwidth
- 288 GB/s
- Power
- 165 W
- € per usable GB
- €84
- Llama 8B Q4
- ~32 tok/s
Biggest model on its own (Q4): gpt-oss-20b
Consumer€850
RTX 4070 Ti SUPER 16 GB16GB
- Bandwidth
- 672 GB/s
- Power
- 285 W
- € per usable GB
- €109
- Llama 8B Q4
- ~75 tok/s
Biggest model on its own (Q4): gpt-oss-20b
Consumer€1,050
RTX 4080 SUPER 16 GB16GB
- Bandwidth
- 736 GB/s
- Power
- 320 W
- € per usable GB
- €122
- Llama 8B Q4
- ~82 tok/s
Biggest model on its own (Q4): gpt-oss-20b
Consumer€1,200
RTX 5080 16 GB16GB
- Bandwidth
- 960 GB/s
- Power
- 360 W
- € per usable GB
- €131
- Llama 8B Q4
- ~107 tok/s
Biggest model on its own (Q4): gpt-oss-20b
Consumer€500
Radeon RX 7800 XT 16 GB16GB
- Bandwidth
- 624 GB/s
- Power
- 263 W
- € per usable GB
- €88
- Llama 8B Q4
- ~69 tok/s
Biggest model on its own (Q4): gpt-oss-20b
Consumer€280
RTX 3060 12 GB12GB
- Bandwidth
- 360 GB/s
- Power
- 170 W
- € per usable GB
- €98
- Llama 8B Q4
- ~40 tok/s
Biggest model on its own (Q4): Qwen3 14B
Consumer€550
RTX 4070 12 GB12GB
- Bandwidth
- 504 GB/s
- Power
- 200 W
- € per usable GB
- €121
- Llama 8B Q4
- ~56 tok/s
Biggest model on its own (Q4): Qwen3 14B
No card found.