Consumer · GDDR6
Radeon RX 7800 XT 16 GB: which LLMs can it run?
On its own, it loads up to gpt-oss-20b in Q4. With several cards, memory adds up and opens bigger models.
Specifications
Memory16 GB
Bandwidth624 GB/s
FP16 compute~74 TF
Power263 W
Indicative price€500
Cloud rental€0.15/h
Against cards at the same price
Models and expected speed
| Model | Q4 | Q8 | FP16 |
|---|---|---|---|
| Qwen3 4B | ✓ ~135 tok/s | ✓ ~78 tok/s | ✓ ~42 tok/s |
| Llama 3.1 8B | ✓ ~69 tok/s | ✓ ~40 tok/s | 2 × · ~32 tok/s |
| Qwen3 8B | ✓ ~68 tok/s | ✓ ~39 tok/s | 2 × · ~31 tok/s |
| Gemma 3 12B | ✓ ~47 tok/s | ✓ ~27 tok/s | 2 × · ~21 tok/s |
| Mistral Nemo 12B | ✓ ~46 tok/s | ✓ ~26 tok/s | 2 × · ~21 tok/s |
| Phi-4 14B | ✓ ~38 tok/s | 2 × · ~33 tok/s | 3 × · ~26 tok/s |
| Qwen3 14B | ✓ ~38 tok/s | 2 × · ~33 tok/s | 3 × · ~26 tok/s |
| gpt-oss-20b | ✓ ~149 tok/s | 2 × · ~130 tok/s | 3 × · ~105 tok/s |
| Mistral Small 24B | 2 × · ~35 tok/s | 2 × · ~20 tok/s | 4 × · ~21 tok/s |
| Gemma 3 27B | 2 × · ~31 tok/s | 3 × · ~27 tok/s | 4 × · ~19 tok/s |
| Qwen3.8 27B | 2 × · ~30 tok/s | 3 × · ~26 tok/s | 4 × · ~18 tok/s |
| Qwen3 30B-A3B | 2 × · ~242 tok/s | 3 × · ~212 tok/s | ✗ |
| Qwen3 32B | 2 × · ~26 tok/s | 3 × · ~22 tok/s | ✗ |
| Qwen2.5 Coder 32B | 2 × · ~26 tok/s | 3 × · ~22 tok/s | ✗ |
| DeepSeek R1 Distill 32B | 2 × · ~26 tok/s | 3 × · ~22 tok/s | ✗ |
| Mixtral 8x7B | 2 × · ~65 tok/s | 4 × · ~74 tok/s | ✗ |
| Llama 3.3 70B | 4 × · ~24 tok/s | ✗ | ✗ |
| GLM-4.5-Air 106B | ✗ | ✗ | ✗ |
| gpt-oss-120b | ✗ | ✗ | ✗ |
| Mistral Large 123B | ✗ | ✗ | ✗ |
| Qwen3 235B-A22B | ✗ | ✗ | ✗ |
| GLM-4.5 355B | ✗ | ✗ | ✗ |
| Llama 3.1 405B | ✗ | ✗ | ✗ |
| DeepSeek R1 671B | ✗ | ✗ | ✗ |
| DeepSeek V3.1 671B | ✗ | ✗ | ✗ |
| Kimi K2 1T | ✗ | ✗ | ✗ |
8k tokens, one user. “2 ×” = number of cards needed, speed with tensor parallelism (vLLM). ✗ = doesn’t fit, even with 4 cards.