Consumer · GDDR7
RTX 5090 32 GB: which LLMs can it run?
On its own, it loads up to Mixtral 8x7B in Q4. With several cards, memory adds up and opens bigger models.
Specifications
Memory32 GB
Bandwidth1,792 GB/s
FP16 compute~210 TF
Power575 W
Indicative price€2,600
Cloud rental€0.80/h
Against cards at the same price
Recipes with this hardware
Models and expected speed
| Model | Q4 | Q8 | FP16 |
|---|---|---|---|
| Qwen3 4B | ✓ ~387 tok/s | ✓ ~225 tok/s | ✓ ~121 tok/s |
| Llama 3.1 8B | ✓ ~199 tok/s | ✓ ~114 tok/s | ✓ ~61 tok/s |
| Qwen3 8B | ✓ ~194 tok/s | ✓ ~111 tok/s | ✓ ~60 tok/s |
| Gemma 3 12B | ✓ ~134 tok/s | ✓ ~77 tok/s | ✓ ~41 tok/s |
| Mistral Nemo 12B | ✓ ~132 tok/s | ✓ ~75 tok/s | ✓ ~40 tok/s |
| Phi-4 14B | ✓ ~110 tok/s | ✓ ~63 tok/s | 2 × · ~50 tok/s |
| Qwen3 14B | ✓ ~109 tok/s | ✓ ~62 tok/s | 2 × · ~50 tok/s |
| gpt-oss-20b | ✓ ~427 tok/s | ✓ ~249 tok/s | 2 × · ~201 tok/s |
| Mistral Small 24B | ✓ ~68 tok/s | ✓ ~39 tok/s | 2 × · ~31 tok/s |
| Gemma 3 27B | ✓ ~60 tok/s | 2 × · ~51 tok/s | 2 × · ~27 tok/s |
| Qwen3.8 27B | ✓ ~58 tok/s | 2 × · ~50 tok/s | 2 × · ~26 tok/s |
| Qwen3 30B-A3B | ✓ ~463 tok/s | 2 × · ~405 tok/s | 3 × · ~329 tok/s |
| Qwen3 32B | ✓ ~50 tok/s | 2 × · ~42 tok/s | 3 × · ~34 tok/s |
| Qwen2.5 Coder 32B | ✓ ~50 tok/s | 2 × · ~42 tok/s | 3 × · ~34 tok/s |
| DeepSeek R1 Distill 32B | ✓ ~50 tok/s | 2 × · ~42 tok/s | 3 × · ~34 tok/s |
| Mixtral 8x7B | ✓ ~125 tok/s | 2 × · ~107 tok/s | 4 × · ~114 tok/s |
| Llama 3.3 70B | 2 × · ~35 tok/s | 3 × · ~30 tok/s | ✗ |
| GLM-4.5-Air 106B | 3 × · ~302 tok/s | 4 × · ~230 tok/s | ✗ |
| gpt-oss-120b | 3 × · ~691 tok/s | ✗ | ✗ |
| Mistral Large 123B | 3 × · ~30 tok/s | ✗ | ✗ |
| Qwen3 235B-A22B | ✗ | ✗ | ✗ |
| GLM-4.5 355B | ✗ | ✗ | ✗ |
| Llama 3.1 405B | ✗ | ✗ | ✗ |
| DeepSeek R1 671B | ✗ | ✗ | ✗ |
| DeepSeek V3.1 671B | ✗ | ✗ | ✗ |
| Kimi K2 1T | ✗ | ✗ | ✗ |
8k tokens, one user. “2 ×” = number of cards needed, speed with tensor parallelism (vLLM). ✗ = doesn’t fit, even with 4 cards.