Mistral Large 123B: how much VRAM?
In Q4_K_M with 8k tokens of context, Mistral Large 123B needs about 81 GB of memory. The cheapest option: Mini-PC Ryzen AI Max+ 395 128 GB (€2,300 for the complete machine).
Model sheet
ollama run mistral-largeWeights on Hugging Face: mistralai/Mistral-Large-Instruct-2411
VRAM by precision and context
| Precision | 4k | 8k | 32k | 128k |
|---|---|---|---|---|
| FP16 | 260 GB | 261 GB | 270 GB | 302 GB |
| FP8 | 131 GB | 132 GB | 140 GB | 173 GB |
| Q8_0 | 139 GB | 140 GB | 148 GB | 181 GB |
| Q6_K | 108 GB | 109 GB | 117 GB | 150 GB |
| Q4_K_M | 79 GB | 81 GB | 89 GB | 121 GB |
| Q3_K_M | 65 GB | 66 GB | 75 GB | 107 GB |
One user. Each simultaneous user adds their own context cache.
Frequently asked questions
How much VRAM does Mistral Large 123B need?
With an 8k-token context and one user, Mistral Large 123B needs about 81 GB in Q4, 140 GB in Q8 and 261 GB in FP16.
What is the cheapest hardware to run Mistral Large 123B?
In Q4, the cheapest option in our catalogue to run Mistral Large 123B is Mini-PC Ryzen AI Max+ 395 128 GB, about €2,300 for the complete machine.
How fast does Mistral Large 123B run?
Expect around 1.9 tokens per second for one user on Mini-PC Ryzen AI Max+ 395 128 GB in Q4 (estimate based on memory bandwidth).
On which hardware?
| Hardware | Cards | Speed | Machine |
|---|---|---|---|
| Mini-PC Ryzen AI Max+ 395 128 GB | 1 is enough | ~1.9 tok/s | €2,300 |
| NVIDIA DGX Spark 128 GB | 1 is enough | ~2.0 tok/s | €4,000 |
| Mac Studio M4 Max 128 GB | 1 is enough | ~4.1 tok/s | €4,400 |
| RTX 3090 24 GB (used) | 4 cards | ~21 tok/s | €5,800 |
| Radeon RX 7900 XTX 24 GB | 4 cards | ~21 tok/s | €6,600 |
| RTX PRO 6000 Blackwell 96 GB | 1 is enough | ~13 tok/s | €9,900 |
| RTX 5090 32 GB | 3 cards | ~30 tok/s | €10,600 |
| RTX 4090 24 GB | 4 cards | ~22 tok/s | €10,800 |
| Mac Studio M3 Ultra 512 GB | 1 is enough | ~6.1 tok/s | €11,500 |
| RTX 6000 Ada 48 GB | 2 cards | ~11 tok/s | €14,900 |
| L40S 48 GB | 2 cards | ~9.6 tok/s | €21,000 |
| A100 80 GB | 2 cards | ~23 tok/s | €28,000 |
| H200 141 GB | 1 is enough | ~36 tok/s | €39,000 |
| B200 192 GB | 1 is enough | ~59 tok/s | €48,000 |
| H100 80 GB | 2 cards | ~37 tok/s | €62,000 |
| RTX 3060 12 GB | doesn’t fit | — | — |
| RTX 4060 Ti 16 GB | doesn’t fit | — | — |
| Radeon RX 7800 XT 16 GB | doesn’t fit | — | — |
| RTX 4070 12 GB | doesn’t fit | — | — |
| RTX 4070 Ti SUPER 16 GB | doesn’t fit | — | — |
| RTX 4080 SUPER 16 GB | doesn’t fit | — | — |
| RTX 5080 16 GB | doesn’t fit | — | — |
Estimated complete machine: cards + host PC (€900 for 1–2 consumer cards, €2,800 beyond, a €6,000 server for datacenter cards). Macs and mini-PCs are sold complete.
Similar models
From: RTX 3060 12 GB · €1,180
From: 2 × RTX 3060 12 GB · €1,460
From: 2 × RTX 4060 Ti 16 GB · €1,800
From: Mini-PC Ryzen AI Max+ 395 128 GB · €2,300