gpt-oss-120b: how much VRAM?
In Q4_K_M with 8k tokens of context, gpt-oss-120b needs about 75 GB of memory. The cheapest option: Mini-PC Ryzen AI Max+ 395 128 GB (€2,300 for the complete machine).
Model sheet
ollama run gpt-oss:120bWeights on Hugging Face: openai/gpt-oss-120b
VRAM by precision and context
| Precision | 4k | 8k | 32k | 128k |
|---|---|---|---|---|
| FP16 | 246 GB | 247 GB | 248 GB | 255 GB |
| FP8 | 124 GB | 124 GB | 125 GB | 132 GB |
| Q8_0 | 131 GB | 131 GB | 133 GB | 140 GB |
| Q6_K | 101 GB | 102 GB | 103 GB | 110 GB |
| Q4_K_M | 74 GB | 75 GB | 76 GB | 83 GB |
| Q3_K_M | 61 GB | 61 GB | 63 GB | 70 GB |
One user. Each simultaneous user adds their own context cache.
Frequently asked questions
How much VRAM does gpt-oss-120b need?
With an 8k-token context and one user, gpt-oss-120b needs about 75 GB in Q4, 131 GB in Q8 and 247 GB in FP16.
What is the cheapest hardware to run gpt-oss-120b?
In Q4, the cheapest option in our catalogue to run gpt-oss-120b is Mini-PC Ryzen AI Max+ 395 128 GB, about €2,300 for the complete machine.
How fast does gpt-oss-120b run?
Expect around 44 tokens per second for one user on Mini-PC Ryzen AI Max+ 395 128 GB in Q4 (estimate based on memory bandwidth).
On which hardware?
| Hardware | Cards | Speed | Machine |
|---|---|---|---|
| Mini-PC Ryzen AI Max+ 395 128 GB | 1 is enough | ~44 tok/s | €2,300 |
| NVIDIA DGX Spark 128 GB | 1 is enough | ~47 tok/s | €4,000 |
| Mac Studio M4 Max 128 GB | 1 is enough | ~94 tok/s | €4,400 |
| RTX 3090 24 GB (used) | 4 cards | ~481 tok/s | €5,800 |
| Radeon RX 7900 XTX 24 GB | 4 cards | ~493 tok/s | €6,600 |
| RTX PRO 6000 Blackwell 96 GB | 1 is enough | ~307 tok/s | €9,900 |
| RTX 5090 32 GB | 3 cards | ~691 tok/s | €10,600 |
| RTX 4090 24 GB | 4 cards | ~518 tok/s | €10,800 |
| Mac Studio M3 Ultra 512 GB | 1 is enough | ~140 tok/s | €11,500 |
| RTX 6000 Ada 48 GB | 2 cards | ~247 tok/s | €14,900 |
| A100 80 GB | 1 is enough | ~349 tok/s | €17,000 |
| L40S 48 GB | 2 cards | ~222 tok/s | €21,000 |
| H100 80 GB | 1 is enough | ~574 tok/s | €34,000 |
| H200 141 GB | 1 is enough | ~822 tok/s | €39,000 |
| B200 192 GB | 1 is enough | ~1,371 tok/s | €48,000 |
| RTX 3060 12 GB | doesn’t fit | — | — |
| RTX 4060 Ti 16 GB | doesn’t fit | — | — |
| Radeon RX 7800 XT 16 GB | doesn’t fit | — | — |
| RTX 4070 12 GB | doesn’t fit | — | — |
| RTX 4070 Ti SUPER 16 GB | doesn’t fit | — | — |
| RTX 4080 SUPER 16 GB | doesn’t fit | — | — |
| RTX 5080 16 GB | doesn’t fit | — | — |
Estimated complete machine: cards + host PC (€900 for 1–2 consumer cards, €2,800 beyond, a €6,000 server for datacenter cards). Macs and mini-PCs are sold complete.
Similar models
From: RTX 4060 Ti 16 GB · €1,350
From: Mini-PC Ryzen AI Max+ 395 128 GB · €2,300
From: Mini-PC Ryzen AI Max+ 395 128 GB · €2,300