DeepSeek V3.1 671B: how much VRAM?
In Q4_K_M with 8k tokens of context, DeepSeek V3.1 671B needs about 424 GB of memory. The cheapest option: 6 × A100 80 GB (€72,000 for the complete machine).
Model sheet
vllm serve deepseek-ai/DeepSeek-V3.1Weights on Hugging Face: deepseek-ai/DeepSeek-V3.1
VRAM by precision and context
| Precision | 4k | 8k | 32k | 128k |
|---|---|---|---|---|
| FP16 | 1,410 GB | 1,410 GB | 1,412 GB | 1,418 GB |
| FP8 | 705 GB | 706 GB | 707 GB | 714 GB |
| Q8_0 | 748 GB | 748 GB | 749 GB | 756 GB |
| Q6_K | 578 GB | 579 GB | 580 GB | 587 GB |
| Q4_K_M | 423 GB | 424 GB | 425 GB | 432 GB |
| Q3_K_M | 346 GB | 346 GB | 348 GB | 355 GB |
One user. Each simultaneous user adds their own context cache.
Frequently asked questions
How much VRAM does DeepSeek V3.1 671B need?
With an 8k-token context and one user, DeepSeek V3.1 671B needs about 424 GB in Q4, 748 GB in Q8 and 1,410 GB in FP16.
What is the cheapest hardware to run DeepSeek V3.1 671B?
In Q4, the cheapest option in our catalogue to run DeepSeek V3.1 671B is 6 × A100 80 GB, about €72,000 for the complete machine.
How fast does DeepSeek V3.1 671B run?
Expect around 226 tokens per second for one user on 6 × A100 80 GB in Q4 (estimate based on memory bandwidth).
On which hardware?
| Hardware | Cards | Speed | Machine |
|---|---|---|---|
| A100 80 GB | 6 cards | ~226 tok/s | €72,000 |
| B200 192 GB | 3 cards | ~443 tok/s | €132,000 |
| H200 141 GB | 4 cards | ~354 tok/s | €138,000 |
| H100 80 GB | 6 cards | ~371 tok/s | €174,000 |
| RTX 3060 12 GB | doesn’t fit | — | — |
| RTX 4060 Ti 16 GB | doesn’t fit | — | — |
| Radeon RX 7800 XT 16 GB | doesn’t fit | — | — |
| RTX 4070 12 GB | doesn’t fit | — | — |
| RTX 3090 24 GB (used) | doesn’t fit | — | — |
| RTX 4070 Ti SUPER 16 GB | doesn’t fit | — | — |
| Radeon RX 7900 XTX 24 GB | doesn’t fit | — | — |
| RTX 4080 SUPER 16 GB | doesn’t fit | — | — |
| RTX 5080 16 GB | doesn’t fit | — | — |
| RTX 4090 24 GB | doesn’t fit | — | — |
| Mini-PC Ryzen AI Max+ 395 128 GB | doesn’t fit | — | — |
| RTX 5090 32 GB | doesn’t fit | — | — |
| NVIDIA DGX Spark 128 GB | doesn’t fit | — | — |
| Mac Studio M4 Max 128 GB | doesn’t fit | — | — |
| RTX 6000 Ada 48 GB | doesn’t fit | — | — |
| L40S 48 GB | doesn’t fit | — | — |
| RTX PRO 6000 Blackwell 96 GB | doesn’t fit | — | — |
| Mac Studio M3 Ultra 512 GB | doesn’t fit | — | — |
Estimated complete machine: cards + host PC (€900 for 1–2 consumer cards, €2,800 beyond, a €6,000 server for datacenter cards). Macs and mini-PCs are sold complete.
Similar models
From: 2 × RTX 3060 12 GB · €1,460
From: 6 × A100 80 GB · €72,000
From: 5 × H200 141 GB · €171,000