Kimi K2 1T: how much VRAM?
In Q4_K_M with 8k tokens of context, Kimi K2 1T needs about 631 GB of memory. The cheapest option: 5 × H200 141 GB (€171,000 for the complete machine).
Model sheet
vllm serve moonshotai/Kimi-K2-InstructWeights on Hugging Face: moonshotai/Kimi-K2-Instruct
VRAM by precision and context
| Precision | 4k | 8k | 32k | 128k |
|---|---|---|---|---|
| FP16 | 2,101 GB | 2,101 GB | 2,103 GB | 2,109 GB |
| FP8 | 1,051 GB | 1,051 GB | 1,053 GB | 1,059 GB |
| Q8_0 | 1,114 GB | 1,114 GB | 1,116 GB | 1,122 GB |
| Q6_K | 862 GB | 862 GB | 864 GB | 870 GB |
| Q4_K_M | 631 GB | 631 GB | 633 GB | 639 GB |
| Q3_K_M | 515 GB | 515 GB | 517 GB | 524 GB |
One user. Each simultaneous user adds their own context cache.
Frequently asked questions
How much VRAM does Kimi K2 1T need?
With an 8k-token context and one user, Kimi K2 1T needs about 631 GB in Q4, 1,114 GB in Q8 and 2,101 GB in FP16.
What is the cheapest hardware to run Kimi K2 1T?
In Q4, the cheapest option in our catalogue to run Kimi K2 1T is 5 × H200 141 GB, about €171,000 for the complete machine.
How fast does Kimi K2 1T run?
Expect around 512 tokens per second for one user on 5 × H200 141 GB in Q4 (estimate based on memory bandwidth).
On which hardware?
| Hardware | Cards | Speed | Machine |
|---|---|---|---|
| H200 141 GB | 5 cards | ~512 tok/s | €171,000 |
| B200 192 GB | 4 cards | ~682 tok/s | €174,000 |
| RTX 3060 12 GB | doesn’t fit | — | — |
| RTX 4060 Ti 16 GB | doesn’t fit | — | — |
| Radeon RX 7800 XT 16 GB | doesn’t fit | — | — |
| RTX 4070 12 GB | doesn’t fit | — | — |
| RTX 3090 24 GB (used) | doesn’t fit | — | — |
| RTX 4070 Ti SUPER 16 GB | doesn’t fit | — | — |
| Radeon RX 7900 XTX 24 GB | doesn’t fit | — | — |
| RTX 4080 SUPER 16 GB | doesn’t fit | — | — |
| RTX 5080 16 GB | doesn’t fit | — | — |
| RTX 4090 24 GB | doesn’t fit | — | — |
| Mini-PC Ryzen AI Max+ 395 128 GB | doesn’t fit | — | — |
| RTX 5090 32 GB | doesn’t fit | — | — |
| NVIDIA DGX Spark 128 GB | doesn’t fit | — | — |
| Mac Studio M4 Max 128 GB | doesn’t fit | — | — |
| RTX 6000 Ada 48 GB | doesn’t fit | — | — |
| L40S 48 GB | doesn’t fit | — | — |
| RTX PRO 6000 Blackwell 96 GB | doesn’t fit | — | — |
| A100 80 GB | doesn’t fit | — | — |
| Mac Studio M3 Ultra 512 GB | doesn’t fit | — | — |
| H100 80 GB | doesn’t fit | — | — |
Estimated complete machine: cards + host PC (€900 for 1–2 consumer cards, €2,800 beyond, a €6,000 server for datacenter cards). Macs and mini-PCs are sold complete.