FR
← All hardware
Datacenter · HBM3e

H200 141 GB: which LLMs can it run?

On its own, it loads up to Mistral Large 123B in Q4. With several cards, memory adds up and opens bigger models.

Specifications

Memory141 GB
Bandwidth4,800 GB/s
FP16 compute~989 TF
Power700 W
Indicative price€33,000
Cloud rental€3.60/h
Build a server with it →Compare

Against cards at the same price

H200 141 GB vs H100 80 GB€28,000
H200 141 GB vs B200 192 GB€42,000
H200 141 GB vs Mac Studio M3 Ultra 512 GB€11,500

Recipes with this hardware

Full DeepSeek R1 in FP8€270,000

Models and expected speed

ModelQ4Q8FP16
Qwen3 4B✓ ~1,035 tok/s✓ ~601 tok/s✓ ~324 tok/s
Llama 3.1 8B✓ ~533 tok/s✓ ~306 tok/s✓ ~163 tok/s
Qwen3 8B✓ ~521 tok/s✓ ~299 tok/s✓ ~160 tok/s
Gemma 3 12B✓ ~359 tok/s✓ ~205 tok/s✓ ~109 tok/s
Mistral Nemo 12B✓ ~353 tok/s✓ ~202 tok/s✓ ~108 tok/s
Phi-4 14B✓ ~294 tok/s✓ ~168 tok/s✓ ~89 tok/s
Qwen3 14B✓ ~292 tok/s✓ ~167 tok/s✓ ~89 tok/s
gpt-oss-20b✓ ~1,143 tok/s✓ ~666 tok/s✓ ~359 tok/s
Mistral Small 24B✓ ~181 tok/s✓ ~103 tok/s✓ ~55 tok/s
Gemma 3 27B✓ ~161 tok/s✓ ~92 tok/s✓ ~49 tok/s
Qwen3.8 27B✓ ~156 tok/s✓ ~89 tok/s✓ ~47 tok/s
Qwen3 30B-A3B✓ ~1,239 tok/s✓ ~724 tok/s✓ ~391 tok/s
Qwen3 32B✓ ~133 tok/s✓ ~76 tok/s✓ ~40 tok/s
Qwen2.5 Coder 32B✓ ~133 tok/s✓ ~76 tok/s✓ ~40 tok/s
DeepSeek R1 Distill 32B✓ ~133 tok/s✓ ~76 tok/s✓ ~40 tok/s
Mixtral 8x7B✓ ~335 tok/s✓ ~191 tok/s✓ ~102 tok/s
Llama 3.3 70B✓ ~62 tok/s✓ ~35 tok/s2 × · ~28 tok/s
GLM-4.5-Air 106B✓ ~359 tok/s✓ ~205 tok/s2 × · ~164 tok/s
gpt-oss-120b✓ ~822 tok/s✓ ~475 tok/s2 × · ~383 tok/s
Mistral Large 123B✓ ~36 tok/s✓ ~20 tok/s2 × · ~16 tok/s
Qwen3 235B-A22B2 × · ~297 tok/s2 × · ~169 tok/s4 × · ~179 tok/s
GLM-4.5 355B2 × · ~205 tok/s3 × · ~174 tok/s6 × · ~185 tok/s
Llama 3.1 405B2 × · ~16 tok/s4 × · ~18 tok/s7 × · ~17 tok/s
DeepSeek R1 671B4 × · ~354 tok/s6 × · ~302 tok/s✗
DeepSeek V3.1 671B4 × · ~354 tok/s6 × · ~302 tok/s✗
Kimi K2 1T5 × · ~512 tok/s✗✗

8k tokens, one user. “2 ×” = number of cards needed, speed with tensor parallelism (vLLM). ✗ = doesn’t fit, even with 8 cards.