FR
← All hardware
Datacenter · GDDR6 ECC

L40S 48 GB: which LLMs can it run?

On its own, it loads up to Llama 3.3 70B in Q4. With several cards, memory adds up and opens bigger models.

Specifications

Memory48 GB
Bandwidth864 GB/s
FP16 compute~362 TF
Power350 W
Indicative price€7,500
Cloud rental€1.00/h
Build a server with it →Compare

Against cards at the same price

L40S 48 GB vs RTX 6000 Ada 48 GB€7,000
L40S 48 GB vs RTX PRO 6000 Blackwell 96 GB€9,000
L40S 48 GB vs Mac Studio M4 Max 128 GB€4,400

Recipes with this hardware

Team server: 70B for 10 people€36,000

Models and expected speed

ModelQ4Q8FP16
Qwen3 4B✓ ~186 tok/s✓ ~108 tok/s✓ ~58 tok/s
Llama 3.1 8B✓ ~96 tok/s✓ ~55 tok/s✓ ~29 tok/s
Qwen3 8B✓ ~94 tok/s✓ ~54 tok/s✓ ~29 tok/s
Gemma 3 12B✓ ~65 tok/s✓ ~37 tok/s✓ ~20 tok/s
Mistral Nemo 12B✓ ~64 tok/s✓ ~36 tok/s✓ ~19 tok/s
Phi-4 14B✓ ~53 tok/s✓ ~30 tok/s✓ ~16 tok/s
Qwen3 14B✓ ~53 tok/s✓ ~30 tok/s✓ ~16 tok/s
gpt-oss-20b✓ ~206 tok/s✓ ~120 tok/s✓ ~65 tok/s
Mistral Small 24B✓ ~33 tok/s✓ ~19 tok/s2 × · ~15 tok/s
Gemma 3 27B✓ ~29 tok/s✓ ~17 tok/s2 × · ~13 tok/s
Qwen3.8 27B✓ ~28 tok/s✓ ~16 tok/s2 × · ~13 tok/s
Qwen3 30B-A3B✓ ~223 tok/s✓ ~130 tok/s2 × · ~106 tok/s
Qwen3 32B✓ ~24 tok/s✓ ~14 tok/s2 × · ~11 tok/s
Qwen2.5 Coder 32B✓ ~24 tok/s✓ ~14 tok/s2 × · ~11 tok/s
DeepSeek R1 Distill 32B✓ ~24 tok/s✓ ~14 tok/s2 × · ~11 tok/s
Mixtral 8x7B✓ ~60 tok/s2 × · ~52 tok/s3 × · ~41 tok/s
Llama 3.3 70B✓ ~11 tok/s2 × · ~9.5 tok/s4 × · ~10 tok/s
GLM-4.5-Air 106B2 × · ~97 tok/s3 × · ~83 tok/s5 × · ~74 tok/s
gpt-oss-120b2 × · ~222 tok/s3 × · ~192 tok/s6 × · ~207 tok/s
Mistral Large 123B2 × · ~9.6 tok/s4 × · ~11 tok/s6 × · ~8.7 tok/s
Qwen3 235B-A22B4 × · ~107 tok/s6 × · ~91 tok/s✗
GLM-4.5 355B5 × · ~92 tok/s✗✗
Llama 3.1 405B6 × · ~8.8 tok/s✗✗
DeepSeek R1 671B✗✗✗
DeepSeek V3.1 671B✗✗✗
Kimi K2 1T✗✗✗

8k tokens, one user. “2 ×” = number of cards needed, speed with tensor parallelism (vLLM). ✗ = doesn’t fit, even with 8 cards.