Consumer · GDDR6
RTX 3060 12 GB: which LLMs can it run?
On its own, it loads up to Qwen3 14B in Q4. With several cards, memory adds up and opens bigger models.
Specifications
Memory12 GB
Bandwidth360 GB/s
FP16 compute~51 TF
Power170 W
Indicative price€280
Cloud rental€0.10/h
Against cards at the same price
Recipes with this hardware
Models and expected speed
| Model | Q4 | Q8 | FP16 |
|---|---|---|---|
| Qwen3 4B | ✓ ~78 tok/s | ✓ ~45 tok/s | ✓ ~24 tok/s |
| Llama 3.1 8B | ✓ ~40 tok/s | ✓ ~23 tok/s | 2 × · ~18 tok/s |
| Qwen3 8B | ✓ ~39 tok/s | ✓ ~22 tok/s | 2 × · ~18 tok/s |
| Gemma 3 12B | ✓ ~27 tok/s | 2 × · ~23 tok/s | 3 × · ~18 tok/s |
| Mistral Nemo 12B | ✓ ~27 tok/s | 2 × · ~23 tok/s | 3 × · ~18 tok/s |
| Phi-4 14B | ✓ ~22 tok/s | 2 × · ~19 tok/s | 3 × · ~15 tok/s |
| Qwen3 14B | ✓ ~22 tok/s | 2 × · ~19 tok/s | 3 × · ~15 tok/s |
| gpt-oss-20b | 2 × · ~129 tok/s | 3 × · ~112 tok/s | 4 × · ~81 tok/s |
| Mistral Small 24B | 2 × · ~20 tok/s | 3 × · ~17 tok/s | ✗ |
| Gemma 3 27B | 2 × · ~18 tok/s | 3 × · ~15 tok/s | ✗ |
| Qwen3.8 27B | 2 × · ~18 tok/s | 3 × · ~15 tok/s | ✗ |
| Qwen3 30B-A3B | 2 × · ~139 tok/s | 4 × · ~163 tok/s | ✗ |
| Qwen3 32B | 2 × · ~15 tok/s | 4 × · ~17 tok/s | ✗ |
| Qwen2.5 Coder 32B | 2 × · ~15 tok/s | 4 × · ~17 tok/s | ✗ |
| DeepSeek R1 Distill 32B | 2 × · ~15 tok/s | 4 × · ~17 tok/s | ✗ |
| Mixtral 8x7B | 3 × · ~56 tok/s | ✗ | ✗ |
| Llama 3.3 70B | ✗ | ✗ | ✗ |
| GLM-4.5-Air 106B | ✗ | ✗ | ✗ |
| gpt-oss-120b | ✗ | ✗ | ✗ |
| Mistral Large 123B | ✗ | ✗ | ✗ |
| Qwen3 235B-A22B | ✗ | ✗ | ✗ |
| GLM-4.5 355B | ✗ | ✗ | ✗ |
| Llama 3.1 405B | ✗ | ✗ | ✗ |
| DeepSeek R1 671B | ✗ | ✗ | ✗ |
| DeepSeek V3.1 671B | ✗ | ✗ | ✗ |
| Kimi K2 1T | ✗ | ✗ | ✗ |
8k tokens, one user. “2 ×” = number of cards needed, speed with tensor parallelism (vLLM). ✗ = doesn’t fit, even with 4 cards.