Llama 3.2 3B

3.2B parameters
reasoningcoding
Parameters
3.2B
Context Length
128K
Architecture
llama
Release Date
2024

Available Quantizations

QuantizationFile SizeQuality
Q2_K1.3 GBLower
Q3_K_M1.6 GBLower
Q4_K_M2.0 GBGood
Q5_K_M2.3 GBGood
Q8_03.4 GBHigh

Can it run? (Q4_K_M quantization)

GeForce RTX 4090
Runs great
~364 tok/s • 24GB VRAM
GeForce RTX 4070
Runs great
~182 tok/s • 12GB VRAM
GeForce RTX 3060 12GB
Runs great
~130 tok/s • 12GB VRAM
Radeon RX 7900 XTX
Runs great
~346 tok/s • 24GB VRAM
Check with your specific GPU →