Mistral Small 3.2 24B

24B parameters
reasoningcodingmultilingual
Parameters
24B
Context Length
128K
Architecture
mistral
Release Date
2025

Available Quantizations

QuantizationFile SizeQuality
Q8_025.6 GBHigh
Q6_K19.9 GBHigh
Q5_K_M17.5 GBGood
Q4_K_M14.7 GBGood
Q3_K_M12.5 GBLower

Can it run? (Q4_K_M quantization)

GeForce RTX 4090
Runs great
~48 tok/s • 24GB VRAM
GeForce RTX 4070
Runs slowly
~5 tok/s • 12GB VRAM
GeForce RTX 3060 12GB
Runs slowly
~3 tok/s • 12GB VRAM
Radeon RX 7900 XTX
Runs great
~46 tok/s • 24GB VRAM
Check with your specific GPU →