Qwen 2.5 Coder 32B

32.5B parameters
coding
Parameters
32.5B
Context Length
128K
Architecture
qwen2
Release Date
2024

Available Quantizations

QuantizationFile SizeQuality
Q2_K12.0 GBLower
Q3_K_M15.3 GBLower
Q4_K_M18.8 GBGood
Q5_K_M21.5 GBGood

Can it run? (Q4_K_M quantization)

GeForce RTX 4090
Runs great
~36 tok/s • 24GB VRAM
GeForce RTX 4070
Runs slowly
~4 tok/s • 12GB VRAM
GeForce RTX 3060 12GB
Runs slowly
~3 tok/s • 12GB VRAM
Radeon RX 7900 XTX
Runs great
~34 tok/s • 24GB VRAM
Check with your specific GPU →