Gemma 2 9B

9B parameters
reasoningcoding
Parameters
9B
Context Length
8K
Architecture
gemma2
Release Date
2024

Available Quantizations

QuantizationFile SizeQuality
Q2_K3.6 GBLower
Q3_K_M4.5 GBLower
Q4_K_M5.5 GBGood
Q5_K_M6.3 GBGood
Q8_09.5 GBHigh

Can it run? (Q4_K_M quantization)

GeForce RTX 4090
Runs great
~129 tok/s • 24GB VRAM
GeForce RTX 4070
Runs slowly
~13 tok/s • 12GB VRAM
GeForce RTX 3060 12GB
Runs slowly
~9 tok/s • 12GB VRAM
Radeon RX 7900 XTX
Runs great
~123 tok/s • 24GB VRAM
Check with your specific GPU →