A100 40GB
Best local AI models for this GPU
VRAM
40 GB
Bandwidth
1555 GB/s
Vendor
NVIDIA
Compatible Models
28
Where to Buy
Top Compatible Models
Top 10 models ranked by performance. Click any model to expand details or view full compatibility.
Memory Usage36.8 GB
Weights
KV Cache
Overhead
Context Length163,840 tokens
Install with Ollama:
ollama run deepseek-r1:70b-q3_k_m
Memory Usage36.8 GB
Weights
KV Cache
Overhead
Context Length131,072 tokens
Install with Ollama:
ollama run llama3.3:70b-q3_k_m
Memory Usage37.8 GB
Weights
KV Cache
Overhead
Context Length131,072 tokens
Install with Ollama:
ollama run qwen2.5:72b-q3_k_m
Memory Usage36.8 GB
Weights
KV Cache
Overhead
Context Length131,072 tokens
Install with Ollama:
ollama run llama3.1:70b-q3_k_m
Memory Usage21.6 GB
Weights
KV Cache
Overhead
Context Length131,072 tokens
Install with Ollama:
ollama run qwen3:32b
Memory Usage23.0 GB
Weights
KV Cache
Overhead
Context Length16,384 tokens
Install with Ollama:
ollama run gemma3:27b
Memory Usage20.1 GB
Weights
KV Cache
Overhead
Context Length131,072 tokens
Install with Ollama:
ollama run mistral-small:24b-q5_k_m
Memory Usage21.6 GB
Weights
KV Cache
Overhead
Context Length131,072 tokens
Install with Ollama:
ollama run qwen2.5:32b
Memory Usage21.6 GB
Weights
KV Cache
Overhead
Context Length131,072 tokens
Install with Ollama:
ollama run qwen2.5-coder:32b
Memory Usage28.0 GB
Weights
KV Cache
Overhead
Context Length131,072 tokens
Install with Ollama:
ollama run qwen3:30b-a3b-q6_k