How We Calculate Compatibility
Our compatibility engine uses real-world measurements and established formulas to predict whether your hardware can run local AI models.
Memory Calculation
Total memory requirement = Model weights + KV cache + Overhead
- Model weights: File size from actual GGUF quantizations (from HuggingFace bartowski/unsloth repos)
- KV cache: Calculated based on context length, number of KV heads, and head dimensions
- Overhead: 500MB for runtime and processing buffers
Speed Estimation
Speed (tokens/sec) is primarily bandwidth-limited for inference:
tokens/sec ≈ (bandwidth_gbps × 1024) / (params_active_billions × bits_per_weight)For MoE models, we use active parameters (not total) for speed calculation.
Ranking Algorithm
Models are ranked by capability score:
score = effective_params × quant_quality × generation_factor × speed_bonus- effective_params: For dense models = total params; for MoE = 0.6 × total params
- quant_quality: Q8_0 (1.0), Q6_K (0.98), Q5_K_M (0.96), Q4_K_M (0.93), Q3_K_M (0.80)
- generation_factor: 1.0 (2025+ models), 0.75 (2024 models), 0.5 (older models)
- speed_bonus: min(1.25, 1 + (tok_s - 10)/200) - favors faster models
Quantization Quality
Different quantization methods offer different trade-offs:
| Quantization | Bits per Weight | Quality |
|---|---|---|
| Q8_0 | 8.5 | Near-original (best) |
| Q6_K | 6.6 | Excellent |
| Q5_K_M | 5.7 | Very good |
| Q4_K_M | 4.85 | Good (recommended) |
| Q3_K_M | 3.9 | Acceptable (30B+ only) |
Validation & Sources
- Model specs: Official model cards and HuggingFace repositories
- GGUF file sizes: Actual measurements from bartowski and unsloth quantizations
- GPU specs: Official vendor specifications and TechPowerUp database
- Performance validation: Cross-referenced with llama.cpp benchmarks and community reports