#quantization
Wiki 9
- arkaung.github.io arkaung's GitHub Pages site with interactive math tutorials; Interactive TurboQuant is the first ingested piece
- Evaluating Quantized Models for Deployment ByteShape on why perplexity, KLD, and BPW don't rank quantized models for deployment
- Interactive TurboQuant — Rotate, Then Quantize arkaung's visual explainer of the DRIVE → EDEN → TurboQuant lineage; one rotation turns outliers into a known distribution, one fixed codebook serves every input
- LLM Quantization Formats Q4_K_M vs UD-Q4_K_XL vs MXFP4 — three approaches to 4-bit weight compression, with the perplexity gap that newer ≠ better
- Local LLMs Deep Dive — Quants, MoE Offload, REAP, ik_llama, Speculative Decoding Vyacheslav's long-form Habr primer covering BF16, K/I-quants, dynamic quantization, `-fit`/`-cmoe`/`-ncmoe`, REAP expert pruning, ik_llama, MTP/EAGLE3, Linux vs Windows
- Neutrino-1 8B Fermion Research's Apache-2.0 8B: a 3.88 GB ternary-coded container for H100, MacBook, and CPU
- Random-Rotation Quantization Rotate vectors by a random orthogonal transform, then quantize each coordinate against a fixed codebook — outliers dissolve, no per-block metadata, one codebook fits all inputs
- REAP — Router-weighted Expert Activation Pruning Cut the least-active experts from an MoE model based on a target-domain calibration dataset; 20% size reduction with most agentic-coding capability retained
- Unsloth Quantization team behind the UD-Q*-XL GGUF family and Unsloth Dynamic 2.0 scheme; most-cited dynamic-quant publisher for Qwen/Gemma/DeepSeek
Toolbox 2
- CrofAI Cheap hosted inference for open-weights models, unusual for publishing each model's quantization
- ik_llama.cpp ikawrakow's llama.cpp fork with IQK quants, Q6 KV cache and faster long-context prompt processing