ik_llama.cpp
- title
- ik_llama.cpp
- type
- toolbox
- summary
- ikawrakow's llama.cpp fork with IQK quants, Q6 KV cache and faster long-context prompt processing
- tags
- llm, inference, local-models, quantization, cpp
- language
- C/C++
- license
- MIT
- created
- 2026-05-13
- updated
- 2026-05-13
ik_llama.cpp is ikawrakow's fork of llama.cpp. Ikawrakow created the K-quants and I-quants that underlie modern GGUF quality (Q4_K_M, IQ4_XS, etc.), worked in mainline llama.cpp for years, then split off after a dispute with ggerganov and now develops his own line of SOTA quants and CPU-side optimizations in this fork.
It is not a drop-in replacement. ikawrakow works alone on the fork; optimizations don't flow between the two projects, and each has features the other doesn't. Picking between them is a workload-by-workload question.
Where ik_llama wins
- IQK quants. New quantization formats not available in mainline llama.cpp.
IQ4_KSon Qwen 3.5 (Vyacheslav's measurement) shows ~3× better KLD thanQ4_K_Mat the same disk size, and matchesUD-Q4_K_XLquality at ~1 GB less. The size win compounds on large models — 20–30 GB matters when fitting GLM-5.1 or DeepSeek V3.2 into 192 GB system RAM. - Q6 KV cache quantization. Mainline llama.cpp ships
q4_0,q8_0,f16for KV cache; ik_llama addsQ6. Useful whenq4_0loses too much andq8_0doesn't leave enough headroom. - Long-context prompt processing. PP throughput holds up over context far better than mainline. Vyacheslav measures 4× higher PP at default batch sizes; mainline needs
-ub 4096 -b 4096to match, costing an extra ~3 GB VRAM. - AVX-512 / Intel CPU paths. Aggressively tuned. Worth comparing on Intel hardware specifically.
Where ik_llama loses
- MXFP4 quants. Partial support, 2-3× slower than mainline. Don't use ik_llama for Gemma 4's MXFP4 builds.
- Vulkan and AMD GPUs. Run but at a fraction of mainline's speed. ROCm support is similar.
- TG (token generation) speed. Even on its own native IQK quants, TG can be slower than the mainline equivalent at the same byte budget. Architecture-specific.
- Build trails mainline. New model architectures land first in mainline; ik_llama catches up.
Practical recommendation
| Situation | Pick |
|---|---|
| RTX 50-series, MXFP4 Gemma 4, agentic coding | mainline llama.cpp |
| Intel CPU heavy, AMD GPU, MXFP4 | mainline |
| Long-context PP-bound workloads on Nvidia | ik_llama |
| Maximum quality at minimum disk for large MoE | ik_llama with Ubergarm IQK GGUFs |
| Just starting out | mainline; revisit ik_llama only when you hit a specific need it solves |
Vyacheslav's testbed (i7-14700, DDR5-4800, Manjaro, Nvidia) shows ik_llama winning on PP at long context but losing on TG for mxfp4. The relative numbers depend on your hardware — don't take ik_llama as automatic upgrade.
Related projects
- Ubergarm (huggingface.co/ubergarm) — publishes IQK GGUFs specifically for ik_llama. Uses dynamic quantization with calibration.
- Thireus's Windows builds (github.com/Thireus/ik_llama.cpp) — pre-built Windows binaries.
Related wiki
- llama-cpp — the upstream / sibling project
- llm-quantization — survey of quants including IQK
- habr-local-llm-quantization-deep-dive — the trade-offs catalog this page summarizes
- unsloth — the alternative dynamic-quant publisher for mainline llama.cpp
Repo: https://github.com/ikawrakow/ik_llama.cpp · C/C++ · MIT