# ik_llama.cpp

`ik_llama.cpp` is **ikawrakow's** fork of llama.cpp. Ikawrakow created the K-quants and I-quants that underlie modern GGUF quality (`Q4_K_M`, `IQ4_XS`, etc.), worked in mainline llama.cpp for years, then split off after a dispute with ggerganov and now develops his own line of SOTA quants and CPU-side optimizations in this fork.

It is *not* a drop-in replacement. ikawrakow works alone on the fork; optimizations don't flow between the two projects, and each has features the other doesn't. Picking between them is a workload-by-workload question.

## Where ik_llama wins

- **IQK quants.** New quantization formats not available in mainline llama.cpp. `IQ4_KS` on Qwen 3.5 ([[habr-local-llm-quantization-deep-dive|Vyacheslav's measurement]]) shows ~3× better KLD than `Q4_K_M` at the same disk size, and matches `UD-Q4_K_XL` quality at ~1 GB less. The size win compounds on large models — 20–30 GB matters when fitting GLM-5.1 or DeepSeek V3.2 into 192 GB system RAM.
- **Q6 KV cache quantization.** Mainline llama.cpp ships `q4_0`, `q8_0`, `f16` for KV cache; ik_llama adds `Q6`. Useful when `q4_0` loses too much and `q8_0` doesn't leave enough headroom.
- **Long-context prompt processing.** PP throughput holds up over context far better than mainline. Vyacheslav measures 4× higher PP at default batch sizes; mainline needs `-ub 4096 -b 4096` to match, costing an extra ~3 GB VRAM.
- **AVX-512 / Intel CPU paths.** Aggressively tuned. Worth comparing on Intel hardware specifically.

## Where ik_llama loses

- **MXFP4 quants.** Partial support, 2-3× slower than mainline. Don't use ik_llama for Gemma 4's MXFP4 builds.
- **Vulkan and AMD GPUs.** Run but at a fraction of mainline's speed. ROCm support is similar.
- **TG (token generation) speed.** Even on its own native IQK quants, TG can be slower than the mainline equivalent at the same byte budget. Architecture-specific.
- **Build trails mainline.** New model architectures land first in mainline; ik_llama catches up.

## Practical recommendation

| Situation | Pick |
| --- | --- |
| RTX 50-series, MXFP4 Gemma 4, agentic coding | **mainline llama.cpp** |
| Intel CPU heavy, AMD GPU, MXFP4 | mainline |
| Long-context PP-bound workloads on Nvidia | ik_llama |
| Maximum quality at minimum disk for large MoE | ik_llama with Ubergarm IQK GGUFs |
| Just starting out | mainline; revisit ik_llama only when you hit a specific need it solves |

Vyacheslav's testbed (i7-14700, DDR5-4800, Manjaro, Nvidia) shows ik_llama winning on PP at long context but losing on TG for mxfp4. The relative numbers depend on your hardware — don't take ik_llama as automatic upgrade.

## Related projects

- **Ubergarm** ([huggingface.co/ubergarm](https://huggingface.co/ubergarm)) — publishes IQK GGUFs specifically for ik_llama. Uses dynamic quantization with calibration.
- **Thireus's Windows builds** ([github.com/Thireus/ik_llama.cpp](https://github.com/Thireus/ik_llama.cpp)) — pre-built Windows binaries.

## Related wiki

- [[toolbox/llama-cpp]] — the upstream / sibling project
- [[llm-quantization]] — survey of quants including IQK
- [[habr-local-llm-quantization-deep-dive]] — the trade-offs catalog this page summarizes
- [[unsloth]] — the alternative dynamic-quant publisher for mainline llama.cpp

Repo: https://github.com/ikawrakow/ik_llama.cpp · C/C++ · MIT