# Unsloth

Unsloth is a team building LLM training and quantization tooling, best known in the local-LLM community for the **UD-…-XL** GGUF series and the underlying [Unsloth Dynamic 2.0](https://unsloth.ai/docs/basics/unsloth-dynamic-2.0-ggufs) scheme. Their builds are typically the default download for new open-weight model releases — `huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF`, `huggingface.co/unsloth/gemma-4-...`, etc.

## What Dynamic 2.0 does

Two ingredients beyond a standard GGUF build:

1. **A large imatrix calibration dataset.** 300K to 1.5M tokens of real chat data are pushed through the model before quantization, measuring per-layer sensitivity. Layers sensitive to quantization stay at 8 or 16 bits; the rest can drop to 4 bits or below without disproportionate quality loss.

2. **The XL suffix preserves attention at near-original precision.** Where standard K-quants compress attn and ffn together with simple K-block scaling, Unsloth's XL recipe quantizes the attention tensors much less aggressively (closer to Q8) and lets ffn carry the size budget.

Originally only the `XL` slot used the UD recipe; recent releases ship `UD` versions at every quant level.

## Result on Qwen 3.6 35B-A3B (KLD vs BF16)

From Vyacheslav's measurements in [[habr-local-llm-quantization-deep-dive]]:

- Classic Q4_K_M ≈ UD-Q3_K_XL — the dynamic Q3 quant is 4.4 GB smaller for roughly the same KLD.
- UD-Q4_K_XL beats Q4_K_M, 1.2 GB larger.
- UD-Q2_K_XL beats IQ2_M at the same byte budget, and is usable for serious work — successfully one-shotted a procedural Minecraft clone on a 4060 16 GB.

This is the KLD-vs-BF16 picture. A separate measurement in the earlier [[local-llm-16gb-vram-tests|Habr tests]] article reported UD-Q4_K_XL with *worse* perplexity-on-WikiText-2 than Q4_K_M on Qwen MoE (+9.7% vs +2.1% degradation). The two metrics disagree on the same comparison. KLD vs the BF16 baseline is the more recent and architecture-specific measurement; Unsloth themselves later removed MXFP4 layers from Qwen GGUFs after reports of computational anomalies, so some of their early Qwen builds were genuinely worse than the canonical Q4_K_M.

## The "three other dynamic quant publishers" landscape

Per [[habr-local-llm-quantization-deep-dive]], Unsloth is one of four notable dynamic-quant creators each with different recipes:

| Publisher | Notable choice | Strength |
| --- | --- | --- |
| **Unsloth** (UD-…-XL) | Attn close to original, dynamic ffn | Most coverage of newest models |
| **Ubergarm** (IQ4_KS) | IQK quants for ik_llama, dynamic | Highest quality at same size on ik_llama |
| **Bartowski** | Custom imatrix on standard names | Reliable across architectures; English-heavy imatrix |
| **mradermacher** (i1-) | imatrix tuned for multilingual | Better non-English support |

All four now use imatrix even for "static" K-quants. The standard recipe (ggml, ollama, LM Studio defaults) has been left behind.

## Cross-references

- [[llm-quantization]] — the survey page; UD-Q*-XL is one branch
- [[habr-local-llm-quantization-deep-dive]] — Vyacheslav's primer with the empirical KLD comparison
- [[local-llm-16gb-vram-tests]] — the earlier perplexity-based measurement (different metric, different conclusion)
- [[toolbox/llama-cpp]] — Unsloth GGUFs target llama.cpp's quant formats
- [[mixture-of-experts]] — the architecture where Dynamic 2.0's behavior is most variable