Unsloth

Unsloth is a team building LLM training and quantization tooling, best known in the local-LLM community for the UD-…-XL GGUF series and the underlying Unsloth Dynamic 2.0 scheme. Their builds are typically the default download for new open-weight model releases — huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF, huggingface.co/unsloth/gemma-4-..., etc.

What Dynamic 2.0 does

Two ingredients beyond a standard GGUF build:

1. A large imatrix calibration dataset. 300K to 1.5M tokens of real chat data are pushed through the model before quantization, measuring per-layer sensitivity. Layers sensitive to quantization stay at 8 or 16 bits; the rest can drop to 4 bits or below without disproportionate quality loss.

2. The XL suffix preserves attention at near-original precision. Where standard K-quants compress attn and ffn together with simple K-block scaling, Unsloth's XL recipe quantizes the attention tensors much less aggressively (closer to Q8) and lets ffn carry the size budget.

Originally only the XL slot used the UD recipe; recent releases ship UD versions at every quant level.

Result on Qwen 3.6 35B-A3B (KLD vs BF16)

From Vyacheslav's measurements in habr-local-llm-quantization-deep-dive:

• Classic Q4_K_M ≈ UD-Q3_K_XL — the dynamic Q3 quant is 4.4 GB smaller for roughly the same KLD.
• UD-Q4_K_XL beats Q4_K_M, 1.2 GB larger.
• UD-Q2_K_XL beats IQ2_M at the same byte budget, and is usable for serious work — successfully one-shotted a procedural Minecraft clone on a 4060 16 GB.

This is the KLD-vs-BF16 picture. A separate measurement in the earlier Habr tests article reported UD-Q4_K_XL with worse perplexity-on-WikiText-2 than Q4_K_M on Qwen MoE (+9.7% vs +2.1% degradation). The two metrics disagree on the same comparison. KLD vs the BF16 baseline is the more recent and architecture-specific measurement; Unsloth themselves later removed MXFP4 layers from Qwen GGUFs after reports of computational anomalies, so some of their early Qwen builds were genuinely worse than the canonical Q4_K_M.

The "three other dynamic quant publishers" landscape

Per habr-local-llm-quantization-deep-dive, Unsloth is one of four notable dynamic-quant creators each with different recipes:

All four now use imatrix even for "static" K-quants. The standard recipe (ggml, ollama, LM Studio defaults) has been left behind.

Cross-references

• llm-quantization — the survey page; UD-Q*-XL is one branch
• habr-local-llm-quantization-deep-dive — Vyacheslav's primer with the empirical KLD comparison
• local-llm-16gb-vram-tests — the earlier perplexity-based measurement (different metric, different conclusion)
• llama-cpp — Unsloth GGUFs target llama.cpp's quant formats
• mixture-of-experts — the architecture where Dynamic 2.0's behavior is most variable