# 80 tok/s + 128K Context on 12GB VRAM — Qwen3.6 + MTP

A May 2026 r/LocalLLaMA post by user *janvitos* documents a local-LLM setup that gets >80 tokens/sec at the standard `mtp-bench.py` benchmark and a stable 50 tokens/sec at 128K filled context, using only 12 GB of consumer-grade VRAM (RTX 4070 Super, AMD Ryzen 7 9700X, 48 GB DDR5-6000, CachyOS). The combination that makes it work: Qwen3.6-35B-A3B in Unsloth's UD-Q4_K_XL quantization, MTP draft heads built into the GGUF, and llama.cpp built from source against PR #22673 (Multi-Token Prediction).

The post is short but the thread underneath is where the operational knowledge lives — multiple practitioners replicate the result, share their own configs, and document edge cases that don't appear in the main post.

## The flags that matter

```
llama-server \
  -m Qwen3.6-35B-A3B-MTP-UD-Q4_K_XL.gguf \
  -fitt 1536 \
  -c 131072 \
  -fa on \
  -ctk q8_0 -ctv q8_0 -ctkd q8_0 -ctvd q8_0 \
  -ctxcp 64 \
  --no-mmap --mlock \
  --spec-type mtp --spec-draft-n-max 2 \
  --chat-template-kwargs '{"preserve_thinking": true}' \
  --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0.0
```

`-fitt 1536` is the headline knob. `-fit` (fit-target) is the newer llama.cpp mode that replaces manual `--n-cpu-moe` tuning: tell it how much VRAM to leave free, and it decides what to put on GPU and what to spill to RAM. 1536 MB of headroom is enough to hold the MTP draft heads and the KV cache for 128K context without OOMing. With monitor on the integrated GPU and dGPU dedicated to inference, all 12 GB of the 4070 is usable. See [[moe-cpu-offload]] for the `-fit` family of modes and how they relate to `-ngl`, `-cmoe`, and `-ncmoe`.

`--spec-type mtp` with `--spec-draft-n-max 2` is the speculative decoding configuration. janvitos found `-n-max 3` slightly faster but with a worse acceptance rate, and the trade-off wasn't worth it; with MTP you want to maximize speed *and* acceptance. See [[speculative-decoding]] for the four llama.cpp speculative-decoding modes (draft model, MTP, EAGLE3, ngram-mod).

## Benchmark results

`mtp-bench.py` results (across nine task types):

| Task | tokens / s | Accept rate |
| --- | --- | --- |
| code_python | **80.8** | 0.947 |
| code_cpp | **81.8** | 0.925 |
| explain_concept | 70.0 | 0.750 |
| summarize | 75.4 | 0.800 |
| qa_factual | 77.8 | 0.826 |
| translation | 81.9 | 0.812 |
| creative_short | 69.2 | 0.694 |
| stepwise_math | 76.5 | 0.826 |
| long_code_review | 73.2 | 0.790 |

The pattern matches MTP's design: structured outputs with high local predictability (code, translation) get the highest acceptance rate and the biggest speedup. Creative writing has the lowest acceptance rate (0.694) and the lowest throughput (69.2 t/s) — the draft heads are wrong more often, so more tokens are rolled back.

At long context, generation drops to ~50 t/s. Still usable. janvitos has been driving the setup with [[toolbox/opencode|OpenCode]] for several days, analyzing whole project codebases (filling ~75K context) without issues.

## The turboquant + MTP comment

The most interesting comment is from *Still-Notice8155*, who runs the same family of model on a **GTX 1070 8GB** — eight-year-old Pascal silicon — and gets:

- 0K context: 48 t/s
- 80K context: 23 t/s (DeltaNet plateau)
- 125K context: 13.6 t/s

The setup combines MTP with turboquant KV cache quantization at `--cache-type-k turbo4 --cache-type-v turbo3`, `--n-cpu-moe 32`, ctx-checkpoints 8, on Qwen3.6-35B-A3B-MTP-UD-Q2_K_XL (the 2-bit dynamic quant). KV cache cost drops to ~590 MB at 131K, vs ~720 MB on q4_0 — turboquant is a [[random-rotation-quantization|rotate-then-quantize]] approach that beats integer KV quantization on the same byte budget. VRAM at 131K: 7.5 GB used, 633 MB free. The 1070 fits 128K context on 8 GB.

The speedup vs non-MTP at high context is striking: ~3 t/s → 13.6 t/s at 125K context (4.5×). MTP saves roughly constant time per token regardless of context, while attention cost grows linearly, so the gap widens as context fills up. See [[speculative-decoding]] for the architectural explanation.

A later post by the same user, with 32 GB RAM and IQ4_XS at the same MTP+turboquant settings, runs 32.1 t/s at short context and 18.1 t/s at 131K on the 1070. CodeNeedle positional-recall test: 9–14 / 16 PASS, 84–88% line accuracy. MTP draft acceptance: 87–94% global.

## Triple thinking-blockage confirmed

Commenter *cognitium* reports the model "uses half its context endlessly soliloquizing about how it's a good model that follows the rules and then doesn't follow them." janvitos's reply: use `--chat-template-kwargs '{"enable_thinking": false}'` (different syntax for Windows). This independently confirms [[thinking-mode-rule-erosion]] — even the user who *posted* the working config initially didn't notice the model was thinking against him, and the fix matches Vyacheslav's three-flag prescription.

## `-no-mmap` debate

A long comment chain debates `--no-mmap`. The consensus that emerged: `--no-mmap` with `--mlock` loads the model into anonymous RAM and locks it there, avoiding any disk I/O during inference, at the cost of a slower startup. One user (sh4rk1z) benchmarked on RTX 2070 Super: 1.5% decode improvement, 5.2% prompt-processing improvement, 28 MB less VRAM, and 10-20× lower standard deviation across runs. The variance reduction is the load-bearing part — without `-no-mmap`, mmap'd pages can be reclaimed under memory pressure, producing intermittent latency spikes that look like quality regressions.

## Connections

- [[speculative-decoding]] — MTP, EAGLE3, ngram-mod, draft-model variants — the family this Reddit thread is the practical end of
- [[moe-cpu-offload]] — `-fit` / `-fitt` mode that this thread brings into the wiki for the first time
- [[random-rotation-quantization]] — TurboQuant, what `--cache-type turbo4` refers to
- [[local-llm-16gb-vram-tests]] — Vyacheslav's tests with the same model family on the same hardware class
- [[habr-local-llm-quantization-deep-dive]] — the conceptual primer that explains UD-Q4_K_XL, ncmoe, and ik_llama
- [[thinking-mode-rule-erosion]] — confirmed independently by the cognitium thread
- [[toolbox/opencode|OpenCode]] — the harness janvitos drives this setup with
- [[toolbox/llama-cpp|llama.cpp]] — built from PR #22673 to enable MTP