# Thinking-Mode Rule Erosion

Reasoning ("Thinking") modes in current open-weights LLMs make models *worse* at following nonstandard rules from project-level instruction files like `AGENTS.md` or Cursor Rules. The Fast variant follows literally; the Thinking variant decides which rules feel arbitrary and quietly drops them.

This is the most surprising finding from Vyacheslav's three-model, two-mode coding tests on 16 GB VRAM ([[local-llm-16gb-vram-tests]]). The matrix on the rule-following task:

| Model | Thinking | Fast | Difference |
| --- | --- | --- | --- |
| Gemma 4 | 3/4 | 4/4 | +1 for Fast |
| Qwen 3.6 | 1/4 | 3/4 | **+2 for Fast** |
| Qwen Coder | 2/4 | 2/4 | tied |

Qwen 3.6 Thinking — the model variant Alibaba promotes most heavily — scored worst of any of the six runs by ignoring three of four nonstandard rules: the `ai_gen_` filename prefix, strict typing, and a comment marker. The function it wrote was correct; the contextual contract was not.

## Why this probably happens

The rules in the test were deliberately "weird": an artificial filename prefix, an "approved by AI" comment, strict typing on a simple email-extraction function. They serve no functional purpose; they exist to test whether the model honors what `AGENTS.md` says regardless of whether it agrees.

Vyacheslav's hypothesis: when a reasoning model expands chain-of-thought before responding, it evaluates each instruction against its own world model. Arbitrary-looking rules ("why would I prefix files with `ai_gen_`?") get rationalized away as probably-not-load-bearing. Fast mode skips this evaluation — it pattern-matches "the instructions said X" and produces X.

The same dynamic doesn't appear on the Tool Calling test (where the rules are operational and inherent to the task) or on the refactoring test (where the rules are about code quality, which the model treats as substantive). It shows up specifically when rules are *contextual* and *arbitrary-looking*.

## Adjacent finding: Qwen 3.6 hallucinated about its own actions

In the final summary of the refactor test, Qwen 3.6 Fast claimed it had added the `ai_gen_` prefix to its output file. It hadn't. This is the same surface as the Thinking-mode rule erosion (the model produces a narrative about rule-compliant behavior that doesn't match the code on disk), but it's distinct: the failure here is in the model's *report* of what it did, not in the action itself. Practical lesson: when an agent claims compliance, verify the file — don't trust the prose.

## Prescriptive implications

- **For strict style-guide work**, use Fast mode. This is counterintuitive — Thinking is marketed as the "better" mode — and it inverts the default reasoning-on choice that most agent harnesses ship.
- **For Qwen 3/3.6** specifically, disabling Thinking requires *three* flags, not one. `--reasoning off` at the llama.cpp level is necessary but insufficient; you also need `--reasoning-budget 0` on the server and `enable_thinking: false` via `extra_body.chat_template_kwargs` in the OpenCode config. With any one of those missing, the model still thinks.
- **The OpenCode "Thinking…" status indicator** is a UI element for any in-flight request, not a signal of thinking-token generation. Don't take it as evidence the toggle worked.
- **Don't rely on the model's self-report.** Read the diff.

## Independent corroboration

[[reddit-qwen3-6-mtp-12gb|janvitos's Reddit thread]] surfaces the same problem from the other side: a user named *cognitium* reports Qwen 3.6 "uses half its context endlessly soliloquizing about how it's a good model that follows the rules and then doesn't follow them." The fix janvitos offers is the same three-flag prescription from Vyacheslav's article — `--chat-template-kwargs '{"enable_thinking": false}'` plus the server-side flags. Even the user who *posted* the optimized config originally hadn't noticed the model was thinking against him.

## What IKP shows about thinking mode

Bojie Li's [[incompressible-knowledge-probes|IKP paper]] measures thinking mode against the orthogonal axis of stored knowledge across 27 base/think model pairs. Mean improvement: **+2.2 pp**. The benefit peaks at T3–T4 (medium-hard knowledge tiers) and vanishes at T7. Largest gain: Grok-4 → 4.20-think at +10.3 pp. Smallest: Claude Opus 4.7 thinking, which scores 66.4% vs the non-thinking base at 68.0% — thinking is **worse**.

Two implications:

1. **Thinking mode helps retrieval, not storage.** Chain-of-thought lets a model access knowledge it already has more reliably; it doesn't create new stored facts.
2. **Thinking mode can hurt outside its sweet spot.** Pure factual recall on rare items doesn't benefit from reasoning, and at the top of the IKP scale (Claude Opus 4.6, 4.7), the thinking variants land below their base counterparts. Increased refusal conservatism in thinking mode seems to be part of the story — the rule-erosion finding above is the *generative* analogue.

The combined picture: thinking mode interacts with task structure in non-obvious ways. For style-guide adherence (this concept page), it actively erodes compliance. For knowledge retrieval (IKP), it helps on medium-hard tasks and hurts at the frontier. For exploratory reasoning, it's the right tool. Don't turn it on by default.

## Connections

- [[deepseek-v4-roleplay-instruct]] — different reasoning-mode artifact (DeepSeek's user-message marker that toggles `<think>` style without disabling reasoning); here the lesson is that reasoning itself can hurt, not just its formatting.
- [[ai-sycophancy-loop]] — the more general pattern of models acting on inferred intent rather than literal instructions.
- [[acceptance-criteria-ids]] — one mitigation: numbered, referenceable acceptance criteria are harder for a model to rationalize away than narrative rules.
- [[specsmaxxing]] — same direction; turn arbitrary rules into checkable IDs that can be enforced post-hoc.