EnglishРусский Map
↑Cross-model code review

On the Self-Verification Limitations of LLMs on Reasoning and Planning Tasks

Wiki summaryllmllm-evaluationverification ↳ show in map Markdown
title
On the Self-Verification Limitations of LLMs on Reasoning and Planning Tasks
type
summary
summary
Stechly, Valmeekam and Kambhampati find GPT-4 self-critique loops lose accuracy while a sound external verifier gains it, even with no critique text
parent
cross-model-code-review
tags
llm, llm-evaluation, verification
sources
arxiv-2402-08115, arxiv-2402-08115-full
created
2026-09-13
updated
2026-09-13

An Arizona State University preprint by Kaya Stechly, Karthik Valmeekam and Subbarao Kambhampati (first posted February 2024, revised August 2024) tests a belief that was common at the time: that an LLM can improve its own answers by critiquing them in a loop. The authors trace that belief to the complexity-theory intuition that checking a solution is easier than finding one, and argue the intuition has no reason to hold for a model whose behaviour is closer to approximate retrieval than to search.

Setup

The model is GPT-4. The domains were chosen because correctness is formally checkable, new instances can be generated, and the solution space cannot be shrunk by pattern matching: Game of 24, graph coloring on 100 generated planar graphs, and STRIPS planning in Blocksworld and its obfuscated twin Mystery Blocksworld. Each domain gets 100 instances.

The self-critique system (LLM+LLM) sends a generation prompt, wraps the answer in a verification prompt sent back to the same model, and if the model rejects its answer, appends the critique and the full history of earlier attempts to the next prompt. The loop runs up to 15 rounds. Two ablations sit beside it. In LLM+sound verifier, an external checker judges each answer (SymPy for Game of 24, a single for-loop over edges for coloring, VAL for plans) at three levels of feedback: pass/fail only, the first error, or every error. In the sampling condition the critique disappears entirely and the same base prompt is re-sent until the verifier accepts an answer.

What happened

Self-critique made things worse. In most domains the loop ended below the model's own first guess, and more rounds degraded output further. The failure was mainly false negatives: the model rejected correct answers often enough that the system moved on to later, worse guesses and timed out. Blocksworld was the exception, with a modest gain that still fell well short of the sound verifier.

The critiques themselves were unreliable. In graph coloring they named edges that do not exist and misstated vertex colors. In planning they hallucinated whether an action's preconditions held. In Game of 24, among expressions guaranteed to equal 24, GPT-4 marked 79.1% as correct while evaluating 81.6% of them to 24, so there were answers it computed correctly and still rejected. The appendix tables for Blocksworld show the gap between noticing and explaining: on plans with an inexecutable step the binary verdict was right 70 times in 100 but the critique was right 8 times, and on random plans every one was flagged while only 2 critiques were correct.

With a sound verifier, accuracy rose substantially in every domain. The amount of feedback barely mattered, and in two domains richer feedback slightly lowered performance. Re-sending the unchanged prompt until the verifier accepted kept most of the gain, and because the prompt carries no history its token cost is, in the authors' words, quadratically lower. A self-consistency baseline (majority answer out of 15) showed no improvement over a single prompt. The authors' reading is that the LLM is useful as an idea generator and that the gains credited to self-critique in earlier work mostly come from having many guesses and a sound check.

Two appendix results bound the conclusion. Asking for chain-of-thought verification raised Game of 24 verification accuracy from 87% to 99%, yet the full loop stayed 6 points behind the sound verifier, and output tokens went up 17 times. And on the Tree of Thoughts test set, 150 plain queries with a sound verifier reached 70% against ToT's reported 74%.

The HTML conversion in the vault drops the main accuracy tables (Tables 1 and 2), so the per-condition accuracies and false-positive and false-negative rates are not recoverable from these files. The numbers above are the ones stated in prose or in the tables that survived.

Limits

This is GPT-4 in 2023–2024 on puzzles and planning, with no code. More important for how the paper gets cited: it compares a model critiquing itself against a provably sound verifier. It never puts a second, different LLM in the verifier seat, so it says nothing directly about whether another model would have done better than self-critique. It shows that an unsound verifier poisons the loop and that a sound one fixes it.

Cross-references

Linked from