Skip to content
Prompt Corpus
Technique20 August 20263 min read

Asking a model to check its own work usually does not help

The instruction "now review your answer and improve it" feels like it must work. The research says it often makes correct answers worse - and explains exactly when it does help.

It is such an obvious move. You get an answer, you ask the model to review it, it comes back with a revision that reads better.

Huang et al. (2024) looked at whether the revision is actually better, and the finding is uncomfortable: models attempting intrinsic self-correction — revising using only their own judgement, with no external feedback and no ground truth — often fail to improve, and frequently make correct answers worse.

Worse, several earlier positive results turned out to depend on an oracle. The loop was told whether the answer was wrong, which is exactly the information you do not have at inference time. Remove the oracle and much of the benefit goes with it.

Why it fails

Two things stack up.

The critique consults the same absent knowledge. If the model did not know the founding date, asking it to check the founding date queries the same gap. There is nothing to correct against.

The draft is in the context. Asking "is this right?" while the answer sits right there means the answer is now evidence. Combine that with the agreeableness that RLHF trained in, and "review your work" reliably produces a polite endorsement.

When it does work

Self-refine and chain-of-verification are real techniques with real results. The condition that separates them from the failing case is specific:

Self-correction works when the check has an external referent. It fails when the check needs the knowledge the model was missing.

Checks that work, because something outside the model can settle them:

  • Is this under 150 words?
  • Does this JSON match the schema?
  • Does every quoted passage appear in the source document?
  • Did the tests pass?
  • Is there a concrete next action with an owner?

Checks that do not:

  • Is this factually correct?
  • Is this good?
  • Are you sure?

Two fixes that matter

Make the critique a checklist, not an assessment. "Improve this" invites agreeable churn. Four numbered criteria with PASS or FAIL and the offending line quoted produces something you can act on. Then constrain the rewrite to the failures, or it churns the parts that were fine.

Verify in a fresh context. This is what chain-of-verification gets right. Generate specific verification questions, then answer them without the draft present. Answering "was it founded in 1987?" cold is a different computation from answering it while looking at your own sentence asserting 1987.

The connection worth noticing

Constitutional AI uses this exact loop — generate, critique against a principle, revise — during training, and it works. The reason is the same reason the good version works here: the critique is against an explicit written referent, not against undirected self-assessment.

Which is a decent summary of the whole thing. Self-critique with nothing to check against is the case that does not reliably work, and the fix is not better wording. It is giving it something real to check against — a criterion, a source, a schema, a test suite.

Full lesson: self-refine and chain-of-verification.

All notes