LLMs, GenAI, RAG, agents, evaluation · insight-0516

Replace "check your answer" prompts with an external verifier

Situation
To cut errors, a team adds a second prompt asking the model to review and fix its own fare calculation.
Action
Use external feedback instead, such as unit tests, a deterministic calculator, schema and business-rule checks or a lookup in the source document, and pass the concrete failure back to the model.
Reason
Experiments found that without external feedback, self-correction did not reliably improve reasoning and sometimes made it worse.
Boundary
Calculation, reasoning, code and extraction tasks with checkable outputs.

Symptom or risk

Accuracy does not improve and sometimes falls as correct answers are changed, while cost doubles.

Revisions for style or completeness against an explicit checklist can help; measure them on the eval set.

Verifiers take engineering time but double as evaluation tooling.

Judging self-correction in experiments that tell the model when it was wrong, information production never has.

Huang et al. found that without external feedback LLMs struggle to self-correct their reasoning, and performance sometimes degrades after self-correction.
Large Language Models Cannot Self-Correct Reasoning Yet