LLMs, GenAI, RAG, agents, evaluation · insight-0533
Let the model reason before you force strict formats on reasoning-heavy tasks
- Evidence type
- empirical finding
- Difficulty
- medium
- Roles
- Applied AI, ML engineer
- Industries
- aviation
Summary: situation, action, reason, boundary
Situation
A multi-step fare-difference calculation is moved into a strict JSON output mode, and accuracy on the eval set drops.
Action
Compare on the eval set three designs, free-text reasoning then a separate formatting call, a schema with a reasoning field placed before the answer, and the strict answer-only schema; keep the most accurate design that still validates.
Reason
A study found that, in its experiments, strict format restrictions lowered performance on reasoning tasks, with stricter constraints causing larger drops; generation is left to right, so the answer cannot use reasoning written after it.
Boundary
Calculations, multi-step classification, decisions with justifications.
Symptom or risk
The team keeps the strict format because parsing is easier, and ships a less accurate calculator.
Exceptions
Simple extraction loses little under strict formats; some providers restrict reasoning fields in schemas, so check documentation.
Trade-offs
Two-step designs add a call; reasoning fields add tokens.
Pitfalls
Placing the reasoning field after the decision field.
Lessons and concepts
Claims and their sources
Tam et al. found that restricting outputs to structured formats such as JSON significantly lowered reasoning performance, with stricter constraints generally causing larger drops.