LLMs, GenAI, RAG, agents, evaluation · insight-1508
Order schema fields so evidence and rationale are written before the verdict field
- Evidence type
- practitioner heuristic
- Difficulty
- medium
- Roles
- Applied AI, ML engineer
- Industries
- aviation
Summary: situation, action, reason, boundary
Situation
A refund-eligibility extractor returns JSON whose first key is eligible, followed by reason and clauses_cited.
Action
Put the quoted clauses and a short rationale before the decision field, confirm in an integration test that your provider preserves property order, and compare verdict accuracy for both orders on the development set.
Reason
Generation runs left to right, so an earlier field conditions later fields and not the reverse; key order is therefore part of the prompt.
Boundary
Decision or classification outputs with justification fields, on providers that follow schema order.
Symptom or risk
The model commits to the verdict before it has written the supporting clauses, and the reason field then rationalises the verdict instead of informing it.
Exceptions
Decoders that reorder keys, and verdicts that are trivially determined by the input.
Trade-offs
The verdict is no longer the first key for human readers, and extra fields cost tokens.
Pitfalls
Assuming a rationale field raises accuracy without measuring it; it can also lengthen outputs and add parse failures.