LLMs, GenAI, RAG, agents, evaluation · insight-1510
Ask a schema for a short explanation, not for step-by-step reasoning
- Evidence type
- documented fact
- Difficulty
- medium
- Roles
- Applied AI, ML engineer
- Industries
- cross-industry
Summary: situation, action, reason, boundary
Situation
An extraction schema includes a step_by_step_reasoning property so reviewers can audit decisions, and a few percent of requests come back as refusals.
Action
Replace the free reasoning property with a bounded explanation or evidence field of one or two sentences; if hidden reasoning is needed, use the provider's thinking feature; log refusal stop reasons apart from schema failures.
Reason
Anthropic documents a refusal category for requests that ask the model to reproduce its internal reasoning in the response text, and says a refusal takes precedence over schema constraints and is billed.
Boundary
Anthropic structured outputs; check whether other providers document the same behaviour.
Symptom or risk
Responses stop with a refusal that does not match the schema, are still billed, and are retried unchanged by a generic repair loop.
Exceptions
Providers and model versions without this behaviour, which can differ and change.
Trade-offs
A short explanation holds less detail, and it is a post-hoc rationale, not a record of the computation.
Pitfalls
Auditing decisions from an explanation field as though it were the model's actual reasoning.
Lessons and concepts
Claims and their sources
Anthropic documents that a schema property asking for the model's step-by-step reasoning may trigger a refusal in the reasoning_extraction category, that a refusal response can fail the schema and is billed, and recommends asking for a short explanation instead (as read on 2026-10-12).