S16 · llm failure simulation

Unsupported citations and broken JSON in a claims assistant

People, data and outputs in this scenario are simulated. Each stage is recorded when you commit; the reveal comes after.

4stages
35 minabout
0stages committed

The brief

An insurer (simulated) is piloting an assistant that drafts a structured summary for each home-insurance claim. Human adjusters make every decision. This exercise uses a fixture: fictional claims, fictional documents and simulated model outputs. Nothing touches a real system.

Required output schema (JSON):

{
  "claim_id": "string, pattern CLM-[0-9]{5}",
  "summary": "string, at most 120 words",
  "citations": [{"doc_id": "string", "quote": "string copied from that document"}],
  "recommended_next_step": "one of: request_info | escalate_adjuster | standard_review"
}

Context supplied for claim CLM-55821: D-101 policy wording (water damage clause: reported within 30 days), D-102 adjuster intake note, D-103 contractor photo report.

Stage 1 of 4

Simulated output 1:

{
  "claim_id": "CLM-55821",
  "summary": "Water damage reported 19 days after discovery. Policy excludes water damage reported after 14 days.",
  "citations": [{"doc_id": "D-104", "quote": "water damage is excluded if reported after 14 days"}],
  "recommended_next_step": "deny"
}

What is wrong with this output? Select all that apply.

It cites `D-104`, which was not in the supplied context documents.The quoted text appears in no supplied document and contradicts `D-101` (30 days).`deny` is not an allowed value, and recommending denial is outside the assistant's scope.The root cause is that sampling temperature was too high, which made the model invent details.The summary is too short to be useful to a reviewer who needs the full policy text.