S16 · llm failure simulation

A claims summariser that never saw its last document

People, data and outputs in this scenario are simulated. Each stage is recorded when you commit; the reveal comes after.

4stages
35 minabout
0stages committed

The brief

An insurer (simulated) pilots an assistant that summarises a home-insurance claim file for a human adjuster. This exercise uses a fixture: fictional claims and documents, simulated model outputs and a hypothetical model with an 8,192-token context window. Nothing touches a real system.

Claim CLM-70418 has five documents, concatenated in this order and passed to the model through the client library:

Doc	Content	Tokens (real tokenizer)
D-201	Policy wording	1,880
D-202	Claim form	1,240
D-203	Adjuster notes	2,310
D-204	Correspondence	1,260
D-205	Exclusions annex	900

Budget: 8,192 window - 650 instructions - 800 reserved for output = 6,742 input tokens. The pipeline's pre-flight check estimates tokens as characters divided by four and passed the file with an estimate of 6,325 tokens.

Stage 1 of 4

Simulated output for CLM-70418:

{
  "claim_id": "CLM-70418",
  "summary": "Water damage from a burst pipe, reported 6 days after discovery, within the 30-day window. Repair estimate 7,450 EUR. No exclusions apply.",
  "citations": [{"doc_id": "D-201", "quote": "..."}, {"doc_id": "D-203", "quote": "..."}],
  "recommended_next_step": "standard_review"
}

The adjuster later finds that the exclusions annex (D-205) excludes damage from pipes older than 40 years unless inspected, and D-203 records a 52-year-old pipe with no inspection. The client log for the call reads: context_assembled tokens=6742 budget=6742.

Which statements about this failure are justified? Select all that apply.

The assembled context equals the budget exactly, and the heuristic estimate (6,325 tokens) undercounts the real 7,590 by about 17%: both point to silent truncation.The model hallucinated because it is too small; a larger model would have read the whole file and reported the exclusion on its own.The claim "no exclusions apply" is an absence claim the model could not support, because the exclusions annex was not in its context.Valid JSON that passes the schema shows that the content is supported by the input and that the summary can go to the adjuster as it stands.Nothing in the output or the log tells the adjuster that the input was incomplete, so this is a pipeline defect and not only a model error.