S31 · stakeholder simulation

Executive brief after an agent paid refunds twice

People, data and outputs in this scenario are simulated. Each stage is recorded when you commit; the reveal comes after.

4stages
35 minabout
0stages committed

The brief

Last night between 01:00 and 07:00 a customer-service agent (a language model with tools, working on refund requests) issued duplicate refunds. The chief operating officer (simulated) asks for a brief before the 09:00 leadership meeting: "Was the AI hallucinating? Do we switch it off?"

Facts from the incident log and the payment ledger (simulated):

4,800 refund requests were processed in the window.
The refund API timed out on 432 calls (9.0%). On a timeout the agent retried the call once, as its tool wrapper is configured to do.
Of the 432 timed-out calls, 268 (62.0%) had in fact already been processed by the payments system. The retry created a second refund. The other 164 had not been processed, so the retry was correct.
The refund call carries no idempotency key. The average refund is 86.40 USD (the average of the duplicates is assumed to be the same). 190 duplicates have been reversed so far.
No refund above the policy limit was issued, and the model's written replies to customers were accurate.

Stage 1 of 4

The first two sentences are what the officer will remember in the meeting.

Which opening do you choose for the brief?

The AI agent made a serious error overnight and started paying customers twice. The model's behaviour is under investigation and finance has been notified so that affected customers can be identified.The language model hallucinated refund instructions overnight, producing 432 incorrect refunds, all of which have since been corrected through the ledger by the finance team.Between 01:00 and 07:00 the agent issued 268 duplicate refunds, about 23,200 USD, after payments-API timeouts triggered unguarded retries; 190 are reversed and 78 (about 6,700 USD) remain open.A small fraction of refunds, 5.6% of the window's volume, were duplicated owing to a transient payments issue, and the matter is largely closed.