SQL, data modeling, data quality, EDA, visualization · insight-1062
Attach an idempotency key to events emitted by retried producers and deduplicate on it downstream
- Evidence type
- documented fact
- Difficulty
- medium
- Roles
- Data analyst, Data scientist, ML engineer
- Industries
- e-commerce, finance, aviation
Summary: situation, action, reason, boundary
Situation
A booking service retries a payment event when a request times out, and some payments are recorded twice in the event table.
Action
Have the producer generate a unique key per logical operation and reuse it for every retry; store it on the event; enforce uniqueness or deduplicate on it in the pipeline. Set a retention window for keys and log key collisions with different payloads.
Reason
Only the producer knows which attempts are the same operation, so the key has to originate there.
Boundary
Payment, order and notification events, and API ingestion with at-least-once delivery.
Symptom or risk
Retries create identical business events with different ingestion timestamps, so counts and revenue are inflated and no column distinguishes the copy from a real second purchase.
Exceptions
Naturally idempotent operations such as setting a status to a fixed value.
Trade-offs
Key storage and lookups; keys expire, so very late retries can slip through.
Pitfalls
Building the key from fields such as customer email and amount, which also collapses two real purchases.
Lessons and concepts
Claims and their sources
Stripe's API lets a client send an idempotency key so a request can be retried without performing the operation twice, returns the saved result of the first request for repeated keys, and errors if parameters differ.