S18 · lifecycle simulation

A cancellation-risk model whose success rewrites its own labels

People, data and outputs in this scenario are simulated. Each stage is recorded when you commit; the reveal comes after.

4stages
50 minabout
0stages committed

The brief

In the course database cancellations are 7.5% of final bookings (7.0% on domestic and intl-short routes, 10.0% on intl-long). The synthetic airline builds a model that flags the highest-risk decile of bookings, those likely to cancel within 14 days, and sends them a rebooking voucher offer intended to keep them. About 10,000 bookings a month fall in the decile; in the pre-launch validation 30.4% of them cancelled.

Values are hypothetical: each retained booking is worth 85 EUR of margin and each offer costs 4.00 EUR including issuance and redemption. All results below are simulated.

Stage 1 of 4

The product manager proposes the label "cancelled within 14 days" and a plan to score every booking daily and send offers to the top decile. The data scientist notes that once offers go out, some bookings that would have cancelled will not.

What do you set up at intake?

Use "cancelled within 14 days" as the label as proposed, and retrain monthly on whatever outcomes were observed, since more recent labels keep the model current and need no extra process.Keep the label and randomly withhold the offer from 5% of flagged bookings, so untreated outcomes and the offer's effect stay measurable throughout the model's whole life.Run a one-off pilot of four weeks with a control group and then send offers to everyone flagged without any holdout, which gets the full benefit to every flagged booking.Change the label to "accepted the offer", which is observed for every treated booking and is the outcome the offer actually targets, so the model optimises what the business pays for.