Statistics, experimentation, causal reasoning · insight-1003
Randomise fare tests by departure or route when treated and control passengers draw on the same seats
- Evidence type
- documented fact
- Difficulty
- hard
- Roles
- Data scientist
- Industries
- aviation, retail, e-commerce
Summary: situation, action, reason, boundary
Situation
A fare experiment splits sessions between a discounted and the standard fare on the same departures.
Action
Randomise at the level where inventory is shared (departure or route-week) and compare those units. With few units, pair similar routes or alternate in time. Describe the user-level difference as an upper bound for a rollout effect, not the launch estimate.
Reason
Individual randomisation assumes one customer's treatment does not change another's outcome; a shared capacity pool breaks that assumption.
Boundary
Price, ancillary and upsell tests on constrained inventory such as seats, rooms and slots.
Symptom or risk
Discounted buyers take seats control would have bought, so the user-level lift overstates what a full rollout delivers, or even has the wrong sign.
Exceptions
Departures that never approach capacity during the test window.
Trade-offs
Far fewer randomisation units, so longer tests or larger detectable effects.
Pitfalls
Checking the average load factor instead of load at the booking-curve moment when demand peaks.
Example
Hypothetical: 10 seats left, half of visitors see a 20% discount; discount buyers take the seats and control sells fewer, so the per-user comparison counts the same seats twice.
Lessons and concepts
Claims and their sources
In price A/B tests where market participants interfere with each other, the standard estimator of the change in profit can have the wrong sign in expectation under a standard revenue-management pricing model.