Statistics, experimentation, causal reasoning · insight-0020

Ask what removed units from the data before analysing the survivors

Situation
An analysis of "routes we still operate" or "customers active today" is used to judge which strategies work.
Action
Rebuild the population as it existed at the start of the period, including units that later closed, churned or were cancelled. Compare outcomes from that starting cohort, and state the exit rule explicitly when only survivors are available.
Reason
Selection on the outcome changes the distribution being studied; the starting cohort restores the comparison the decision needs.
Boundary
Route profitability, customer lifetime value, fund performance, product line reviews and model training sets built from current entities.

Symptom or risk

Units that failed were dropped, so the surviving set looks healthier and any trait common among survivors looks like a success factor.

Questions explicitly about survivors, such as "how do active customers use feature X", are valid on survivors if labelled.

Reconstructing history needs snapshot or SCD data and more effort than querying current tables.

Joining to a current dimension table that silently drops closed routes or deleted accounts.

Hypothetical: routes launched in 2022 and dropped in 2023 vanish from a "current routes" join, inflating average launch performance.