Feature engineering and preprocessing · insight-1102
Build target encodings in time order on time-stamped data, because random cross fitting borrows from the future
- Evidence type
- practitioner heuristic
- Difficulty
- hard
- Roles
- Data scientist, ML engineer
- Industries
- aviation, banking
Summary: situation, action, reason, boundary
Situation
A table of time-stamped outcomes, such as no-shows by departure date, is target encoded with a standard k-fold cross fit and then judged on a later period.
Action
Compute each row's encoding only from rows before its timestamp, with a gap for label delay, using an expanding window you write yourself; keep one full-history encoding for serving.
Reason
A prediction at time t may only use what was known at t. Random folds put later labels into earlier rows, and scikit-learn's cross fitting requires every row to be validated exactly once, so it cannot take a forward-chaining splitter.
Boundary
Mean-of-outcome and frequency encodings on churn, no-show, delay, fraud and demand data with drift or trend.
Symptom or risk
Each training row's encoding includes labels from rows that happen after it, so the model learns a relationship production can never reproduce, and the live encoding is built from older history.
Exceptions
Genuinely stationary categories evaluated with a deliberately shuffled split.
Trade-offs
Early rows get thin history and fall back to the global mean; custom code replaces a one-line transformer.
Pitfalls
Counting outcomes that were not yet known at row time, such as bookings whose flight had not departed.
Lessons and concepts
Claims and their sources
TargetEncoder.fit_transform cross fits with KFold or StratifiedKFold splits, shuffled by default, and a custom splitter must place every sample index in a validation fold exactly once.