Skip to content
ML Academy

Classical ML

Interview question

  • midlevel
  • 10 minsuggested time
  • 2Data scientist, ML engineer

A teammate adds a target-encoded travel_agency_id (about 4,000 agencies, many with fewer than ten bookings) to a no-show model. Validation ROC-AUC jumps from 0.71 to 0.86. Two weeks after release, production AUC is 0.70. What most likely happened, how would you confirm it, and how should the feature be built?

Tests: Diagnosing target-encoding leakage from its symptoms, confirming it experimentally, and implementing out-of-fold, smoothed, time-aware encodings with handling of unseen categories.

day 9day 15

Sign in to practise: answers are saved before model answers are shown.