Skip to content
ML Academy

Production and system design

Interview question

  • midlevel
  • 10 minsuggested time
  • 2ML engineer, Data scientist

Before launch, you replay 10,000 recent bookings through the new online no-show scoring service and through the offline batch pipeline used for training. For 7% of bookings the two predicted probabilities differ by more than 0.05. Same model file in both. How do you investigate, and what do you put in place so this cannot slip into production?

Tests: Training-serving skew: online/offline parity testing, localising feature-level differences, common skew causes, and preventive engineering (shared feature definitions, served-feature logging, parity monitors).

day 2day 9day 19

Sign in to practise: answers are saved before model answers are shown.