Skip to content
ML Academy

Production and system design

Interview question

  • stafflevel
  • 25 minsuggested time
  • 2ML engineer, Data scientist

A churn model scores 20,000 customers a week. Churn is defined as no activity for 60 days, so the true outcome for a scored customer is known only two months later. The business wants to know within days whether the model is still working. How do you design monitoring and the decision to retrain under this label delay?

Tests: Monitoring when ground truth is delayed: proxy and leading indicators, matured-cohort evaluation, statistical resolution of small windows, and decision rules for retraining.

day 19day 52day 58

Sign in to practise: answers are saved before model answers are shown.