Skip to content
ML Academy

Classical MLdraft — in review

Interview question

  • stafflevel
  • 25 minsuggested time
  • 2Data scientist, ML engineer

A lender's credit model is retrained yearly on loans it approved. Leadership now wants to raise the approval rate from 40% to 60% of applicants and asks you to certify that the new model is safe. Last year: 100,000 applicants, 40,000 approved, 2,000 of them defaulted. How do you evaluate and decide, given that outcomes exist only for approved applicants? (All figures hypothetical.)

Tests: Selective labels and selection bias in decision systems: why offline metrics on approved loans cannot validate a looser policy, and what evidence can be bought.

day 5day 2day 13day 18

Sign in to practise: answers are saved before model answers are shown.