Skip to content
ML Academy

Business judgment

Interview question

  • stafflevel
  • 20 minsuggested time
  • 3Data scientist, ML engineer, Applied AI

Over a year, your team's recommendation model improved click-through rate by 15% in successive A/B tests, but revenue per visitor is flat and returns increased. Leadership asks what went wrong and how to set objectives differently. What do you say?

Tests: Goodhart's law in ML objectives, proxy validation, guardrail metrics, long-term effects, and designing an objective hierarchy for a team.

day 5day 8day 16

Sign in to practise: answers are saved before model answers are shown.