Skip to content
ML Academy

Business judgment

Interview question

  • seniorlevel
  • 15 minsuggested time
  • 2Data scientist, ML engineer

Leadership sets the goal "reduce route-level forecast MAPE by 20% this year" and ties bonuses to it. After six months the average MAPE across 200 routes (unweighted) dropped from 21.3% to 16.6%. A colleague tells you the team re-tuned only the 190 small routes. How do you evaluate the claim, and what would you propose instead? (Hypothetical figures.)

Tests: Goodhart-style metric gaming, aggregation choices (weighted vs unweighted), aligning a metric with decisions, and how to give leadership a target that cannot be gamed cheaply.

day 1day 16

Sign in to practise: answers are saved before model answers are shown.