Skip to content
ML Academy

Business judgment

Interview question

  • stafflevel
  • 25 minsuggested time
  • 2Data scientist, ML engineer

Your team of five engineers (about 100 productive engineer-days per month) operates 14 production models, each costing around 4 engineer-days a month in monitoring, retraining and incident work, so 56 days, more than half of your capacity. Product keeps asking for new models. You must decide which models to keep, merge or retire. Four examples (hypothetical annual value; maintenance days per month): A, 420k, 4 days; B, 90k, 4 days; C, 35k, 4 days and it reads personal data under an old consent; D, 260k, 8 days. How do you decide, and how do you take the decision to stakeholders?

Tests: Portfolio and lifecycle thinking for ML systems, value per maintenance cost, risk and governance considerations, and handling stakeholder resistance to retiring features.

Sign in to practise: answers are saved before model answers are shown.