Skip to content
ML Academy

DL, NLP and LLMs

Interview question

  • stafflevel
  • 25 minsuggested time
  • 3Applied AI, ML engineer, Data scientist

A customer-support routing step calls a large hosted LLM 30 million times a month. An engineer proposes distilling it into a small self-hosted model. Hypothetical figures: large model 0.004 per call, small model 0.0003 per call plus 8,000 per month infrastructure, 60,000 one-off build cost. Offline, the large model is 94% accurate and the distilled one 90%. Each routing error costs about 0.30 in rework. Do you do it, and how do you decide?

Tests: Unit economics of model substitution that includes error cost, break-even reasoning, cascades, and an evaluation and rollout plan suitable for a platform decision.

day 43day 54day 55day 58

Sign in to practise: answers are saved before model answers are shown.