Skip to content
ML Academy

Statistics

Interview question

  • seniorlevel
  • 15 minsuggested time
  • 1Data scientist

A small test on a new checkout button gives 12 purchases from 200 control visitors and 20 purchases from 210 treatment visitors (hypothetical). The frequentist two-proportion test gives p = 0.18, and a PM asks "so what is the chance the new button is better?" A colleague proposes a Beta-Binomial analysis. Walk through what each approach says and what you would report.

Tests: Bayesian updating for proportions, interpreting a posterior probability vs a p-value, prior sensitivity, expected loss for decisions, and the limits of small samples.

day 3day 4

Sign in to practise: answers are saved before model answers are shown.