Practical insights
Situation, action, reason and boundary — one decision per card. Filter by topic, role and difficulty; search covers the full text.
- 1000published
- 25in review
- 8topic families
Results
- Use a pre-period covariate (CUPED-style adjustment) to shrink variance before buying more traffic
An experiment on revenue per user needs more weeks than the roadmap allows, and the metric has a strong user-level history.
hardempirical finding - Match the variance formula to the randomization unit when metrics are computed per page or per session
Users are randomized, but the metric is click-through per page view or conversion per search, and the analysis treats each page as an independent observation.
harddocumented fact - Leave a washout gap between switchback periods when a pricing or ranking change lingers
A fare-ranking algorithm is global, so the team alternates it on and off by hour and compares the two states.
harddocumented fact - Expect the best of many arms to disappoint on rollout and re-estimate the winner in a fresh sample
A pricing-page test runs 10 variants, and the top one shows +3.1% and is shipped on that number.
harddocumented fact