Practical insights
- 1000published
- 25in review
- 8topic families
Results
- Write a data contract for every upstream table a model depends on
A model breaks because an upstream team renamed a status code and nobody told the data science team.
easypractitioner heuristic - Monitor distributions to catch semantic changes that schemas cannot see
A price column switches from cents to currency units in a release; schema, types and null rates are unchanged.
easypractitioner heuristic - Deduplicate late-arriving corrections by business key and load time, explicitly
A flight-day feed sometimes sends the same row twice, the second with an updated revenue figure.
mediumpractitioner heuristic - Minimise personal data in features and give training snapshots a retention period
A churn model's training set copies full customer records, including fields the model never uses, and snapshots are kept indefinitely.
mediumdocumented fact - Write a datasheet for any dataset other teams will train on
A curated training table is published internally, and three teams start building models on it.
easypractitioner heuristic - Track unit cost per decision, not just the monthly cloud bill
Monthly ML infrastructure cost rose 40%, and nobody can say whether it is growth, waste or a new model.
easypractitioner heuristic - Declare model incidents early and give one person stakeholder communication
A pricing model has produced obviously wrong fares for an hour, and engineers are debugging while managers message them for updates.
easydocumented fact - Run blameless postmortems with triggers defined before incidents happen
After a model rollback, the team discusses who merged the bad change and moves on without writing anything.
easydocumented fact - Negotiate an ambiguous AI request down to one decision, one metric and a baseline
A director asks the team to 'use AI to optimise the network' within a quarter.
easypractitioner heuristic - Open an executive memo with the decision, the evidence range and the main limit
A four-page analysis of an overbooking model starts with data sources and methods, and the recommendation is on page four.
easyopinion - Report a null result with its interval and detectable effect, not as 'no effect'
A test of a new recommendation model shows no statistically significant revenue change, and the summary says 'the model does not work'.
mediumdocumented fact - Present forecasts to decision makers as ranges tied to the actions they support
Leadership receives a single number for next season's demand and builds the capacity plan on it.
easypractitioner heuristic - Pair any model-driven KPI target with guardrail metrics to resist Goodhart effects
A call-centre routing model is rewarded on average handle time, and handle time falls while repeat calls rise.
mediumpractitioner heuristic - Classify data-quality checks into blocking and warning tiers before they fire
A new validation suite fails the nightly pipeline on a minor null-rate rise, and the business misses its morning report.
easypractitioner heuristic - Estimate the value of a model with a back-of-envelope calculation before building it
A team proposes a three-month project to improve a no-show prediction model.
easypractitioner heuristic - Limit and explain forecast churn between model versions
A new forecasting model version changes next month's plan by 15% for some routes, though inputs barely changed.
mediumpractitioner heuristic - Keep a decision and assumption log so a model survives handover
A modelling project is handed to a new team, and the reasons for its target definition, exclusions and thresholds exist only in old chat threads.
easypractitioner heuristic - Count rows that are unique on indirect identifiers before releasing a table
An extract with names and ids removed, but birth date, postcode and gender intact, is to be shared with a partner for research (hypothetical).
mediumdocumented fact - Treat hashed customer ids as pseudonymous personal data and keep the key apart
Emails in a training table are replaced by plain hashes, and the table is called anonymous and sent to a vendor (hypothetical).
mediumdocumented fact - Look for bunching just past a KPI target to catch gaming
A branch bonus needs 95 percent of tickets resolved within the service target, and many branches report exactly 95 to 96 percent (hypothetical).
hardpractitioner heuristic - Version KPI definitions like code and restate history when they change
Finance changes 'active customer' from 30 to 90 days of activity in a dashboard, and the trend shows a jump in March (hypothetical).
easypractitioner heuristic - List who bears each error type before setting the threshold
A claims-flagging threshold is set from investigator hours alone, while flagged customers wait weeks for payment (hypothetical).
mediumpractitioner heuristic - Replace probability words with numeric ranges in written risk statements
A risk memo calls a forecast regression 'likely' and a launch delay 'possible', and three readers take away different odds (hypothetical).
easypractitioner heuristic - Use quantile dotplots, not only interval bars, when non-specialists decide under uncertainty
A gate-staffing planner receives delay forecasts as 80 percent interval bands and acts on the midpoint anyway (hypothetical).
mediumempirical finding - Quote an automation benefit as residual human workload, not as the share automated
A business case says a triage model will 'automate 80 percent' of 10,000 monthly cases (hypothetical).
easypractitioner heuristic - Walk stakeholders through twenty concrete predictions before debating the metric
A review meeting keeps stalling on whether an F1 of 0.71 is good (hypothetical).
easypractitioner heuristic - Gate project ranking on decision owner and label readiness before scoring value
A backlog of fourteen ideas is ranked by sponsor seniority and novelty (hypothetical).
mediumpractitioner heuristic - Keep a small permanent randomised control share to measure what the model earns
Six months after launch, finance asks what the retention model earned, and the only evidence is before-and-after revenue (hypothetical).
mediumpractitioner heuristic - Rehearse a handover by letting the new owner run a full cycle while the builder watches
The builder of a monthly forecasting pipeline is moving teams, and a handover document has been written (hypothetical).
easypractitioner heuristic