Engineering, production, governance, cost, communication · insight-1274
Mitigate first with a kill switch, then diagnose, when a model-driven action is hurting users
- Evidence type
- documented fact
- Difficulty
- medium
- Roles
- ML engineer
- Industries
- aviation, e-commerce, banking
Summary: situation, action, reason, boundary
Situation
A pricing model starts producing clearly wrong fares, and the team spends an hour in notebooks looking for the root cause while it keeps publishing.
Action
Build a switch that disables the model-driven action or routes to a fallback without redeploying, rehearse it, use it as soon as impact is confirmed, and preserve inputs, outputs and logs before the analysis starts.
Reason
Restoring acceptable behaviour comes first; root-cause analysis can continue once users stop being harmed.
Boundary
Automated decisions with external effects: prices, limits, notifications, routing.
Symptom or risk
Every extra minute of investigation adds harm, and a team that must redeploy to stop the action does so under time pressure.
Exceptions
If the failure is silent data corruption that a fallback would also propagate, freeze the writes instead of switching.
Trade-offs
Switching to a fallback costs revenue or accuracy, and the fallback must itself be maintained.
Pitfalls
A switch that nobody has authority to flip at night.
Lessons and concepts
Claims and their sources
The SRE book chapter on effective troubleshooting says that during an incident the first aim is to make the system work as well as it can, that stopping the harm comes before finding the root cause, and that evidence such as logs should still be preserved.