Classical ML, validation, metrics, interpretation, debugging · insight-1183
Score anomalies against a peer group, not against the whole population
- Evidence type
- practitioner heuristic
- Difficulty
- medium
- Roles
- Data scientist, ML engineer
- Industries
- aviation, banking, manufacturing
Summary: situation, action, reason, boundary
Situation
One detector (isolation forest, local outlier factor, z-score) is fitted on flights, machines or accounts that sit in very different regimes.
Action
Fit separate detectors per peer group with enough rows, or normalize scores by group quantile, and report the alert rate by group. Aim for a similar review burden per group, and define peers from attributes known before the outcome.
Reason
Unusual means unusual relative to comparable units; neighbourhood-based methods such as local outlier factor are explicitly relative to the surrounding density.
Boundary
Fraud, maintenance, billing and operational monitoring with heterogeneous units.
Symptom or risk
The global model flags the rare regime itself and misses oddities inside the dominant one; alerts pile up in one segment and reviewers learn to ignore them.
Exceptions
Groups with too few rows; use the global model with the group as a feature instead.
Trade-offs
More models to maintain and small groups give unstable thresholds.
Pitfalls
Defining peer groups from the outcome or from the anomaly itself.
Lessons and concepts
Claims and their sources
Local outlier factor measures the local density deviation of a point with respect to its neighbours, so it can handle data in which abnormal samples have different underlying densities.