Classical ML, validation, metrics, interpretation, debugging · insight-0151
Treat SMOTE as an experiment with known caveats, not a default fix for imbalance
- Evidence type
- empirical finding
- Difficulty
- medium
- Roles
- Data scientist, ML engineer
- Industries
- healthcare, manufacturing
Summary: situation, action, reason, boundary
Situation
Someone proposes SMOTE because the positive class is 3% of rows.
Action
Run SMOTE only inside training folds, compare it with no resampling plus a tuned threshold and with class weights, and check calibration and the business metric, not just recall at 0.5.
Reason
SMOTE changes the training distribution, not the information content; gains reported at a default threshold often disappear after threshold tuning.
Boundary
Weak or distance-based learners on numeric features with genuinely scarce positives.
Symptom or risk
Synthetic points are interpolated between neighbours, which can create impossible rows (fractional one-hot values, invalid dates) and blur noisy minority regions into the majority.
Exceptions
Mixed-type data needs variants designed for categoricals, or no interpolation at all.
Trade-offs
Possible recall gains at fixed threshold versus distorted probabilities and more pipeline complexity.
Pitfalls
Oversampling before splitting, which puts synthetic neighbours of test points into training.
Lessons and concepts
Claims and their sources
SMOTE creates synthetic minority examples by interpolating between a minority sample and its nearest minority neighbours.
In risk prediction, SMOTE and random resampling led to overestimated risks without improving discrimination.