Classical ML, validation, metrics, interpretation, debugging · insight-0187

Encode known directional relationships with monotonic constraints in gradient boosting

Situation
Domain logic requires a direction (higher debt never lowers default risk, more days to departure never lowers remaining bookings), yet partial dependence wiggles against it.
Action
Set per-feature constraints (monotonic_cst in HistGradientBoosting, monotone_constraints in LightGBM or XGBoost), then compare held-out metrics and slice errors with the unconstrained model.
Reason
Constraints restrict splits so predictions move in one direction as the feature increases, holding other features fixed, which encodes knowledge and reduces noise-fitting.
Boundary
Credit, insurance pricing, demand and risk models with a strong, defensible prior direction.

Symptom or risk

Non-monotone artefacts produce indefensible individual decisions and fail model review, even with good aggregate metrics.

HistGradientBoosting does not support constraints for multiclass targets or categorical features.

Usually a small accuracy cost for robustness and explainability; a large cost signals the prior may be wrong or confounded.

Assuming marginal constraints guarantee monotone behaviour of interactions or of effects across feature combinations.

HistGradientBoosting monotonic_cst sets per-feature constraints that only constrain marginal effects; they are unsupported for multiclass and categorical features.
Ensembles: Gradient boosting, random forests, bagging, voting, stacking (scikit-learn User Guide 1.11)
LightGBM monotone_constraints and XGBoost monotone_constraints take 1, -1 or 0 per feature.
Parameters (LightGBM documentation)Monotonic Constraints (XGBoost tutorial)