Classical ML, validation, metrics, interpretation, debugging · insight-1168
Read min_child_weight as a sum of Hessians, not a row count, when moving from squared error to logistic loss
- Evidence type
- documented fact
- Difficulty
- hard
- Roles
- Data scientist, ML engineer
- Industries
- banking, finance
Summary: situation, action, reason, boundary
Situation
A min_child_weight tuned for regression is reused for a binary:logistic model, or is expected to mean at least 20 rows.
Action
Translate the leaf size you accept into Hessian units: n rows at probability p give about n x p x (1 - p). Set min_child_weight from that and sweep it on a log scale; use a row-count parameter such as LightGBM's min_child_samples when a plain count is meant.
Reason
XGBoost's parameter is a minimum sum of instance weight (Hessian) in a child; with squared error that equals a row count, with logistic loss it does not.
Boundary
XGBoost and LightGBM's min_sum_hessian_in_leaf.
Symptom or risk
Under logistic loss each row adds at most 0.25 to the Hessian sum, and less when the model is confident, so the same value blocks far more splits.
Exceptions
Squared-error objectives with unit weights, where weight and count coincide.
Trade-offs
Hessian limits are harder to explain than a row count.
Pitfalls
Changing scale_pos_weight or sample weights afterwards, which rescales the Hessian sums.
Example
20 rows at p = 0.05 give 20 x 0.05 x 0.95 = 0.95, just under a limit of 1; the same 20 rows at p = 0.5 give 5.
Lessons and concepts
Claims and their sources
XGBoost's min_child_weight is the minimum sum of instance weight (Hessian) needed in a child; in linear regression this corresponds to a minimum number of instances per node.