Classical ML, validation, metrics, interpretation, debugging · insight-1141
Set LightGBM min_data_in_leaf from how many rows you would trust to estimate a rate
- Evidence type
- documented fact
- Difficulty
- medium
- Roles
- Data scientist, ML engineer
- Industries
- cross-industry
Summary: situation, action, reason, boundary
Situation
A leaf-wise boosted model on a few thousand rows overfits and the usual response is to lower num_leaves again and again.
Action
Raise min_data_in_leaf to the smallest count at which a leaf's average is believable for your outcome rate, then tune num_leaves and the depth cap.
Reason
Leaf-wise growth can build deep, narrow branches for a fixed leaf count; a minimum leaf size limits that directly.
Boundary
LightGBM and similar leaf-wise learners on small and mid-sized data, particularly with rare positives.
Symptom or risk
Leaves that rest on three or four rows produce extreme scores for rare cohorts; training loss falls while validation loss rises.
Exceptions
On very large data hundreds of rows per leaf are usually reached anyway, so the parameter matters less.
Trade-offs
Too large a value limits depth and underfits genuine small cohorts.
Pitfalls
Setting num_leaves equal to 2 to the max_depth, which the documentation calls a poor conversion in practice.
Example
Hypothetical: with a 2% event rate, a 20-row leaf expects 0.4 positives, so its score is mostly noise; 500 rows expect about 10.
Lessons and concepts
Claims and their sources
LightGBM documentation calls min_data_in_leaf a very important parameter to prevent over-fitting in a leaf-wise tree, notes a leaf-wise tree is typically much deeper than a depth-wise tree for a fixed number of leaves, and advises num_leaves <= 2^max_depth.