Classical ML, validation, metrics, interpretation, debugging · insight-0169

Know that HistGradientBoosting switches early stopping on automatically for larger datasets

Situation
A HistGradientBoosting model behaves differently on a full dataset than on a sample, or training sizes vary across experiments.
Action
Set early_stopping, validation_fraction and n_iter_no_change explicitly, log n_iter_ after fitting, and keep settings identical across compared runs.
Reason
With the default early_stopping='auto', early stopping is enabled when the training set has more than 10,000 samples.
Boundary
scikit-learn HistGradientBoostingClassifier and HistGradientBoostingRegressor.

Symptom or risk

Above a size threshold the estimator silently holds out a validation fraction and stops early; results change for reasons nobody configured.

If you rely on outer cross-validation for stopping, disable internal early stopping to avoid two competing mechanisms.

Internal stopping saves compute but removes a random fraction of training rows, which matters on time-ordered data.

The internal validation split is random by default, so on temporal data it can leak future rows into the stopping decision.

HistGradientBoosting enables early stopping by default when there are more than 10,000 samples, using validation_fraction of the training data.
Ensembles: Gradient boosting, random forests, bagging, voting, stacking (scikit-learn User Guide 1.11)