Classical ML, validation, metrics, interpretation, debugging · insight-1178
Choose the number of clusters from the action you can take and a stability check, not from inertia
- Evidence type
- documented fact
- Difficulty
- medium
- Roles
- Data scientist, Data analyst
- Industries
- retail, aviation, telecom
Summary: situation, action, reason, boundary
Situation
A segmentation run reports inertia for k from 2 to 15 and the team names segments at the apparent elbow.
Action
List the k values the business can operationalize (campaigns, route groups, pricing tiers). For each, run k-means++ with several seeds, compare silhouette and agreement between seeds, and take the smallest k whose clusters are stable and distinguishable on a held-out profile.
Reason
Inertia reaches zero at k equal to the number of rows and only says lower is better; k-means can stop in local minima that depend on initialization.
Boundary
Customer, route and asset segmentation with k-means or similar centroid methods.
Symptom or risk
Inertia falls as k grows by construction and is not normalized; the elbow is often invisible, so the chosen k reflects the plot rather than any decision.
Exceptions
Purely exploratory looks, where several k can be shown side by side.
Trade-offs
Operational constraints can force a k that is statistically second best.
Pitfalls
Comparing inertia across different feature sets or scalings.
Lessons and concepts
Claims and their sources
scikit-learn states that k-means inertia is not normalized (lower is better, zero is optimal), that the algorithm can converge to a local minimum depending on initialization, and that k-means++ initialization is generally better than random.