Classical ML, validation, metrics, interpretation, debugging · insight-0190
Do not use the silhouette score to choose between centroid and density-based clustering
- Evidence type
- documented fact
- Difficulty
- medium
- Roles
- Data scientist
- Industries
- logistics, cross-industry
Summary: situation, action, reason, boundary
Situation
You compare k-means and DBSCAN results using the silhouette coefficient and pick the higher score.
Action
Use silhouette to compare settings within one algorithm family, judge across families by stability and downstream usefulness, and tune DBSCAN's eps with a nearest-neighbour distance plot.
Reason
Silhouette compares within-cluster and nearest-cluster distances, which matches the geometry k-means assumes; DBSCAN's results hinge on eps and density, not convexity.
Boundary
Cluster model selection on unlabeled data.
Symptom or risk
The metric favours compact convex clusters, so it can prefer k-means even when the real structure is elongated or density-defined.
Exceptions
If clusters are known to be roughly spherical, silhouette comparisons across methods are less misleading.
Trade-offs
Internal metrics are cheap; external validation with a downstream outcome is costlier but closer to the decision.
Pitfalls
Leaving DBSCAN's eps at its default; too small marks most points as noise, too large merges clusters.
Lessons and concepts
Claims and their sources
k-means assumes convex, isotropic clusters; the silhouette coefficient is generally higher for convex clusters than for density-based clusters such as those from DBSCAN; DBSCAN's eps is crucial, with too small values labelling most data as noise and too large values merging clusters.
The silhouette compares each point's average within-cluster dissimilarity with its dissimilarity to the nearest other cluster.