SQL, data modeling, data quality, EDA, visualization · insight-1068
Plot the rows before trusting a summary table, because identical statistics can hide different shapes
- Evidence type
- practitioner heuristic
- Difficulty
- easy
- Roles
- Data analyst, Data scientist
- Industries
- cross-industry
Summary: situation, action, reason, boundary
Situation
A segment comparison table shows near-identical mean, standard deviation and correlation for two route groups, and the summary is about to be filed as "no difference".
Action
Before concluding, draw the raw points or distributions for each group (scatter, strip or small multiples) and note outliers, clusters and non-linearity. Keep the plot next to the table in the analysis.
Reason
Graphs reveal structure that the chosen statistics were not designed to capture.
Boundary
Group comparisons, correlations, regression summaries and model diagnostics.
Symptom or risk
Summary statistics compress a dataset to a few numbers, and very different patterns, such as clusters, a single leverage point or a curve, can give the same values.
Exceptions
Millions of points, where binned or density displays replace raw scatter.
Trade-offs
Plots need judgement; nothing automates the reading.
Pitfalls
Looking only at a fitted line, which is itself a summary.
Lessons and concepts
Claims and their sources
Anscombe's 1973 paper argued for graphs in statistical analysis using small datasets that share usual summary statistics but look very different.
Matejka and Fitzmaurice generate datasets with the same statistical properties but very different appearance when graphed, using small random changes guided by simulated annealing.