https://blog.revolutionanalytics.com/2017/05/the-datasaurus-...
p(data | stats) = p(stats | data) * p(data) / p(stats).
and p(data) is only strong for a "blob / cloud" of points, so when there's some correlation the observed stats tell you that you likely have a blob having some degree of correlation.
We just spent five years since COVID appeared to argue about statistics, with tons of bad analysis of very complicated data fuelling political rage up to this day.
The US health secretary is currently using data with "strong structure" to deny vaccines and to falsely pin down convenient targets for everything from cancer to autism.
* https://en.wikipedia.org/wiki/G._E._M._Anscombe
:)
But sometimes you are at the mercy of the data and your visualization of choice. Box plots, for example, are great at showing more than just how the data is centered, but it is possible to encounter situations where the box plots of the data remain static while the underlying data is clearly changing [0].
As always it is good to know about these things and continue to add to the arsenal (violin plots, in the example above) of tools and intuition needed to tease out the story behind the data.
0: https://www.research.autodesk.com/publications/same-stats-di...
https://www.linkedin.com/posts/panela_loved-adding-ancombes-...
I recommend putting together the Quintet in one image, so that the original 4 charts, plus the new one are all visible and interpretable together. It will be learning aid for decades to come.
Linear correlation is just one pattern the data can have.
Unfortunately many social science publications have reviewers who know only the basics and can't judge or accept statistically valid analysis that is outside their competence. Fit it into line or nothing.
This will require improvements to vision models, RL frameworks, etc, but will be interesting to see how much it can broaden current abilities.
And was just thinking about it the other day. I had a bug aggregating sleep-data from an iPhone, which comes in the form of sleep-samples.
I was trying to fix it, both by prodding Claude Code to fix the problem, and looking at debug logs of the sleep-samples, but we weren't getting anywhere. I asked Claude Code to graph the samples, and BAM, saw it right away. (the problem was that HealthKit returns you sleep-samples from ALL devices, not just the priority one)
Maybe not exactly the same thing as Anscombe/Tufte were getting at, but I was reminded of it, and the value of visualising data.
djoldman•6h ago
See also:
https://en.wikipedia.org/wiki/Datasaurus_dozen
djoldman•6h ago
usually there are more than 2 or 3 columns in our data :(
imurray•4h ago
lamename•50m ago
sunrunner•5h ago