This is a really cool small scale experiment. More of this type of work is need so we can have fairer and more transparent search options.
I'd be curious to see the sensitivity of the ratings to things like rater composition (is it a quirk of Redditors they like Bing better?) and search topics.
Also makes me wonder how much of the ratings rank to do with a decline in quality (or diversification/drift away from the Redditor vector) of Google's own search raters (the index is heavily influenced by manual rating).
UnmappedStack•1w ago
It'd be very interesting to run the same experiment except rather than on different engines, just on different versions of Google in different years. Sadly I can't really search 2005 Google anymore of course but it would be interesting nonetheless to see if it's Google in general or rather the recent "enshittification" of their results. It may be intriguing to expand to survey takers outside of Reddit to get a potentially less biased set of results, yes.
botacode•1w ago
Wonder if you could reconstruct synthetic versions of Gsearch with archive pages. I'd guess SEO/SEM companies have the data to build a small version of this and track the changes over time.
botacode•1w ago
I'd be curious to see the sensitivity of the ratings to things like rater composition (is it a quirk of Redditors they like Bing better?) and search topics.
Also makes me wonder how much of the ratings rank to do with a decline in quality (or diversification/drift away from the Redditor vector) of Google's own search raters (the index is heavily influenced by manual rating).
UnmappedStack•1w ago
botacode•1w ago