(minor use case I had recently was I was trying to find old Japanese blogs for Tamagotchis, which I gather there were a ton of in the 90s but almost none survive today - imagine if I could get those instead of the 1,000,000 sites just trying to sell them to me)
Would be a very nice feature to have indeed, though the data is a bit too inaccessible to index as far as I can tell (even though I've not given it any serious effort, so maybe it is?)
The entire game is combining a bunch of weak indicators into a strong one.
Crawler results depend on domain authority. If page owner, or page contents page change the ranking may, or should change.
However original author also could change contents, and page ranking should not be changed. So this is not easy to determine what to do with domain of it becomes inactive, or changes contents dramatically.
Currently I use only 30 day window to keep track of domains. After that period inactive domain is thrown out of the window.
However valuable domains, even if dead, reside longer. My UI provides easy link to wayback machine. So even for dead links I can browse them.
I noticed also that some domains, even if expired do serve contents, even if author left it alone. Page contents is served, but with a text that it expired.
Not the first sudden and unwelcome discontinuity, either.
Google came close to thinking that I was dead, and turned out when I recently checked to be still looking for me under eu., years after the fact.
And with a broader view, this sort of stuff happens to the world, and there are enough people in the same boat that it is worth thinking of false positives when major upheavals occur. They can range from ISPs just up and deciding to close up shop with zero notice (which also happened to me) to international geopolitical upheavals. Who knows! If Brexit happened, it is conceivable that one day, the island of Niue might eventually prevail and then decide overnight that non-Niue citizens may not own a nu. domain. (-:
I wonder how many times Marginalia would have declared me dead, by now. (-:
55555•5h ago
AznHisoka•4h ago
marginalia_nu•4h ago