Presumably but I wish they'd also focus on optimizing code/making pages fully cacheable instead of rate-limits and blocking
swatcoder•9m ago
Making requests is inherently cheaper than delivering responses, even with caches. Efficiency improvements can buy a little time on a given resources but won't solve the problem of bot saturation now that everybody can spawn a custom bot in about 12 seconds and is being encouraged to do so.
Rate limits, blocking, and pay-per-use are the only roads out and even those might not last as models get better at hacking and masquerading.
The internet we want to use LLM's with is simply not one that can support LLM's, and with LLM's not going anywhere, the whole experience of the internet is going to be forced into some radically less open and more expensive paradigm.
Policies like this just represent the beginning of the transition.
Frieren•16m ago
We need a new non-commercial version of the internet. Free of bots, free of ads, ... you pay to access social media optimized to be interesting enough to be worth paying instead of addictive enough to keep you scrolling to show you more ads.
It may look impossible right now. But what is impossible for real is to continue as we are. The damage that internet does to society is increasing by the day while its value is reduced (economic value, social value).
phoe-krk•9m ago
> We need a new non-commercial version of the internet.
You'd need a force strong enough to prevent it from falling prey to this tragedy of the commons, and that force would need to be stronger than the incentives to commercialize it. And that's where plenty of contemporary scraping-based salaries lay.
immortalist•9m ago
Impossible to create
OtherShrezzing•3m ago
I'm not so certain. I used to buy a broadsheet newspaper, which was full of ads. Now I pay a few hundred a year, and get the same newspaper online, with no ads at all.
So, the precursor to online media has already gone through this paradigm shift.
Zambyte•2m ago
It's not, it just requires creativity. One example I can think of is limiting connectivity by distance between nodes. Something like Meshtastic seems unlikely to ever be commercialized in the same way that the Internet has been. Sure, you would lose some useful applications of the Internet, but you would gain other things.
ddtaylor•16m ago
> A request that arrives with no credentials gets 60 requests per hour per IP address.
One request per minute.
mplanchard•13m ago
Average, yes, but the way they phrase it, it could be a token bucket or similar, where you can do 60 quick requests and then be blocked for a bit while the bucket refills
sparkling•14m ago
I noticed that recently Github.com has some kind of weird bot detection on public repos. I have a browser extension for switching User Agents for a specific legacy site, sometimes i forget to turn it off and Github will require me to login to view public repos.
All of this is most likely due to mass scraping by LLMs. Welcome to the total shitification of the web.
296012•14m ago
Congrats on making the world worse with AI. All this performative data scraping and uploading and no progress at all.
speedgoose•5m ago
No progress?!
jtwaleson•14m ago
I think it's because people are building agentic flows, reducing the amount of developer seats needed. It's the first step towards usage based pricing.
heavensteeth•5m ago
That's this internet. Nothing precludes you from dropping big tech and solely engaging with the "indie web"[0][1] if you so choose. You're free to build your own website with its own RSS feed, join webrings[2] of like-minded people, and engage organically.
[0] One of many similar initiatives, I'm sure. Not an endorsement.
Careful what you wish for. The ticket to entry will (have to) be a proof of identity, the ultimate nail in the coffin of privacy and anonymity on the web.Outside of that, chaos.
tempest_•19m ago
nijave•17m ago
swatcoder•9m ago
Rate limits, blocking, and pay-per-use are the only roads out and even those might not last as models get better at hacking and masquerading.
The internet we want to use LLM's with is simply not one that can support LLM's, and with LLM's not going anywhere, the whole experience of the internet is going to be forced into some radically less open and more expensive paradigm.
Policies like this just represent the beginning of the transition.
Frieren•16m ago
It may look impossible right now. But what is impossible for real is to continue as we are. The damage that internet does to society is increasing by the day while its value is reduced (economic value, social value).
phoe-krk•9m ago
Except it's non-commercial, therefore valuable, therefore commercialized, therefore commercial.
You'd need a force strong enough to prevent it from falling prey to this tragedy of the commons, and that force would need to be stronger than the incentives to commercialize it. And that's where plenty of contemporary scraping-based salaries lay.
immortalist•9m ago
OtherShrezzing•3m ago
So, the precursor to online media has already gone through this paradigm shift.
Zambyte•2m ago