As per the wiki page for the C10k problem[1] where it says "_was_ [emphasis mine] the problem of optimizing computer networking stacks to handle a large number of clients at the same time". This was solved over a decade ago...
Now get off my lawn!
(TBF I think the author acknowledges this)
Who is?!
You see something, you say something, or you do something.
At first, I assumed this was a single blog post, especially with a "Table of Contents" at the top. I guess it's following some old-school blogging conventions, but AFAICR, even those sites retained individual pages for separate posts.
- AWS
- Google Cloud
- Azure
- Alibaba Cloud
- Oracle
and a handful of other ASNs on almost all of my properties. It would be one thing if it was simple scraping, but it's often genuine agentic abuse (just spamming for vulnerabilities* or scraping millions of pages with very little valuable or dynamic content). These clouds must get the abuse happening under their own roof under control.I banned Alibaba's Cloud two years ago and it still hits one of my domains 1 million times per day, despite every route 4xx-ing during that time.
Even Apple has aggressively ramped up its scraping in the past few months, but thankfully they do respect robots.txt.
It's frustrating because some of my real users route through these clouds via their work VPNs.
* like leaked .env routes, brute forcing admin panels, and so on
That's a big reason that I never post any of my apps here.
Welcome to hell, kid...
On the other hand, “multiple times per minute” doesn’t seem that high? Even if it was 60x per min that’s fully within the limits of what a 2015 laptop in the closet can handle.
ben_w•47m ago
https://www.sammystraus.com/#using-llms-to-make-the-emacs-we...
sammy0910•22m ago