I agree that it is sad to see many online book stores moving from "selling" to "renting". But that is a completely different problem.

As a personal note, I know the pain of not being able to access scientific papers because they were behind paywalls, and I had to search for drafts to be able to read them. But that model was well in place circa 2010, thus it's and old tactic applied to a new field: books (and others).

Then I can just start pirating them much more pervasively. Problem solved. Buying mere access to things like books and media has its uses, but to make it obligatory as an alternative to true ownership is a tendency that can go die.

Thus, regardless how well one optimizes his site delivery (static site, minimizing, CDN, caching, etc.) a stampede of bot crawling does in the end become a DDoS, which if it doesn't take down the infrastructure, it might leave a deep hole in one's budget.

For example, for one of the sites I manage, I get daily peaks of ~300 requests per second measured at the backend, for a site that already employs heavy caching (both client-side, CDN, and even server-side). This wasn't so a few months back, and the site didn't just jump in popularity.