(There are several biological threats the author hasn't considered. I'll just leave it at that.)
If there is a fully general superintelligence - in the current state of alignment - we will all die for some reason. It doesn't really matter what the reason is.
The real deal needs a lab messing around with viruses and whatever happened a few years ago ... and it did not destroy humanity.
AI will really aid the universe in generating more problems. Maybe it's not a supervirus, but hell, maybe it is.
"we could be attacked by an infinity of problems, which means that worrying about all of them is statistically unpractical"
This is such a massively overblown worry that AI companies who are seeking regulatory capture love to roll out whenever they are trying to gin up some FUD.
Creating viruses at home is extremely difficult, even if you have a perfect instruction manual and someone sitting behind you telling you what to do. It's full of trial and error and validating each step requires rigor.
It's also a process which requires fairly expensive and specific supplies, which can be easily gatekept. Instead of trying to gate-keep knowledge of biology, we can do things like KYC around specific types of lab supplies (like we already do with fertilizer which can make bombs, or specific types of supplies needed to make nukes). Heck the Biden admin already added a KYC requirement for plasmids and plasmid companies already have implemented screening for sequences which could be dangerous.
It'll probably just hack and disable our water works, power stations and supply chains and let us all starve.
> PhD students in 2026 using state-of-the-art AI software are spending months or years trying to design simple peptide binders that inhibit some enzyme or pull down some protein
This confuses me, as state-of-the-art AI software is moving quite quickly. Any software that is currently state-of-the-art AI.. hasn't existed for years, maybe not even for months.
It also seems quite odd to claim that there are no sufficiently dangerous diseases when we have high-security research labs dedicated to exploring.. exactly these kinds of diseases.
And, even if it's not a disgruntled teenager, it just takes one leak from a lab that has 10xd productivity using AI tools.
This is a real risk that existed before AI, one that I think we can now confidently say that society is under-prepared for since 2020. It is certainly magnified by AI, even if it's not 100x.
The author even seems to recognize this:
> So, if you’re worried about AI-designed superviruses, think COVID, not Ebola. And if you’re now a little less worried, maybe you understand where I’m coming from.
This is exactly where I think most people are starting from, and that's _why we're worried_.
infinite_spin•17m ago
mepiethree•8m ago
mepiethree•6m ago