Associated post: Introducing Plan A

https://www.astralcodexten.com/p/introducing-plan-a

> Then, in the mid-2030s, they pause at AIs around the level of top human geniuses.

They being the US and China and by agreement.

It would be ideal, but there’s far too much money on the table to overcome human nature.

So my hope is we hit some kind of limits naturally.. Wishful thinking?

If carbon taxes are already a lethal policy for an political campaign, it's absurd to think that fears of ASI will create any real movement around pausing AI.

If there is any movement to pause AI development, it will come from the general public's dislike of these companies. Not from the AI safety angle.

I wonder if they are double-counting Anthropic's leased capacity from SpaceX under SpaceX again.

I'm sure some people will have issue with my phrasing but, honest question:

Are there examples of where we have collective decided not to pursue knowledge? Successfully?

I guess nuclear weapons might be the best example though research doesn't seem have to actually "stopped" as much as gone underground and we still have country trying to climb that ladder.

But I don't know how relevant that is to LLMs/AI. It almost feels like pandora's box is open and our only option is continue to improve them. There is clearly value in what they do and while I can absolutely see the dangers, for example: authoritative governments and surveillance, I'm not convinced to throw the baby out with the bathwater.

All of technology back to the printing press (and probably before that) could also be said to make it easier for governments to oppress their citizens. Making laws (and enforcing them!) to prevent governments from doing these things feels like that route forward, not trying to stick our heads in the sand.

Perhaps I'm horribly naive, perhaps I just see the SciFi future I've spent my life reading and dreaming about on the horizon and I'm blinded by the reality, perhaps my ideals around "knowledge deserves to be free/accessible" are misguided. I don't know.

People overestimate progress in physical world. 2035: robot population will soon be larger than the human one

I'd bet that in most places 9 years is about the time needed to build a residential building. I think a good way to think about this is to think of this as producing a serial car. From pitching and capital acquisition to building a prototype to software, regulatory and then the final product which needs multiple factories and supply chains. Yes, of course robots sound cooler and there are compounding effects yada yada, but on the other side there are as many obstacles as things that accelerate this product (like capital acquisition and fearmongering of gov to bend regulatory stuff faster).

More wild speculation, now with wishful thinking spread on top.