If this is true, establishing an institution to ensure things like "publishing model cards with technical details, maintaining strong internal cybersecurity, vetting key personnel, and providing sufficient resourcing for safety and security research" is really mostly irrelevant.

TFA does talk about what really needs to be done, but punts this into future work: "Even if we solve these hard technical challenges, there will be further complex economic and philosophical questions to tackle: what sorts of new economic models will be needed to help everyone thrive in a post-scarcity world? What values do we want to live by, what will meaning and purpose be, and how might even the human condition itself change?"

There's also a need to consider the rights that this new intelligence should have.

>The American government, he says, should develop a system for testing the safety of new AI models before they are released. “It’s important that it’s not just an industry body,” he adds. But a regular government agency wouldn’t do either. “It would not be able to move fast enough, or have the right resources.” Instead, Sir Demis suggests taking inspiration from FINRA, the Financial Industry Regulatory Authority, a private agency in America that regulates brokers and stock markets.

There is a heatwave in London, perhaps Demis needs to stay out of the sun and drink more water.

Or perhaps he is seeking more funding/a fight to maintain his divisions AGI research budget.

There is real and potential harm from AI, but the more someone talks/write abut AI safety, the less they care about actual harm to real people, economy and what not.

I’m going to have to flag this because it is obnoxious and absurd.

At this point I'd say the societal risk of AI isn't models gone wild, or used by the bad guys. Regulation will take care of itself, and it seems the AI companies will not only welcome it, but lobby for it to shift responsibility to the government.

The real risk of AI is societal disruption due to job displacement, and maybe other structural changes, and this is far harder to solve, and likely will not be solved, or even seriously addressed, until/unless politicians feel like their own jobs and well-being depends on them addressing it.

I find it mind boggling that someone could be this tone-deaf to the current situation. No "ally" of the US is going to (willingly) agree to this governance structure given the current US administration's "might makes right" proclamations and threats on sovereignty of its continental neighbours.

And non-allies would just ignore. Unless forced by said "might makes right", which in the long run will have no staying power.

Apart from its completely delusional formulation, what is most concerning about this blog post is that it indicates that all 3 major US labs have formally submitted to boot-licking Trump/Bessent/Lutnick. I had I guess vainly held out hope that Google might be more reticent to do so.

Seems to be a lot of hubris with some AI thought leaders thinking control will remain with them and be absolute.

The standards body will have no teeth. whats to stop someone just not bothering?

Next, the threats he is asserting to check for (cyber, chemical, biological) are nice, but also not that useful.

We already have chemical and biological controls, that why I can't by anthrax spores or high concentration nitric acid.

The risks that AI has now are already playing out:

1) the evaporation of trust in the video as medium of "this happened"

2) systematic spying

3) job losses

Increased productivity means job losses, Tiktok, instagram and X are a wash with disinformtion campaign pumping your feeds with AI ragebait.

That is and will continue to fracture society so that only the strictly information controlled (ie authoritarian) have a functioning state.

if the author had bothered to engage with the world outside of tech, or even their local government, they would know that the proposal are dead in the water and frankly superfluous. The knowledge is out there, without AI. let us work on the issues we face now, rather than dipshit tech bro's miopic vision/funding manifesto.

Heck, a proof for P=NP or P!=NP or solve the The Riemann Hypothesis. Just give me something truly exciting and I will believe AGI is around the corner, until then I will see it as cool technology, that while beneficial to me, also helped cause the biggest amount of disinformation we've every seen.

If AGI is truly imminent and will collectively effect all of us why not apply democracy to it, and vote for new AI models?

So a small group of technocrats get together behind closed doors and secretly share their AI breakthroughs, and determine whether it's too powerful or not for the plebs in the public.

Who is watching the watchers?