Gasoline has gone from barely being able to power stationary farm machines to now being the fuel that underpins our entire economy. So, great news all around, right?

> which predict an exponential increase

And was that actually delivered?

Real question: If a model goes from 80% accurate to 85% accurate is that an exponential increase in "cognitive capabilities?" Are we considering training costs and effort?

I feel significantly less sympathy for Anthropic's Supply Chain Risk designation if they believe the government should have this power over them. You get what you sign up for.

I like to stay up to date on things but more and more I’m finding myself pointing codex at a URL and saying “get to the point”.

As annoying as their tone is, the real big danger is what they are setting up for. All this fear-mongering around Mythos, the overly aggressive controls on Fable, and these manifestos they keep writing, are part of setting up for REGULATORY CAPTURE. Even collaborating with the Pope and the Interfaith Alliance (https://iafsc.org/our-work/faith-ai-covenant) are part of creating a vast support network for regulations and restrictions. Those regulations will help those faith organizations or the government or whatever, but will also help Anthropic’s bottom line.

Those regulations will not support your civil liberties. They will restrict speech, access to AI, and allowed uses of AI. They will lead to bans on use of models from some countries like China, and also bans on open-source or open-weight models.

If Dario wants to be trusted, he needs to explicitly say in writing that Anthropic will not support any legal or regulatory restrictions on open-source AI, open-weight AI, or Chinese models. Otherwise, what he is really saying - even as he claims he is trying to ‘defend democracy’ - is that he and Anthropic do not truly support fundamental rights like our right to speech.

It’s not just Anthropic either. OpenAI had their own recent polemic, pushing for regulations like mandatory safety reviews by agencies for “frontier” models (https://news.ycombinator.com/item?id=48387246). It’s a dead giveaway that these companies have no moats, are in serious danger of being a commodity, and are now in the process of using regulations and enshittification to hold onto money and power.

So, basically, make open-weight models illegal. It's nice for Dario to come out and say this so explicitly.

- You shall not embed copyrighted material in your models.

- You shall not bombard every little website in existence with 1 million scraping queries per day.

- You shall not use your political influence to pump and dump your AI (or rocket?) company.

- You shall not imperill the whole IT sector by buying all CPU and memory chips.

These new rules will affect every society directly in a positive way. Thanks.

I'm not going to claim that the CEO of pre-IPO company has no incentive to bolster the claims of his tech, but to completely disregard everything he is saying based on that seems awfully binary.

I don't know whether people are just high on copium, spouting "it's just fancy autocomplete" or "only humans can really be creative" on every LLM-related thread, but it is impossible to deny that in a span of a few years we've gone from models that could barely put together a sentence, to something maybe not equivalent to a junior developer, but at least resembling it.

And sure, you can point out every flaw that current day LLMs have, just how everyone pointed out that Stable Diffusion couldn't generate accurate hands (until it could 6 months later!). But the gradient is pretty clear and I am yet to see a well-argued narrative from anyone why scaling laws should fail in the next year or two (by which point it feels like we're going to have a real problem, extrapolating the current trajectory).

I'm very glad this discussion is at least being had, and I wish everyone would get off their high-horse and take things a bit more seriously.

People get income from one of three places: capital income, labor income, or the welfare state. If this technology truly unlocks a holy panacea of productivity with a commensurate drop in employment then capital’s share of the national income can and should provide for a wider and deeper welfare state. Nothing new need be invented here. Dario’s long and only somewhat organized list of policy interventions makes appropriate preparedness sound like a manic pulling of any and all levers when a simple theory of distribution will suffice.

We all want to nuclear codes so badly. We are addicted to intelligence and labour so badly that we simply can't concieve that a pro-social actor might want us all not to have it, and for good reason.

I mean... Obviously, insiders like Oppenheimer (who dedicated their lives to considering the implications of the technology under discussion), they just feared nuclear proliferation because they wanted all the profits for themselves, right :(

It's not clear to me on which side of the coalition USA is meant to be in this divide. And as an European I'm not sure whether being in China's or USA's coalition is better in the long term.

In general, this deliberate mongering of ever more geopolitical division is extremely harmful. As is the Trump bootlicking.

1. I remain unconvinced that "the exponential" for LLM based agents is more transformative than the shift from paper to computing. A lot of what LLM agents might do just seems like more of what computing and internet already did, including workers getting displaced.

2. I remain unconvinced that we are close to superintelligence, or that LLM based agents will get there through a self improving loop. Major progress is possible, but high confidence in this path is unwarranted.

3. I remain unconvinced that people at AI model companies are expert judges here. They are hyping themselves up, and calling software engineers "AI researchers" does not make them experts on these social, economic, political, or even technical issues.

> AI has become a major commercial technology

>Frontier AI models, like airplanes, should be required to go through technical testing and auditing, and their release should be blocked or reversed as a threat to public safety if they do not meet high standards of safety

> AI companies that develop advanced AI models must have strong security standards that protect their model weights

Anyway Dario's financial interests aside. This is an interesting breakpoint for me.

> Second, any response to AI-driven job displacement needs to address both the need to provide for everyone economically, and the need for people to find meaning, purpose, and agency. The latter is ultimately more important

To me this reads as an out of touch statement. I think the majority of people on earth work to keep a roof over their heads. Of course work can be a source of meaning, purpose, and agency, but to call it the more important aspect on a societal level is a sort of rich person like Dario statement to make.

They are asking for FAA style preclearance and third party audits. That literally means no new AI startup can emerge. Do they not know that audits cost money?

Protect your own monopoly, protect your customers' regulations. They want strong regulation like the FAA to raise barriers to entry for the foundation models they themselves build, but then why do they want to loosen FDA regulations? While at the same time driving token consumption from their own customers.

They talk about permanent job displacement and UBI. I usually call this "a morally packaged safe landing."

AI will generate astronomical productivity gains and capital profits, which AI companies privatize. So why should the social costs be paid by national taxes? In my opinion, something like "We will donate all of our AI companies' revenue for the next 10 years to society" would show genuine sincerity.

Then they say, if we do not develop AI, China will eat our lunch, and they go after China. But is not this really about preventing Chinese dumping, maintaining our own token prices, and asking the world to beat down China so that they can preserve global tech hegemony?

But by blocking China from the CUDA ecosystem, now the CANN ecosystem has emerged, has it not? If China develops techniques that reliably reduce inference costs, who knows how things will turn out then.

Honestly, I like Anthrpic's Claude, but the Anthropic CEO's rhetoric is so stale. It is not that it feels hypocritical. It is that this is just a one dimensional rhetorical tactic that assumes the public is stupid.