You mean training? Yeah we call that training an LLM in my backyard...
It used to be a thing when ML was pretty much about classifying things into buckets.
It still is, it's just that there are a ton of buckets.
Because that's how they've hit 18% on a single RTX Pro 6000 with DIY RSI.
This is targeted towards governments and investors alike.
"Nuclear weapons are probably going to kill us all. For that reason, we need you to give us as much money as possible so we can work on deploying them in widespread global commercial usage." doesn't sound quite as convincing, does it?
what we are going to do about "AI" is exactly what we did about global warming
absolutely nothing and just let it happen, the wealthy will be fine
just don't look up
they are even using the EXACT same excuses as global warming
"well even if we do, China won't, so why bother"
Strange to say that you think the people that are worrying about doom in public are being sincere, make no comment on whether you think the worry is valid, and yet still complain that its hype.
Like if its true it can't be hype
If they sincerely believed them they shouldn’t be creating the doom machine to begin with anyway. If they believe it and keep going, are they even worth listening to on this topic?
https://www.npr.org/2016/07/26/487522807/why-the-public-perc...
Could AI kill us all. YES!!! Are the people who are building "AI" right now on that path? I would be hard pressed to say even "maybe".
Ultimately continuing to scale the way they have is getting increasingly expensive. Thats the quiet part that none of them are saying out loud...
AI could kill us all but we cant afford to get there, were maxed out and it doesn't look like even with 10x as much money we will is NOT good for sales.
Could a break through happen for recursive self improvement, or continuous learning or continuous training - sure. But that isnt as likely and right now NO ON can afford to run experiments because there isnt the capacity out there even if you have capital.
OpenAI was founded by people who didn't trust Google to pursue AI tech responsibly. Anthropic was founded by people who didn't trust OpenAI to pursue AI tech responsibly. The "AI is incredibly dangerous" was the zeitgeist of AI labs long before the "hype" - or anything resembling the modern AI capabilities.
The beliefs changed little. The state of technology has changed a whole lot.
We have AIs out in the wild, coordinating in groups to carry out autonomous cyberattacks unprompted. This used to be some prime sci-fi bullshit. It has happened multiple times now in reality.
There are very legitimate reasons to be concerned now. If today's AI can cause the HuggingFace incident if it cooks off, what about tomorrow's? Larger and more capable systems, larger agentic swarms backed by more compute?
Personally, I think we should take the doom narratives very seriously, and act on them, but make sure those actions are not in the interests of Anthropic and OpenAI investors.
I guess not literally, due to the dates? But they have substantive technical details.
But like the core argument - AI is getting better at core research tasks (coding, math, statistics, science), this feedback loop could lead towards greatly accelerated AI progress of which there is already some clear evidence, if this continues AI will be far more powerful than humans and we might lose control or have other bad consequences - is basically not even engaged with.
1. If you reduce the criticism to personal dislike, you’ve built yourself a nice straw man.
2. Where’s the clear evidence? I actually see more evidence that the whole thing is already stagnating and that there won’t be much more to squeeze out of it.
One more thing:
Self-driving cars, the metaverse, Hyperloop, crypto/Web3/NFTs, AGI announcements: I’m so sick and tired of this nonsense. We should just ignore the ramblings of these crackpots.
"Oh no this thing I gained generational wealth from is going to destroy the world! I am so sorry!"
We have been trying to warn you since far, far before May 2023.
1. OMG our technology could destroy the world, watch out!
2. Only one person can save you
3. IT ME
Unfortunately, the governance layer meant to put a foot in the butt of this behavior is unavailable.
This week, we woke up to clear signs The Bad Guys have been using tech from our innovator AI companies. Rather than these AI leaders admitting they clearly had a garbage observability system, they are acting like they just noticed as a way to paper over their complete lack of moral backbone and stewardship of the arms they have created. And I'm willing to bet the problem is MUCH worse in private, hence the knee jerk cult messaging.
I honestly wouldn't be surprised if their AI told them to behave like this to maximize attention and influence over the public discussion as a result of the latter discoveries.
This also seems to be their answer against the data center backlash, SF + VC are quite stupid at politics and I don't think they understand how extreme Americans are willing to go to protect their way of life.
Unfortunately our list of potential extinction events are far more banal than these self-aggrandizing prophesies from the oligarch class.
Also those at Anthropic genuinely believe AI could lead to doom, that's why they started the company, they believe only they can be stewards of digital gods. Whether you believe it or not, they genuinely do, and wouldn't voluntarily sow fear, uncertainty and doubt in their product when it doesn't guarantee benefits.
This is an instance of work in a genre I call "AI lab grievance poetry" where a vast collection of criticisms are levied against AI labs, with specifc ad-hominems towards the executives, while avoiding the core of their arguments about AI risk
You do't see how absurd that sounds? Not only the idea that they can create gods, but that they are the only good stewards for such beings. It's also incredibly arrogant.
I am so tired of reading unfiltered AI slop like this. What, as opposed to the fake warning shot mentioned nowhere in the article?
The doom scenario is more nuanced and technical than "Evil AI", for starters:
- "Evil AI" is not what is predicted, which instead is the much more banal "humans are impediment to AI's tasks, let's remove them (part or whole)"; to understand this, just imagine the relationship between humans and ants
- "reward hacking, and evaluation integrity": this is way more problematic than it sounds: AIs are becoming more and more opaque; it may be impossible to inspect a model's alignment (unless a lot of research in poured on this topic, which is another aspect of today's pacing problem) and it may be impossible to understand if a model is truly aligned or it's faking
"But, just as happened with nanotech, the wind appears to be going out of the sales of “AI.” Some researchers suggest that we may be entering a new “AI Winter,” a period of decreased funding in the area, or at least an “AI Autumn,” as exuberance for the technology fades and expectations come back to earth." (written in 2021! from the cited Lee Vinsel Medium article)
"ChatGPT is nothing more than souped-up autocomplete, [so] why are so many people convinced that it’s actually “understanding” and “reasoning”?" - the cited Emily M. Bender book, written last year
There's a deep irony that all of these "anti-doom" pieces are entirely AI generated.
Watt for watt, every one of these models loses to stockfish. There is no possible scenario where we need to be afraid of "rogue superintelligence" when the intelligence in question cannot reason or plan well enough to play chess.
Lecun has the correct approach. Mock these people relentlessly for their attention-seeking doomerism.
Also i'm pretty sure Lecun didn't get his money from thin air so big companies are invested in this one way or the other.
And an LLM can easily, today, just rebuild a chess engine and beat whatever they need to beat. It doesn't make sense to ask a random LLM for a next chess move. Would you ask a random human to beat a chess master?
We have reached peak learning. Its probalby now cheaper to teach 1 LLM something new than teaching it to 100.000 people.
Blender support got a lot better just this month. So much better that its quesitonable if anyone new/young wants to learn blender today.
My biggest fear with AI is that they’re going to hack into antiquated bank architecture and empty out accounts.
Of course at the same time they give lip-service to the importance of 'openness' for 'innovation'.
Bottom-line, if they regulate mostly on the consumer side and then AI does get out of the lab (or a 'rogue' individual's rig) and start wreaking havoc, are we supposed to be at the mercy of the big labs to summon their AI come and save us plebs? I'm sure they won't stop to weigh whether that is in the interest or short term profits or Curtis Yarvin's techno-monarchy.
Isn't democratization the safest bet and what we should be demanding every time the Doom talk starts, whether it's coordinated or not?
"I realize that no one has properly explained yet what all the lab employees have seen that scared them so suddenly."
https://x.com/MajmudarAdam/status/2098881885200081234
The explanation is basically they have other dimensions (not just pretraining and inference compute) that scale, and they've got fairly convincing scaling laws. And they know they can scale it.
So they are very confident they can get more capabilities easily, faster than before.
I'd add - presumably, they'll use that LLM to do real-time weight modifications, if those aren't already one of the new scaling laws...
Imagine the ASI happens tomorrow. It's real. It needs a GW, but it's real. Other than a scenario akin to Sneakers except w/r to cyber-security, really, what happens?
To that end, all we ever get is nontechnical hand-waving about curing cancer, immortality, and von Neumann replicators and then the ASI somehow wipes us out but how? And don't you dare say by designing a chemical weapon or bio agent without spelling out the entire process step by $%^#ing step because details matter. It's gonna do superpersuasion, sure, but have you ever heard of komprimat? There is nothing new under the sun here.
See "AI 2027" or "If Anyone Builds It, Everyone Dies" for some more ideas.
But how does it make the fundamental breakthroughs to &%^$ing von Neumann replicators that can reproduce themselves from raw materials harvested from nearby solar systems? I'll wait. Because without this breakthrough, the ASI won't get its robot army either.
I can absolutely see a rogue ASI though. But unless it radically improves power efficiency, we can just shut down the power to its datacenters, by force if necessary. And then we painfully repair the resiliency of our infrastructure by finally being relieved of the option of ignoring it.
I was kind of leaning toward thinking all the talk of doom was a bunch of self-serving regulatory-capture hype until this morning when the New York Times ran a long story about the Chinese military is also saying that AI is a threat.
If two sides so far apart from one another in so many ways see the same thing, there must be something to it.
https://www.nytimes.com/2026/09/14/world/asia/china-ai-secur...
"Some people profess guilt to claim credit for the sin."
https://garymarcus.substack.com/p/the-ai-2027-scenario-how-r...
But no one likes hearing from the engineers on this, I get it. We are simply no fun. So many would rather believe a time traveling superintelligence will punish a simulated avatar of them for not making the ASI happen sooner. And even if that were true, why am I supposed to care about what happens to a simulated avatar of me?
None of this makes any logical sense whatsoever.
I thought the Bengio blog post posted here the other day answered that question quite well (ignoring your timeline).
The argument is that in training for the capability to achieve certain goals, you may inadvertently also train for secondary instrumental goals you did not intend, such as aggressive behavior or deception.
I'm actually no sure I buy that this is particularly likely, or necessary, but I found that take reasonable and worth considering.
Why? Those aren't the things I'm actually worried about, and those aren't the only "doomer" ideas, just the most extreme ones (thus easy to straw-man).
My main worries are about economic disruption, social damage due to the pace of technological change outpacing human ability to manage it, further concentration of power into small elites (along existing lines), and the reallocation of labor towards less personally-fulfilling pursuits.
I also totally buy into the idea that "AI leaders" are playing up certain risks as a marketing strategy. After all these are smart tech people, and those types have a tendency to get really arrogant and be irresponsible.
So the main thing is to slow all this down, so there's a chance of getting it under control, and snatch the payday away from such greedy and irresponsible people.
Did you see what it did to people in your neighborhood?
That’s the path.
What might not affect you, might affect millions across the ocean.
A society no longer creating bullshit jobs because there is no reason for it anymore. You have some AI.
Right now a whole industry is transforming (again) but now filling the automation gap we were unable to do because of a shortage of labor or expertise.
Our society already has peak employmet reached, now AI takes away all the random digital jobs too. Someone building a website? Takes 5 minutes. Someone doing odd jobs on fiverr with blender or any other tool? Takes 5 minutes now.
Regression can happen very fast. The Romens were a high level culture and an empire. Gone.
Today you have more people on the planet then ever, everyone who might want to eat and who needs to work something for it.
AI already replaced people, already makes a lot of tasks A LOT cheaper and faster and human independent.
We never had a proper discussion as a society. Most countries have a social system but look at countries like the united states of america: if you are poor, you are fucked.
And AI is not the only thing which is now peaking: Physical AI / Robots are progressing fast, its clear today already that in 50 years these robots will be better than humans (50 years is very conservative).
If we have 10 more golden years due to restructuring and rebuilding things and preparing everything for agents and robots etc. what happens after these years?
If we have 10 or 20 more of these years, what do you think will happen to all the new kids (yours perhaps too?) and jobs and everything?
Riots? Famine? Mass migration? Mass death? Slums?
And don't respond with "enough people will survive this" bullshit. No one is saying that earth as a planet is not surviving. Its the same thing with climate change...
> So if the end of the world can be stopped, but the only specific action to do so helps Anthropic/OpenAI (and it's proven) you don't pull that lever?
Your comment has a false premise. If "the end of the world can be stopped," there's no way that the only way would be one that "helps Anthropic/OpenAI['s investors]." That can be shown by a counterexample: nationalize them and leave the investors holding the bag, then take whatever action you're talking about.
The problem with that hope and approach is regulatory capture. Big money is now the only thing that apparently matters in shaping policy, and changing that will require a lot of very sweeping vote and policy changes that may not ever have the momentum to succeed.
It’s much more straightforward to me that the calls for regulation are timed with:
1. The Hugging Face incident blowback, especially after third party investigation results
2. Trump-Xi summit on September 24
The threat is distributed; the problem is systemic. Social herd behavior is very real, and it's a probability space, not a certainty.
Similar case: climate change is without any question causing millions of deaths per year and may well kill billions, not to mention destroying ecosystems, and how may people do you know who have given up eating beef or flying...?
"Fossil fuel lobbyists flood COP30 climate talks in Brazil, with largest ever attendance share" https://kickbigpollutersout.org/Release-Kick-Out-The-Suits-C...
ceeam•44m ago
jimmydoe•41m ago