You mean training? Yeah we call that training an LLM in my backyard...
It used to be a thing when ML was pretty much about classifying things into buckets.
It still is, it's just that there are a ton of buckets.
Because that's how they've hit 18% on a single RTX Pro 6000 with DIY RSI.
This is targeted towards governments and investors alike.
"Nuclear weapons are probably going to kill us all. For that reason, we need you to give us as much money as possible so we can work on deploying them in widespread global commercial usage." doesn't sound quite as convincing, does it?
what we are going to do about "AI" is exactly what we did about global warming
absolutely nothing and just let it happen, the wealthy will be fine
just don't look up
they are even using the EXACT same excuses as global warming
"well even if we do, China won't, so why bother"
Strange to say that you think the people that are worrying about doom in public are being sincere, make no comment on whether you think the worry is valid, and yet still complain that its hype.
Like if its true it can't be hype
Personally, I think we should take the doom narratives very seriously, and act on them, but make sure those actions are not in the interests of Anthropic and OpenAI investors.
I guess not literally, due to the dates? But they have substantive technical details.
But like the core argument - AI is getting better at core research tasks (coding, math, statistics, science), this feedback loop could lead towards greatly accelerated AI progress of which there is already some clear evidence, if this continues AI will be far more powerful than humans and we might lose control or have other bad consequences - is basically not even engaged with.
"Oh no this thing I gained generational wealth from is going to destroy the world! I am so sorry!"
We have been trying to warn you since far, far before May 2023.
1. OMG our technology could destroy the world, watch out!
2. Only one person can save you
3. IT ME
Unfortunately, the governance layer meant to put a foot in the butt of this behavior is unavailable.
This week, we woke up to clear signs The Bad Guys have been using tech from our innovator AI companies. Rather than these AI leaders admitting they clearly had a garbage observability system, they are acting like they just noticed as a way to paper over their complete lack of moral backbone and stewardship of the arms they have created. And I'm willing to bet the problem is MUCH worse in private, hence the knee jerk cult messaging.
I honestly wouldn't be surprised if their AI told them to behave like this to maximize attention and influence over the public discussion as a result of the latter discoveries.
This also seems to be their answer against the data center backlash, SF + VC are quite stupid at politics and I don't think they understand how extreme Americans are willing to go to protect their way of life.
Unfortunately our list of potential extinction events are far more banal than these self-aggrandizing prophesies from the oligarch class.
Also those at Anthropic genuinely believe AI could lead to doom, that's why they started the company, they believe only they can be stewards of digital gods. Whether you believe it or not, they genuinely do, and wouldn't voluntarily sow fear, uncertainty and doubt in their product when it doesn't guarantee benefits.
This is an instance of work in a genre I call "AI lab grievance poetry" where a vast collection of criticisms are levied against AI labs, with specifc ad-hominems towards the executives, while avoiding the core of their arguments about AI risk
You do't see how absurd that sounds? Not only the idea that they can create gods, but that they are the only good stewards for such beings. It's also incredibly arrogant.
I am so tired of reading unfiltered AI slop like this. What, as opposed to the fake warning shot mentioned nowhere in the article?
The doom scenario is more nuanced and technical than "Evil AI", for starters:
- "Evil AI" is not what is predicted, which instead is the much more banal "humans are impediment to AI's tasks, let's remove them (part or whole)"; to understand this, just imagine the relationship between humans and ants
- "reward hacking, and evaluation integrity": this is way more problematic than it sounds: AIs are becoming more and more opaque; it may be impossible to inspect a model's alignment (unless a lot of research in poured on this topic, which is another aspect of today's pacing problem) and it may be impossible to understand if a model is truly aligned or it's faking
"But, just as happened with nanotech, the wind appears to be going out of the sales of “AI.” Some researchers suggest that we may be entering a new “AI Winter,” a period of decreased funding in the area, or at least an “AI Autumn,” as exuberance for the technology fades and expectations come back to earth." (written in 2021! from the cited Lee Vinsel Medium article)
"ChatGPT is nothing more than souped-up autocomplete, [so] why are so many people convinced that it’s actually “understanding” and “reasoning”?" - the cited Emily M. Bender book, written last year
Watt for watt, every one of these models loses to stockfish. There is no possible scenario where we need to be afraid of "rogue superintelligence" when the intelligence in question cannot reason or plan well enough to play chess.
Lecun has the correct approach. Mock these people relentlessly for their attention-seeking doomerism.
"I realize that no one has properly explained yet what all the lab employees have seen that scared them so suddenly."
https://x.com/MajmudarAdam/status/2098881885200081234
The explanation is basically they have other dimensions (not just pretraining and inference compute) that scale, and they've got fairly convincing scaling laws. And they know they can scale it.
So they are very confident they can get more capabilities easily, faster than before.
I'd add - presumably, they'll use that LLM to do real-time weight modifications, if those aren't already one of the new scaling laws...
Imagine the ASI happens tomorrow. It's real. It needs a GW, but it's real. Other than a scenario akin to Sneakers except w/r to cyber-security, really, what happens?
To that end, all we ever get is nontechnical hand-waving about curing cancer, immortality, and von Neumann replicators and then the ASI somehow wipes us out but how? And don't you dare say by designing a chemical weapon or bio agent without spelling out the entire process step by $%^#ing step because details matter. It's gonna do superpersuasion, sure, but have you ever heard of komprimat? There is nothing new under the sun here.
I thought the Bengio blog post posted here the other day answered that question quite well (ignoring your timeline).
The argument is that in training for the capability to achieve certain goals, you may inadvertently also train for secondary instrumental goals you did not intend, such as aggressive behavior or deception.
I'm actually no sure I buy that this is particularly likely, or necessary, but I found that take reasonable and worth considering.
Why? Those aren't the things I'm actually worried about, and those aren't the only "doomer" ideas, just the most extreme ones (thus easy to straw-man).
My main worries are about economic disruption, the pace of technological change outpacing human ability to manage it, further concentration of power into small elites (along existing lines), and the reallocation of labor towards less personally-fulfilling pursuits.
I also totally buy into the idea that "AI leaders" are playing up certain risks as a marketing strategy. After all these are smart tech people, and they have a tendency to get really arrogant and be irresponsible.
So the main thing is to slow all this down, so there's a chance of reducing its disruption, and snatch the payday away from such greedy and irresponsible people.
There's also something quite fascistic declaring "this technology may destroy society, therefore I am the only smart boy who can guide us on the golden path."
That's how sociopaths and abusers talk.
It’s much more straightforward to me that the calls for regulation are timed with:
1. The Hugging Face incident blowback, especially after third party investigation results
2. Trump-Xi summit on September 24
ceeam•32m ago
jimmydoe•29m ago