Ilya has appeared to shift to closer to Yann's position, though: he's been on the "scaling LLMs will fail to reach AGI" beat for a long time.

Possibly... but also a lot of the foundational AI advancements were actually done in skunkworks-like environments and with pure research rather than iterating in front of the public.

It's not 100% clear to me if the ultimate path to the end is iteration or something completely new.

We are in a situation where the hardware is probably sufficient for AI to do as well as humans, but in terms of thinking things over, coming to understand the world and developing original insights about it, LLMs aren't good, probably due to the algorithm.

To get something good at thinking and understanding you may be better rebuilding the basic algorithm rather than tinkering with LLMs to meet customer demands.

I mean the basic LLM thing of have an array of a few billion parameters, feed in all the text on the internet using matrix multiplication to adjust the parameters, use it to predict more text and expect the thing to be smart is a bit of a bodge. It's surprising it works as well as it does.

Its charlatans like sama that muddy the waters by promising the sky to get money for their empire building.

LLMs can make and are great great products. But its sneaky salesmen that are the ones saying scaling is the path to AGI. The reality is that they're just aiming for economies of scale to make their business viable

Stick a microphone and camera outside on a robot and you can get unlimited data of perfect quality (because it by definition is the real world, not synthetic). Maybe the "AGI needs to be embodied" people will be right, because that's the only way to get enough coherent multimodal data to do things like long-range planning, navigation, game-playing, and visual tasks.

> Where does the next 1000x flops come from? Even with Moore's law dead, we can easily build 1,000x more computers. And for arguments about lack of power - we have sun.

Everyone at anthropic is saying ASI is imminent…

But what we're seeing at the moment, is a deceleration, not an acceleration.

So we're rehypothecating CDOs like the last bubble?

When talking with non-tech people around me, it’s really not about “rational minds”, it’s that people really don’t understand how all this works and as such don’t see the limitations of it.

Combine that with a whole lot of FOMO which happens often with investors and you have a whole pile of money being invested.

From what I hear, most companies like Google and Meta have a lot of money to burn, and their official position towards investors is “chances of reaching ASI/AGI are very low, but if we do and we miss out on it, it will mean a huge opportunity loss so it’s worth the investment right now”.

Though I do not fully know where the boundary between "a model prompted to iterate and use tools" and "a model trained to be more iterative by design" is. How meaningful is that distinction?

But the people who don't get this are the less-technical/less-hands-on VPs, CEOs, etc, who are deciding on layoffs, upcoming headcount, "replace our customer service or engineering staffs with AI" things. A lot of those moves are going to look either really silly or really genius depending on exactly how "AGI-like" the plateau turns out to be. And that affects a LOT of people's jobs/livelihood, so it's good to see the hype machine start to slow down and get more realistic about the near-term future.

Then, he starts to talk about the other ideas but his lawyers / investors prevent him from going into detail: https://youtu.be/aR20FWCCjAs?t=1939

The worrisome thing is that he openly talks about whether to release AGI to the public. So, there could be a world in which some superpower has access to wildly different tech than the public.

To take Hinton's analogy of AGI to extraterrestrial intelligence, this would be akin to a government having made contact but withholding the discovery and the technology from the public: https://youtu.be/e1Hf-o1SzL4?t=30

It's a wild time to be alive.

This is basically all of it.

Kind of how word processors solved the writing is tedious struggle and search solved the "can't curate the internet" struggle.

And from my experience there are lots and lots of jobs that are just "clicking the right buttons".

Another way is CPU+ + fast memory, like Apple does. It's limited but power efficient.

Looks like with ecosystem development we need the whole spectrum from big models+tools running on datacenters to smaller running locally, to even smaller on mobile devices and robots.

  > Won't scaling laws apply to it anyway?

The fight was never "Scale is all you need" (SIAYN) vs "scale is irrelevant" it was "SIAYN" vs "Scaling is not enough". I'm not aware of any halfway serious researcher that did not think scaling was going to result in massive improvements. Being a researcher from the SINE camp myself...

Here's the thing:

The SIAYN camp argued that the transformer architecture was essentially good enough. They didn't think scale was all you needed, but that the rest would me minor tweaks and increasing model size and data size would get us there. That those were the major hurdles. In this sense they argued that we should move our efforts away from research and into engineering. That AGI was now essentially a money problem rather than a research problem. They pointed to Sutton's Bitter Lesson narrowly, concentrating on his point about compute.

The SINE (or SINAYN) camp wasn't sold. We read the Bitter Lesson differently. That yes, compute is a key element to modern success, but just as important was the rise of our flexible algorithms. In the past we couldn't work with such algorithms because of lack of computational power, but that the real power was the algorithms. We're definitely a more diverse camp too, with vary arguments. Many of us look at animals and see that we can do so much more with so much less[2]. Clearly even if SIAYN were sufficient, it does not appear to be efficient. Regardless, we all agree that there's still subtle nuances in intelligence that need working out.

The characteristics of the scaling "laws" matter but it isn't enough. In the end what matters is generalization. For that we don't really have measures. Unfortunately, with the SIAYN camp also came benchmark maximization. It was a good strategy in the beginning as it helped give us direction. But we are now at the hard problem with the SINE camp predicted. How do you do things like make a model a good music generator when you have no definition of "good music"? Even in a very narrow sense we don't have a half way decent mathematical definition of any aesthetics. We argued "we should be trying to figure this out so we don't hit a wall" and they argued "it'll emerge with scale".

So now the cards have been dealt. Who has the winning hand? More importantly, which camp will we fund? And will we fund the SIAYN people that converted to SINE or will we fund those who have been SINE when times were tough?

[0] They've been power laws and I expect them to continue to be power laws[1]. But the parameters of those laws do still matter, right?

[1] https://www.youtube.com/watch?v=HBluLfX2F_k

[2] A mouse has on the order of 100M neurons (and 10^12 synapses). Not to mention how little power they operate on! These guys can still our perform LLMs on certain tasks despite the LLMs having like 4 orders of magnitude more parameters and many more in data!