Quite exciting. Without diminishing the amazing value of LLMs, I don't think that path goes all the way to AGI. No idea if Carmack has the answer, but some good things will come out of that small research group, for sure.
I do agree that it is not particularly groundbreaking, but it's a nice "hey, here's our first update".
- Someone who is a programmer but follows a hypermasculine cliche and makes sure everyone knows about it.
- An insult used by other developers for someone who is more physically fit or interested in their health than themselves.
- An insult used by engineers or other people who are not happy with the over representation of men in the industry. So everyone is lumped in the category.
- Someone who is obsessed with the technology and trying to grind their skills on it to an excessive level.
sounds like a person who respects their own profession though
Isn't "interested in their health" a signal that they are interested in themselves, rather than the opposite?
B. A person who is physically and culturally indistinguishable from A
It's still a lot better to really learn and discover it yourself to really get it.
Also it's hard to determine how much time someone spent on particular topic.
seems that we are learning in layers, one of the first layers being 2D neural net (images) augmented by other sensory data to create a 3D if not 4D model (neural net). HRTFs for sound increases the spatial data we get from images. With depth coming from sound and light and learnt movements(touch) we seem to develop a notion of space and time. (multimodality?)
Seems that we can take low dimensional inputs and correlate them to form higher dimensional structures.
Of course, physically it comes from noticing the dampening of visual data (in focus for example) and memorized audio data (sound frequency and amplitude, early reflections, doppler effect etc). That should be emergent from training.
Those data sources can be inperfectly correlated. That's why we count during a lightning storm to evaluate distance. It's low dimensional.
In a sense, it's a measure of required effort perhaps (distance to somewhere).
What's funny is that it seems to go the other way from traditional training where we move from higher dimensional tensor spaces to lower ones. At least in a first step.
In this case, John is going off on this inane tangent because of his prior experience with hardware and video games instead of challenging himself to solve the actual hard and open problems.
I’m going to predict how this plays out for the inevitable screenshot in one to two years. John picks some existing RL algo and optimizes it to run in real time on real hardware. While he’s doing this the field moves on to better and new algorithms and architectures. John finally achieves his goal and posts a vid of some (now ancient) RL algo playing some Atari game in real time. Everyone says “neat” and moves on. John gets to feel validated yet all his work is completely useless.
He’s a AAA software engineer but the prerequisites to build out cutting edge AI require deep formal math that is beyond his education and years at this point.
Nothing to stop him playing around with AI models though.
All deep formal math is a boundary to a thing.
I'm pretty excited to see him in this domain. I think he'll focus on some DeepSeek style improvements.
Having JC focusing on, say, writing a performant OSS CUDA replacement could be bigger than any of the last 20 announcements from openai/goggle/deepmind/etc
He's probably one of the most qualified people around.
So I asked Ilya, their chief scientist, for a reading list. This is my path, my way of doing things: give me a stack of all the stuff I need to know to actually be relevant in this space.
And he gave me a list of like 40 research papers and said, 'If you really learn all of these, you'll know 90% of what matters today! And I did. I plowed through all those things and it all started sorting out in my head.
To put it another way, the idea that John Carmack is going to do groundbreaking research in AI is roughly as plausible as the idea that Yann LeCun is going to make a successful AAA video game. Stranger things have happened, but I won’t be holding my breath.
One question I would have about the research direction is the emphasis on realtime. If I understand correctly he's doing online learning in realtime. Obviously makes for a cool demo and pulls on his optimisation background, and no doubt some great innovations will be required to make this work. But I guess the bitter lesson and recent history also tell us that some solutions may only emerge at compute levels beyond what is currently possible for realtime inference let alone learning. And the only example we have of entities solving Atari games is the human brain, of which we don't have a clear understanding of the compute capacity. In which case, why wouldn't it be better to focus purely on learning efficiency and relax the realtime requirement for now?
That's a genuine question by the way, definitely not an expert here and I'm sure there's a bunch of value to working within these constraints. I mean, jumping spiders solve reasonably complex problems with 100k neurons, so who knows.
I don't think it's clear how much of a human brains function exists at birth though, I know it's theorised than even much of the sensory processing has to be learned.
Existing at birth is not the same thing as innate. Puberty is innate but it is not present at birth.
"A reality check for people that think full embodied AGI is right around the corner is to ask your dancing humanoid robot to pick up a joystick and learn how to play an obscure video game."
Games generally are solvable for AI because they have feedback loops and a clear success or failure criteria. If the "picking up a Joystick" part is the limiting factor, sure. But why would we want robots to use an interface (especially a modern controller) heavily optimized for human hands; that seems like the definition of a horseless carriage.
I'm sure if you compared a monkey and a dolphins performance using a joystick you'd get results that aren't really correlated with their intelligence. I would guess that if you gave robots an R2D2 like port to jack into and play a game, that problem could be solved relatively quickly.
They also claimed it "learned" to play by playing itself only however it was clear that most of the advanced techniques were borrowed from existing AI and by observing humans.
No surprise they gave up on that project completely and I doubt they'll ever engage in anything like that again.
Money better spent on different marketing platforms.
AI clearly isn't at human level and it's OK to admit it.
Neurons have finite (very low) speed of signal transfer, so just by measuring cognitive reaction time we can deduce upper bounds on how many _consecutive_ neuron connections are involved in reception, cognitive processing, and resulting reaction via muscles, even for very complex cognitive processes. And the number is just around 100 consecutive neurons involved one after another. So “the algorithm” could not be _that_ complex in the end (100x matmul+tanh?)
Granted, a lot of parallelism and feedback loops are involved, but overall it gives me (and many others) an impression that when the AGI algorithm is ever found, it’s “mini” version should be able to run on modest 2025 hardware in real time.
Biological neurons are way more complex than that. A single neuron has dentritic trees with subunits doing their own local computations. There are temporal dynamics in the firing sequences. There is so much more complexity in the biological networks. It's not comparable.
Using real life robots is going to be a huge bottleneck for training hours no matter what they do.
And I say this while most certainly not being as knowledgeable as this openai insider. So it even I can see this, then it's kinda bad, isn't it?
>> "they will learn the same lesson I did"
But don't get me wrong! Since this is a long-term research endeavor of his, I believe really starting from the basics is good for him and will empower him to bring something new to the table eventually.
I'm surprised though that he "only" came so far as of now. Maybe my slight idolization of Carmack made me kinda of blind to the fact that this kind of research is a mean beast after all and there is a reason that huuuuge research labs dump countless of man-decades into this kind of stuff with no guaranteed breakthroughs.
I'm nowhere as good at my craft as someone who works for openai, which the author of that tweet seems to be, but if even I can see this, then it's bad, isn't it?
Why not tackle robotics if anything. Or really just be the best AGI and everyone will be knocking on your door to license it in their hardware/software stacks, you will print infinite money.
kamranjon•6h ago
johnb231•5h ago
https://github.com/Farama-Foundation/Arcade-Learning-Environ...
The goal is to develop algorithms that generalize to other tasks.
sigmoid10•4h ago
[1]https://github.com/cshenton/atari-leaderboard
mschuster91•4h ago
Continuous training is the key ingredient. Humans can use existing knowledge and apply it to new scenarios, and so can most AI. But AI cannot permanently remember the result of its actions in the real world, and so its body of knowledge cannot expand.
Take a toddler and an oven. The toddler has no concept of what an oven is other than maybe that it smells nice. The toddler will touch the oven, notice that it experiences pain (because the oven is hot) and learn that oven = danger. Place a current AI in a droid toddler body? It will never learn and keep touching the oven as soon as the information of "oven = danger" is out of the context window.
For some cases this inability to learn is actually desirable. You don't want anyone and everyone to be able to train ChatGPT unsupervised, otherwise you get 4chan flooding it with offensive crap like they did to Tay [1], but for AI that physically interacts with the meatspace, constant evaluation and learning is all but mandatory if it is to safely interact with its surroundings. "Dumb" robots run regular calibration cycles for their limbs to make sure they are still aligned to compensate for random deviations, and so will AI robots.
[1] https://en.wikipedia.org/wiki/Tay_(chatbot)
sigmoid10•4h ago
mschuster91•2h ago
It is, at least if you wish to be in the meatspace, that's my point. Every day has 86400 seconds during which a human brain constantly adapts to and learns from external input - either directly as it's being awake or indirectly during nighttime cleanup processes.
On top of that, humans have built-in filters for training. Basically, we see some drunkard shouting about the Hollow Earth on the sidewalk... our brain knows that this is a drunkard and that Hollow Earth is absolutely crackpot material, so if it stores anything at all then the fact that there is a drunkard on that street and one might take another route next time, but the drunkard's rambling is forgotten maybe five minutes later.
AI, in contrast, needs to be hand-held by humans during training that annotate, "grade" or weigh information during the compilation of the training dataset, in order that the AI knows what is written in "Mein Kampf" so it can answer questions upon it, but that it also knows (or at least: won't openly regurgitate) that the solution to economic problems isn't to just deport Jews.
And huge context windows aren't the answer either. My wife says me, she would like to have a fruit cake for her next birthday. I'll probably remember that piece of information (or at the very least I'll write it down)... but an AI butler? I'd be really surprised if this is still in its context space in a year, and even if it is, I would not be surprised if it weren't able to recall that fact.
And the final thing is prompts... also not the answer. We've seen it just a few days ago with Grok - someone messed with the system prompt so it randomly interjected "white genocide" claims into completely unrelated conversation [1] despite hopefully being trained on a ... more civilised dataset, and to the contrary, we've also seen Grok reply to Twitter questions in a way that suggest that it is aware its training data is biased.
[1] https://www.reuters.com/business/musks-xai-updates-grok-chat...
sigmoid10•2h ago
That's not even remotely true. At least not in the sense that it is for context in transformer models. Or can you tell me all the visual and auditory inputs you experienced yesterday at the 45232nd second? You only learn permanently and effectively from particular stimulation coupled with surprise. That has a sample rate which is orders of magnitude lower. And it's exactly the kind of sampling that can be replicated with a run-of-the-mill persistent memory system for an LLM. I would wager that you could fit most people's core experiences and memories that they can randomly access at any moment into a 1000 page book - something that fits well into state of the art context windows. For deeper more detailed things you can always fall back to another system.
vectorisedkzk•2h ago
sigmoid10•2h ago
mr_toad•47m ago
aatd86•4h ago
This is a continuous process.
epolanski•2h ago
Doesn't the article states that this is not true? AI cannot apply to B what it learned about A.
mschuster91•1h ago
epolanski•38m ago
That's essentially what we're looking for when we talk about general intelligence, the capability to adapting what we know to what we know nothing about.
newsclues•2h ago
gregdeon•36m ago
I think we could eventually saturate Atari, but for now it looks like it's still a good source of problems that are just out of reach of current methods.
cryptoz•4h ago
modeless•4h ago
If it is substantially more sample efficient, or generalizable, than prior work then that would be exciting. But I'm not sure if it is?
RetroTechie•2m ago
If so, scaling up may be more of a distraction rather than helpful (besides wasting resources).
I hope he succeeds in whatever he's aiming for.
tschillaci•3h ago
albertzeyer•3h ago
His goal is to develop generic methods. So you could work with more complex games or the physical world for that, as that is what you want in the end. However, his insight is, you can even modify the Atari setting to test this, e.g. to work in realtime, and the added complexity by more complex games doesn't really give you any new additional insights at this point.
gadders•1h ago