Biological brains are extremely good continual learners. They don't need to be trained ahead of time at all for basic motor skills. They'll learn on their own and sharing information merely speeds the process up.
There is no such thing as a data problem. The scaling paradigm isn't obsolete because it doesn't work, it's obsolete because it appears to be wholly unnecessary.
Think about it, why the hell would it be necessary for every single person wanting a robot butler in the house to take a video of their kitchen and then blend it with tens of thousands of other kitchens, just so the robot knows your specific kitchen in and out? The other kitchens are irrelevant to you. You don't care about them.
That aside, arguments from handwaving about biology are extremely weak - this is the same proof structure used to explain that because we only have two eyes, a machine only needs 2 cameras to drive a car safely. It wants you to forget everything that happens after the light hits the retina, and difference between an always-wet lens that's cleared every second or two and a camera. i.e. it's a noble goal to have some model that never saw a kitchen also be an expert chef, but, there's no logical reason to claim that'd ever be the case.
"Tesla in particular has a fleet of over 100,000 cars deployed in the real world that it is constantly collecting and then annotating data from. These cars are being teleoperated by experts, making the data ideal for large-scale supervised learning."
I'm not affiatled with them.
mohsen1•7mo ago
margalabargala•7mo ago
It's in a similar vein to how "you can prove that a single layer neural network exists that does the same thing as a combination of many neural networks", something that led to a lot of people focusing on single level NNs for "purity's sake" and causing an AI winter.
Like yeah, maybe you don't "need" sensor and motor data. Especially if you build what you're calling a "good" model.
But making a "good" model that gets results might be near impossible and building a "less good" model that does use sensor data, and performs far better on real tasks, might be way easier for us mere mortals to do.
yorwba•7mo ago