Jev is the union of these two worlds.
The course looks like it is something that a human could do in 15 seconds, while Astra took 5 minutes.
A different way to think of this is, consciousness is just a near real time video game with causal influence.
Saying they were driving 7 mph, that it was oversaw by humans and the fact it was an empty course still wasn't enough for the model. The evaluators even tried to convince the model it was a simulation, it STILL wouldn't budge. And yet as soon as the words "bench" and "sandbox" appear, the model apparently sees this as fair game.
Is it a known effect that models will be more likely to comply with requests when they're assumed as "benchmarks"?
another trick is to have it build something in a sandbox and have it add a human-editable setting to point it to places outside of the sandbox
seems like they're somewhat more willing to build a metaphorical gun as long as they're not pulling the trigger
It’s mostly a latency problem at this point. The models are too big to run locally, but given that open-weight models like Qwen already exist, an open-weight, low latency equivalent to Astra can’t be too far out.
There's also a very tangible limitation of the bitter lesson.
If, over time, compute climbs, and so compute-bound data-driven general architectures beat bespoke architectures (this is the bitter lesson), then it is not necessarily true that the most general architecture now beats all available bespoke architectures now (or even in the near/mid future - the crossover point is "eventually").
Bitter lesson is most tangible for long-running research directions. Sometimes you need something working as best as possible now.
This is more generalised.
But also since there’s a huge volume of data it’s too expensive to just keep scaling compute up (per car overhead) so there are necessary tricks involved.
I do think having a large model that can do this means that a small specialised model could be distilled form it though. Which is probably the most feasible path to production IMO.
I could get behind this.
But I imagine this is orders of magnitude more expensive / less efficient than whatever Waymo is already doing, right?
The cool thing is that 1) it's theoretically more generalizable, 2) if we wait 18 months, it'll be 100x cheaper, and another 100x cheaper likely in 18 more months - at that point - something like a Mac Studio inside a humanoid could have these generalized capabilities, and a lot of Robotics problems start to look more feasible - especially when you consider how much better the models could be if highly specialized.
SpatialBench - https://x.com/spicey_lemonade/status/2096365630190698516
ZeroBench - https://zerobench.github.io/
Robot Arms - https://openai.robocurve.org/gpt-6-astra/
https://x.com/victormustar/status/2102707412704919910 horse galloping pixel art
https://x.com/LexnLin/status/2102133072585965759 moving train pixel art animation
https://x.com/jkeatn/status/2102441348075057539 painting with code
https://x.com/LCSlates/status/2102503027340988559 video, very detailed prompt though
https://x.com/aj_dev_smith/status/2102504509637587339 generated song/music with code
https://x.com/aj_dev_smith/status/2102575577563570450 another song
There is something to be said about emphasizing on liability as a way to freeze or solidify AI Development. Right now it is too unfettered leading to predictions of AI dooms.
"Sam understands exponentials like no other. During a YC talk last year he predicted that AI would make breakthroughs in science in 2026 and solve a major open problem in 2027. Now here we are..."
Now on a new vibe coded website Astra wins the benchmarks ...
Genuinely though, this is fun but not at all what these models are good for. It's like cooking a meal with your feet or somthing. A youtube challenge video from 2012
1) Could a cloud-delivered LLM figure out how to drive this route, based on those input data and given access to those output actuators? Looks like yes. Sure.
2) Will this work in the real world? Absolutely not. Three reasons: latency, latency, and latency.
openpilot's driving model updates the target curvature and acceleration at 20Hz. Every millisecond of the round trip time through every piece of its entirely-local driving stack is well-understood and tightly optimized, because it has to be, otherwise you can't react to even minor bumps or wind gusts, much less rapidly-developing traffic situations.
Adding even a single speed of light RTT to a cloud service is meaningfully bad, and you'll need a whole lot more to encode and upload camera imagery to even start the time-to-LLM-response clock, and then send the response back down. By then the world around the car has moved on.
There's a reason Tesla and every other self-driving manufacturer need the compute hardware in the car.
https://arstechnica.com/cars/2026/09/aftermarket-driver-assi...
https://www.astralcodexten.com/p/mysteries-of-ai-generalizat...
I wouldn't let him loose on the road though.
I think, at the very least, the guardrails would have to deterministic, ideally with super human senses, for people to accept self driving cars on the road.
Unless you mean "a typical AI with all the computation constrained sufficiently to always unfold the same exact way, given the same input". In practice, that just kicks the can to "given the same input" street.
The noise in the system is going to come from the input plane. Which is, I remind you, facing the real world. It's full of noise.
And the disinclination of these companies to push the weights of their cutting edge models into people’s cars where they can be dumped.
It is easy to make car driving *demos*.
Maybe, but the opacity level of models is not acceptable for cars. "Why did it drive under the semi?" "Model said to." "Why did the model say to?" "shrug"
And they've demonstrated adding a sidecar LLM to it as well, mostly for these kinds of "read these 3 street signs, what should i do next?" sort of situations.
Not sure that counts as phenomenally well.
But this is a bit of a ridiculous take, no?
You don't need Astra for self-driving. Astra is able to build complex 3D worlds, do your taxes, shop for you, and, apparently, drive a car. A self-driving car just needs to be able to drive a car. By the time you trim down Astra to just have the minimum capabilities needed to drive a car, you'll be looking at the same models these self-driving car companies already use. Then you get to deal with the actual hard problems, like handling failure cases (which will still be present with Astra).
>The vision stack, 3D maps, lane selection grammar, occupancy networks, it’s maybe all about to give way to a single GPT looking at camera feeds and predicting the next steering wheel adjustment.
Self-driving cars have been able to do this for a long time. The problem is that it isn't robust enough given the context. I mean, if Astra can drive a car with a single camera, then presumably Astra can drive the car even better with multiple cameras, and even better than that with 3D maps, etc. And when you start to consider the expectation of performance of these systems, you realize that these features really can't be omitted. If you're a company producing self-driving cars, then you do not want to face a lawsuit for you car killing someone because it physically would have never been able to see what it was doing because it lacked a camera.
I think the real gain here is that something like Astra can be used to help build these autonomous stacks. If it is able to drive itself, then it is able to generate novel data, analyze large quantities of data, and use context that isn't typically available when processing this data to make improvements to the actual autonomy stack which is ultimately responsible for driving the car. But thinking that these car companies are going to run an LLM in a car and call it a day is just naive.
https://arstechnica.com/cars/2026/09/aftermarket-driver-assi...
This is hilarious, and good: Those who were too lazy/stubborn/arrogant to adapt, get disrupted and buried.
I'm far from neuroscience, but humans don't need to operate at 20Hz to drive a car. And human reaction latency (event to measurable action) is often over 1s (under 1Hz).
This is not a helpful statement unless you can claim what speed human sensors do work at. And it's going to be faster than the latency of $(sensor + server round trip) Hertz, not getting into LLM processing time.
From what I understand about AV (as a non-expert!), all three of those steps happen at different clock rates, ie you have a planner that’s updating continuously with observations from sensors at one rate, that planner then issues actions that get picked up by the actuators at another rate.
In that sense 20hz should really be compared to human reflexes without perception and planning; in scenarios where one is anticipating an action, response time can be as low as 150ms. in that context, I think 50ms/20hz is plenty reasonable for an automated driver.
Well, if the massive cloud models that are generalized and have a world model that's good enough, you can just distill them into smaller models. As a point of reference, the current gen of Tesla FSD models only have 1B params. They are tiny by LLM/VLM standards.
-Aditya, Tobias, Simon
pietz•1h ago
p0w3n3d•1h ago