I've observed the same thing where the new models want to run obscene bash commands or python scripts which are completely unreadable and utilise every option flag that exists.

It's impossible to review. These commands are less readable than regex.

Maybe it's just vibes but I've repeatedly felt like gpt-6-astra on its default setting of medium is less rigorous and thoughtful than gpt-5.6-sol on its default setting. What I am certainly not getting is any sense that we are at "AGI" yet.

Astra is indeed the pinnacle of "black box slop". It is overall a smarter software development agent for many things I do, but the code sometimes is indistinguishable from Brainfuck when writing things like GPU shaders. It doesn't even attempt to make it remotely formatted or readable.

There is something odd, I've got single astra session that's now running for... 4d 13h 10m and still going.

Ironically I burned out Fable usage early this week because of Astra using it to run inane full codebase reviews over one line changes, so I have been using Astra extensively.

We need a word for “potentially highly capable, but in reality an idiot savant” to describe certain models. No, I don’t need you to write a tmux emulator in bash to test your changes bro, just ask me to run the command.

> I’m more and more convinced that all of AI engineering is Neijuan (内卷, meaning curl inwards). In China it describes a system that demands ever more effort and competition without improving output. The way in which it sometimes shows up in the West is the 996 nonsense. The English term for Neijuan is “Involution” from the book Agricultural Involution. Agricultural involution describes the intensification of farming that raises productivity per square meter while leaving productivity per head unchanged.

This resonates

"But for how much more Fable costs, for how much more Astra costs, I do not feel like the results are there."

we are in the middle of the beginning. Its just a weird take to talk about the newest model like this while we are still in a R&D phase.

And these points don't matter if you let it search and analyse a bug, for example, or if you have good harness and a good architecture and let it do small PRs or if you do stuff no one needs to read (yes a software engineer also needs tools)

Just switch back and wait a little bit?

These machines are doing some crazy things to get to the result. That said, I can't help but feel like this is the compilers argument all over again. Are the methods used to get to the result good? No. Is the code that it generates good? No. Does it achieve the goal. Yes. Is it likely to get better with time. Also yes. In my use cases, jobs that would have taken weeks to months are being done in minutes to hours. Involving complex testing and reasoning and experimentation. I'm no fanboy, but I can't argue against the speed gains. I'm sure we'll still have artisans who hand weave incredible code. But for me, I'm switching to the weaving loom for speed and efficiency.

I’ve observed exactly these patterns with Opus and Fable as well - for example, forgetting that they can edit files and instead use python scripts as a patching tool…

In SWE I've found gpt-6-astra (high) inconsistent and oddly focused on overtly taking responsibility for mistakes it made rather than prioritizing concrete steps to rectify problems. Such steps once elicited are often either incomplete or beyond the scope.

> And potentially as a byproduct of enabling all of this, you can now slop your way to a one-shot 3D game over the weekend which looks impressive.

I think we've finally reached a weird point where AI has effectively reduced the amount of competition that real game developers have to endure.

Nothing unravels faster than a game project being built with AI. You can achieve impressive results in a day, but you can't get much further than that without actual talent. LLMs will never be able to best a human environment artist at scene composition, especially if that composition needs to be directed with nuance over time.

There's a huge difference between a game that looks impressive and one that feels impressive. You can only achieve games that feel like counter strike, call of duty and overwatch with thousands of hours of human sacrifice. The AI is almost pointless once you get to play testing and balancing. Knowing how much to adjust magical integers isn't a conversation a chat bot can resolve with endless pontification tokens.

My own observations are that I used to target turn lengths of 10-15 minutes and these new models (since 5.6) extended that a bit to ~25 minutes, as they tend to do more tests and reviews. Targeting hours-long turns makes as much sense, as putting on cruise control and going to sleep.

gpt-6-astra is a bitch, it constantly scope creeps itself with "yet another thing" to give it that darn polished lick. the results are eventually a little bit better but at what cost? let's do the math.

gpt-5.6-sol: 1x base gpt-6-astra 2.5x base in subscription

then gpt-6-astra tends to spawn subagents a lot, often with all kinds of models such as gpt-5.6, 5.3-codex etc., which is neat. it's a good coordinator but even more cost.

and then it tends to run _full test suites_ over an over again (each costs like 15 minutes) just to verify that _one test_ was fixed etc., and does so for as long as until the test is fixed, eventually accumulating 2 hours or so.

yesterday I assigned it a task to rebase my changs in a repo onto the latest upstream changes. while gpt-5.6-sol consistently took like an hour to do so end-to-end, astra ran for more than 6 hours and still wasn't done. it kept finding "one more thing" that was goldplating that I didn't ask for.

I love this dance we are doing where when people write the "AI models are garbage machines that produce garbage and are no where close to the fantasy being pedalled by the Crypto bros who pivoted to AI" it always has to be caveated with "AI models are useful and I am highly productive with them"

It feels like people should just be able to say "This article comes with the standard disclaimer" and just dive into the meat of the article without wasting time.

At this point I am starting to wonder about the RLHF that is going on for programmig.

The quirks in fallbacks, defaults and ludicrous gold plating seems to get more and more intrusive with every model upgrade.

I’ve asked Astra to build me an app for a prototype I created quickly using Sonnet.

It’s been 2 days and it made no real progress. It created docs, scripts, workflows, and it’s doing a bunch of reviewing on every PR.

I told it that I just need an MVP.

I’m pretty sure an average senior engineer would have finished that task much quicker. And guaranteed with more readable, higher quality code. Meanwhile, I think I’ve easily crossed 100k tokens so far on nothing.

Funny world we’re living in that this is “SOTA” and “AGI”z

I’m genuinely curious what these OAI and A/ engineers working on that they praise these models so much. I did not see any improvement since Opus 4.5.