6 months behind is probably a bit rich, I find the frontier models more capable in sustained agent and compiler tasks. Simple tool-chains and basic reasoning seems more or less on par across the board, at least for my use cases.

I have been experimenting with dissassembly and rewrite of 90s dos games in Rust.

It's not something I have experience in so I rely heavily on the models setting the direction and automating the work.

It's something where the level of intelligence has a huge impact. I still haven't succeeded, it's mostly an experiment for me to test the edges of model capabilities and see how they tackle such a hard problem.

https://squishy.franzai.com/ - fable 5

even with opus 4.8 it could not come up with the necessary algos to reliable create harder and harder levels - in the now famous big fable 5 gap of 2026 opus even screwed up the algo a few times and needed to recover it from git.

I feel like the converse question is more relevant: what projects have open source agents been good enough to implement on their own?

It all depends on how hands on you are willing to be.

If you are using the model to write to code faster with extensive human oversight you can develop a lot faster using the non-frontier models. I was doing that extensively last summer.

But now my thought process is I want to focus on architecture and product direction. I have not seen Sonnet level models be capable of performing autonomously enough to take a feature end to end reliably enough to be completely hands off. In fact there a many cases where Opus will fail as well where Fable will succeed.

Of course that is not to say that Fable will always do things correctly. It will happily take an under-specified problem statement and happily use up all of your usage to build the wrong thing, while Opus at least recently stops constantly to check in.

I was worried about some messup with taxes (my tax advisor messed up here), I managed to get it sorted on time - this was super naive but in -germany when they send you the taxes they mention the "cents" place as well in a very weird way, I assumed that was the entire number and assumed the tax issue I had was 10x larger than the amount it really was (10x and not 100x as my brain is in a place of pressure due to other circumstances and I wasn't thinking clearly)

GPT 5.6 incorrectly stated that I had nothing to do, Fable got the issue correctly and I was able to see that that was indeed the cents place and that I was more worried than I realized, and I managed to get a temporary solution setup (that I verified and I am sure is correct).

Which is to say it managed to relieve me of quite a bit of stress haha

I don't bother to use open-weight models, but for all of the "security" or "security adjacent" work I have tried recently, GPT 5.5 x-high has been the first model that produced useful output.

Even very old models could spot the most glaring issues, but it's a different story if you scan a source repository where humans can't find security vulnerabilities even after hours of reading through the code. Feed something like that to, say, Gemini Pro 3.1 and you'll get a bunch of false positives back, nit-picking, or variants of "this could be insecure if the code around it changes in unreasonable ways in the future".

Feed the same thing into GPT 5.5 x-high and then tens of minutes later it'll find half a dozen unauthenticated remote code execution vulnerabilities, arbitrary file read/write vulnerabilities, or similar.

Until it got nerfed, Mythos was similarly a huge step up for a lot of people working on code security.

I haven't seriously used GLM/Qwen/etc for agentic work. I will say, that the GPT and Claude models from 6 months ago were very usable within their respective agentic harnesses (this matters a lot). But there is no way you could convince me to go back to GPT-5.0 from GPT-5.6 for a serious amount of agentic work.

I could probably accomplish most of the same tasks with GPT-5.0, but it would take a lot more involvement from me, more troubleshooting, and significantly more time.

Coming up with good French grammar quizzes. GPT5.6 is noticeably better.

Older models usually pitch the difficulty too hard/easy and rely on me remembering the gender of words.

I’ve not noticed a big programming improvement with the newer models.