Articles such as this really make me wish that the current generation of LLMs were more often described as workable implementations of "the infinite monkey theorem" --- the descriptions/images in this article are esp. worth showing to aperson to whom one is trying to describe how an AI model "creates" an image.

Short version: A Qwen-2.5 7b model that has been turned into a diffusion model.

A couple notable things: first is that you can do this at all, (left to right model -> out of order diffusion via finetuning) which is really interesting. Second, the final version beats original by a small margin on some benchmarks. Third is that it’s in the ballpark of Gemini diffusion, although not competitive — to be expected for any 7B parameter model.

A diffusion model comes with a lot of benefits in terms of parallelization and therefore speed; to my mind the architecture is a better fit for coding than strict left to right generation.

Overall, interesting. At some point these local models will get good enough for ‘real work’ and they will be slotted in at API providers rapidly. Apple’s game is on-device; I think we’ll see descendants of these start shipping with Xcode in the next year as just part of the coding experience.

Does diffusion allow for 'size editing'? Unsure how to ask this, or if this (most likely) reveals a fundamental misunderstanding of my own, but: for an image, the size is set (say, 256x256.) For text, if each token were a pixel, it's very small. The article image showed text colour-coded by generation order. What if it would need to, say, insert another line for the rest of a comment sentence? How would it even know the size upfront, the way an image size is known?

Here's the paper that they wrote: https://arxiv.org/pdf/2506.20639

It's notable that this was an intern project.

Jetbrains has 100MB models of languages for their IDEs that can auto complete single lines. It's good but I think we can do better for local code auto complete. I hope Apple succeeds in their on device AI attempts.

Points I thought were interesting:

> Apple’s model is built on top of Qwen2.5‑7B, an open-source foundation model from Alibaba. Alibaba first fine-tuned that model for better code generation (as Qwen2.5‑Coder‑7B), then Apple took it and made its own adjustments.

> it still doesn’t quite reach the level of GPT-4 or Gemini Diffusion.

Looks like this is targeted more at "better autocomplete" than "powers your next agent." Makes sense given Apple's interest in the on-device experience.

I often write out of order so it’s interesting to me that we have a model that can do so as well.

> The result is faster code generation, at a performance that rivals top open-source coding models

So even though it is faster (than what?) it still doesn’t beat top models?

If we're getting almost-comparable results from models that can run locally (especially on dedicated silicon that we know Apple are good at designing and shipping at scale), I think we might hit a turning point soon. This was an intern project, so hopefully they'll now push a bit more resource at it: a 7b param model on my laptop that is getting near Gemini or Claude standards is going to win my money, every day.

On a purely tech point, I'm not working very near the cutting edge on AI research, but hadn't realised so much had been done - and was possible - with diffusion models for text. Will need to dig into that, as it looks fascinating.

The model itself I'm actually less fussed about. The integration with the ide and the ability to easily get the model the right context is what I struggle with (vscode is passable with the recent 101 update). I find pretty much all models in copilot offer similar-ish performance.

I've been looking for good/best-practice workflow setups to work with docker python/fastapi backends and vue frontends...but I haven't found much.

If anyone has tips for where to look, I'd genuinely appreciate it!

Zed are doing out-of-order edits with their model, I'm not sure what is new here. I strongly suspect that theirs works directly with the CRDT that the editor uses, because it's able to continue similar deletes for the user (deletes would otherwise be invisible to most autocomplete models).

Apple's is open weights, so that's a big deal.

Would love to try this in ollama/llama.cpp. Using llama.cpp for VsCode is painful since (realistically) I can only generate on the order of <100 tokens at a time