> trained at just 1/9 the cost of Qwen3.7-Plus, while outperforming it across the board
:)
It is very relevant and for a certain group of us, far more impactful to our work the next month(s) than any blog post could be.
I wonder if I could get this running through vLLM on 6x Nvidia L4 - the 3.6 worked great on 4 cards but sadly TP6 just isn’t a thing and I don’t have 8 cards available, maybe it’s gonna be okay with like TP2 and MTP. I have no idea at this time, probably need to test out what even might be possible.
This "next" release adds a new concept, first public release with n-grams, I think. And it's in a MoE size that is likely to be very fast and cheap to serve (faster than 27b for sure). It's also well suited for inference on alternative compute (i.e. sparks, macs, etc) so it's relevant to local users.
Qwen's advances do (currently) have merit.
> trained at just 1/9 the cost of Qwen3.7-Plus, while outperforming it across the board
Opus 4.6 Max self-hosted at 30 tok/s on a 5k Macbook in Aug 2026. The LLM timelines are crazy.
Self-learning/improving would be even better but that's still a long way to go.
How much memory does this translate to and what quantization (if any) were applied?
Also, heating my home during the winter is nice.
Oh, also, I use llamacpp with --reasoning-budget; very simple way to move on.
Qwen 3.8 flash: $0.16 / $0.47
Compared to
Deepseek 0723: $0.03 / $0.075
(units in USD/m tok)
Original error: llama.cpp does not support this GGUF's model architecture ('qwen4exp')
Edit : Saw the pull request, should arrive soon enough https://github.com/ggml-org/llama.cpp/pull/27742
6B active params helps around the memory bandwidth constraints, but a 128GB box can probably run the Q3/Q4 quants fairly easily with a decent context size. This might actually be better for strix users than 27B, which was already very good.
Luna is $0.20 / $1.20 vs $0.16 / $0.47 with Qwen.
whwhyb•46m ago