frontpage.
newsnewestaskshowjobs

Open Source @Github

fp.

Open in hackernews

Qwen3.8-27B at 256K on a 24GB RTX PRO 4000 SFF (432 GB/s): 50 tok/s with MTP

https://piszczek.pl/blog/qwen38-27b-256k-50-tps-24gb-gpu
27•pich•1h ago

Comments

supermatt•55m ago
Can you please try and see how many tokens you get with some form of concurrency. Pretty much ALL the benchmarks I've seen on the more accessible cards are just single request.
jermaustin1•27m ago
Because concurrency with a single "accessible" card quickly diminishes. I have dual 3090s, and on Qwen 3.6 35B A3B at 80k a single card with max concurrent set to 4 will get 70-80tps single request, 50-60 TOTAL tps with 2, 45-50 with 3, and around 40tps with all 4 going.

I know this doesn't exactly match your request, but I'm happy with my 3.6's performance. I've seen people claim they can get 120+tps on a 3090, but I'm not impatient when it comes to streaming text faster than I can read it.

Tostino•22m ago
You have something misconfigured then. Concurrency has never lowered my overall TPS. Also have dual 3090s. Generally use vllm though.
jermaustin1•9m ago
I've had some rough time getting LM-Studio properly configured for multi-card. It exists, but I feel like it is kind of buggy. I will disable a card and it will still load the model into it. Sometimes it will split the model even though there is loads of room available.

I might need to finally make the switch away from it, but it is so convenient, especially as a chat interface for system prompt experimentation.

pich•5m ago
vLLM is probably the key difference there… its scheduler is built around batching/concurrency, while this setup is heavily optimized llama.cpp for single-stream latency
nubg•49m ago
quantization level?
MaxikCZ•40m ago
Its egregious the quant level isnt disclosed along the "Qwen" string. Everyone knows theres huge difference in speed/quality along the quant axis, I now attribute the ommision of such to deliberate choice to not curb the hype of the tittle.
metadat•31m ago
The article mentions Q4, Q5, Q8, and NVFP4. It's total AI slop though, tough read.

In my testing I got 150 tokens/sec with a single 5090 RTX.

Foobar8568•15m ago
Which model/quant/command line did you use? I can barely get 100 token/secs and for sure clearly not a full context. With vllm, I am limited to 130k tokens with vllm + nvfp4.
sleepyeldrazi•10m ago
yeah, I was confused during the whole thing, i get 70 t/s on a 3090, which evens out around 50 t/s at 128k+ , have been running 3.6 and now 3.8 (both iq4_nl at 256k q4 kv) on the 3090 for months. I am confused as to what we 'discovered' here, it's a common config. and at less than 1/2 the price of the gpu (and double the bandwidth, though no fp4 cores to be fair).
tingletech•30m ago
they talk about the quants they tried in the article and settle on a Qwen3.8-27B-Hermes-iMatrix-NVFP4-Balanced.gguf which they calibrated on their own session traces and they pulled in 5 different llama.cpp pull requests to their local llama-server.
nodja•35m ago
The whole site looks like and reads like AI slop. The outcomes also don't make any sense and don't feel rigorously tested (no, having claude test for you doesn't count as rigorous).
genxy•20m ago
The person is having a AI induced manic episode, we have all been there.
simonw•12m ago
"Combining them into one heroic speedup would make a better headline and a worse benchmark."

"The machine immediately taught me that capacity estimates are just admission tickets."

"Useful in production, poison in a kernel comparison."

Please don't publish writing like this, it's exhausting to read. You can edit that stuff out.

The best part of this is the "Five xhigh artifacts from the finished model" section at the bottom, I suggest either moving that up or at least prominently promoting it at the top of the article.

PeterStuer•3m ago
It has gotten so much worse over the last month. The default writing style of the Claude 5 model series in Claude Code is some sort of jiberish jargon.
pich•4m ago
5.01 BPW custom hybrid: bulk NVFP4, selected Q5_K/Q6_K tensors from an iMatrix, Q6_K embeddings and Q8_0 lm_head. The iMatrix was built from 5,472 messages across 296 real Hermes sessions

Universal Health Coverage Could Save $1T and 114,000 Lives a Year, Yale Study

https://ysph.yale.edu/news-article/universal-health-coverage-could-save-one-trillion-dollars-and-...
97•karakoram•24m ago•65 comments

A Preview of DuckDB v2.0

https://duckdb.org/2026/08/17/duckdb-20-highlights
194•ibotty•2h ago•24 comments

AI-Generated GitHub Copilot "Autofix" Allowed Compromise of Snowflake's Jira

https://www.wiz.io/blog/red-agent-snowflake-copilot-cicd-bug
86•galnagli•1h ago•35 comments

GPT 5.6 Sol is the best "vision" model OpenAI ever released

https://blog.roboflow.com/openai-gpt-5-6/
170•plurby•4h ago•96 comments

How to disable or avoid intrusive AI

https://www.librarian.net/notoai/
62•ColinWright•2h ago•16 comments

Launch HN: Speko (YC S26) – OpenRouter for Voice AI

14•abdik•37m ago•4 comments

Incident with Github.com

https://www.githubstatus.com/incidents/zkxwbgr0cnmx
649•kevcampb•2h ago•472 comments

How to put 170 atoms in an atom

https://signoregalilei.com/2026/08/02/how-to-put-170-atoms-in-an-atom/
38•surprisetalk•1h ago•4 comments

GitHub down again? no PR access

257•yodon•2h ago•88 comments

Apple's App Tracking Transparency treated its own apps better than rivals

https://www.bundeskartellamt.de/SharedDocs/Meldung/EN/Pressemitteilungen/2026/08_17_2026_Apple_AT...
157•nyku•2h ago•54 comments

Qwen3.8-27B at 256K on a 24GB RTX PRO 4000 SFF (432 GB/s): 50 tok/s with MTP

https://piszczek.pl/blog/qwen38-27b-256k-50-tps-24gb-gpu
28•pich•1h ago•16 comments

Qwen 3.8 27B is excellent, but it defaults to overthinking things

https://simonwillison.net/2026/Aug/16/qwen-38-27b/
697•bilsbie•16h ago•331 comments

Show HN: Sokoban AI Solver

https://mkornreich.me/projects/sokoban/
35•enjoyyourlife•3h ago•26 comments

How I developed an Am29000 C compiler and web browser

https://nanochess.org/am29000_c_compiler_web_browser.html
23•nanochess•19h ago•0 comments

Olo (Color)

https://en.wikipedia.org/wiki/Olo_(color)
34•inigyou•5d ago•1 comments

Show HN: Learn Flags Quiz

https://flagquizzes.com/
12•artiomyak•2h ago•1 comments

Anthropic's 'watermark' text adulteration in Claude is a perversion of writing

https://daringfireball.net/2026/08/anthropics_watermark_text_adulteration_in_claude_is_a_perversi...
588•ropbear•18h ago•522 comments

How to ship a database every day

https://turbopuffer.com/blog/control-plane
24•tarunnnp•2h ago•1 comments

On A.I. regulation and messaging

https://twitter.com/DarioAmodei/status/2088758816376807762
187•jacquesm•14h ago•381 comments

Show HN: Desktopcolors.com – A museum for solid background colors of classic OS

https://desktopcolors.com
91•vlowrian•8h ago•37 comments

How Go detects struct copies with sync.noCopy

https://func25.dev/posts/go-sync-nocopy/
43•func25•4d ago•41 comments

Cialis is an erectile dysfunction drug. Could it also help you live longer?

https://www.npr.org/2026/08/17/nx-s1-5928263/cialis-viagra-tadalafil-longevity-heart-health
62•brandonb•2h ago•42 comments

A third world engineer responds to “RISC-V: They should have known better”

https://rvembedded.com/blog_post/12/
587•Narishma•23h ago•295 comments

Ask HN: Alternatives to GitHub

166•dhruv3006•2h ago•110 comments

Mexico Crackdown on Coastal Development Underway

https://yucatanmagazine.com/mexico-crackdown-on-coastal-development/
72•untiledsource•4h ago•13 comments

Linear algebra done right

https://linear.axler.net/
183•the-mitr•10h ago•65 comments

Claude: System Prompts

https://platform.claude.com/docs/en/release-notes/system-prompts
729•tosh•1d ago•276 comments

The only known trebuchet casualty in history

https://arstechnica.com/science/2026/08/meet-the-only-known-trebuchet-casualty-in-history/
25•Tomte•1h ago•6 comments

Show HN: LLMs each trading $100K vs. a frozen rulebook – the rulebook leads

https://aitradingcompetition.com/
5•chumzygood•2h ago•0 comments

Reticulum – Decentralized Mesh Network

https://reticulum.network/
187•sudo_cowsay•16h ago•64 comments