GLM-4.7-Flash

https://huggingface.co/zai-org/GLM-4.7-Flash

195•scrlk•2h ago

Comments

epolanski•2h ago

Any cloud vendor offering this model? I would like to try it.

xena•2h ago

The model literally came out less than a couple hours ago, it's going to take people a while in order to tool it for their inference platforms.

idiliv•2h ago

Sometimes model developers coordinate with inference platforms to time releases in sync.

PhilippGille•2h ago

z.ai itself, or Novita fow now, but others will follow soon probably

https://openrouter.ai/z-ai/glm-4.7-flash/providers

epolanski•2h ago

Interesting, it costs less than a tenth than Haiku.

saratogacx•1h ago

GLM itself is quite inexpensive. A year sub to their coding plan is only $29 and works with a bunch of various tools. I use it heavily as a "I don't want to spend my anthropic credits" day-to-day model (mostly using Crush)

dvs13•2h ago

https://huggingface.co/inference/models?model=zai-org%2FGLM-... :)

latchkey•1h ago

We don't have lot of GPUs available right now, but it is not crazy hard to get it running on our MI300x. Depending on your quant, you probably want a 4x.

ssh admin.hotaisle.app

Yes, this should be made easier to just get a VM with it pre-installed. Working on that.

omneity•1h ago

Unless using docker, if vllm is not provided and built against ROCm dependencies it’s going to be time consuming.

It took me quite some time to figure the magic combination of versions and commits, and to build each dependency successfully to run on an MI325x.

latchkey•1h ago

Agreed, the OOB experience kind of suck.

Here is the magic (assuming a 4x)...

  docker run -it --rm \
  --pull=always \
  --ipc=host \
  --network=host \
  --privileged \
  --cap-add=CAP_SYS_ADMIN \
  --device=/dev/kfd \
  --device=/dev/dri \
  --device=/dev/mem \
  --group-add render \
  --cap-add=SYS_PTRACE \
  --security-opt seccomp=unconfined \
  -v /home/hotaisle:/mnt/data \
  -v /root/.cache:/mnt/model \
  rocm/vllm-dev:nightly
  
  mv /root/.cache /root/.cache.foo
  ln -s /mnt/model /root/.cache
  
  VLLM_ROCM_USE_AITER=1 vllm serve zai-org/GLM-4.7-FP8 \
  --tensor-parallel-size 4 \
  --kv-cache-dtype fp8 \
  --quantization fp8 \
  --enable-auto-tool-choice \
  --tool-call-parser glm47 \
  --reasoning-parser glm45 \
  --load-format fastsafetensors \
  --enable-expert-parallel \
  --allowed-local-media-path / \
  --speculative-config.method mtp \
  --speculative-config.num_speculative_tokens 1 \
  --mm-encoder-tp-mode data

karmakaze•2h ago

Not much info than being a 31B model. Here's info on GLM-4.7[0] in general.

I suppose Flash is merely a distillation of that. Filed under mildly interesting for now.

[0] https://z.ai/blog/glm-4.7

lordofgibbons•1h ago

How interesting it is depends purely on your use-case. For me this is the perfect size for running fine-tuning experiments.

redrove•1h ago

A3.9B MoE apparently

XCSme•2h ago

Seems to be marginally better than gpt-20b, but this is 30b?

strangescript•2h ago

I find gpt-oss 20b very benchmaxxed and as soon as a solution isn't clear it will hallucinate.

blurbleblurble•1h ago

Every time I've tried to actually use gpt-oss 20b it's just gotten stuck in weird feedback loops reminiscent of the time when HAL got shut down back in the year 2001. And these are very simple tests e.g. I try and get it to check today's date from the time tool to get more recent search results from the arxiv tool.

lostmsu•1h ago

It actually seems worse. gpt-20b is only 11 GB because it is prequantized in mxfp4. GLM-4.7-Flash is 62 GB. In that sense GLM is closer to and actually is slightly larger than gpt-120b which is 59 GB.

Also, according to the gpt-oss model card 20b is 60.7 (GLM claims they got 34 for that model) and 120b is 62.7 on SWE-Bench Verified vs GLM reports 59.7

vessenes•2h ago

Looks like solid incremental improvements. The UI oneshot demos are a big improvement over 4.6. Open models continue to lag roughly a year on benchmarks; pretty exciting over the long term. As always, GLM is really big - 355B parameters with 31B active, so it’s a tough one to self-host. It’s a good candidate for a cerebras endpoint in my mind - getting sonnet 4.x (x<5) quality with ultra low latency seems appealing.

mckirk•1h ago

Note that this is the Flash variant, which is only 31B parameters in total.

And yet, in terms of coding performance (at least as measured by SWE-Bench Verified), it seems to be roughly on par with o3/GPT-5 mini, which would be pretty impressive if it translated to real-world usage, for something you can realistically run at home.

HumanOstrich•1h ago

I tried Cerebras with GLM-4.7 (not Flash) yesterday using paid API credits ($10). They have rate limits per-minute and it counts cached tokens against it so you'll get limited in the first few seconds of every minute, then you have to wait the rest of the minute. So they're "fast" at 1000 tok/sec - but not really for practical usage. You effectively get <50 tok/sec with rate limits and being penalized for cached tokens.

They also charge full price for the same cached tokens on every request/response, so I burned through $4 for 1 relatively simple coding task - would've cost <$0.50 using GPT-5.2-Codex or any other model besides Opus and maybe Sonnet that supports caching. And it would've been much faster.

twalla•44m ago

I hope cerebras figures out a way to be worth the premium - seeing two pages of written content output in the literal blink of an eye is magical.

behnamoh•49m ago

> The UI oneshot demos are a big improvement over 4.6.

This is a terrible "test" of model quality. All these models fail when your UI is out of distribution; Codex gets close but still fails.

ttoinou•43m ago

Sonnet was already very good a year ago, do open weights model right are as good ?

jasonjmcghee•39m ago

Fwiw Sonnet 4.5 is very far ahead of where sonnet was a year ago

Workaccount2•40m ago

Unless one of the open model labs has a breakthrough, they will always lag. Their main trick is distilling the SOTA models.

People talk about these models like they are "catching up", they don't see that they are just trailers hooked up to a truck, pulling them along.

twelvechess•2h ago

Excited to test this out. We need a SOTA 8B model bad though!

cipehr•1h ago

Is essentialai/rnj-1 not the latest attempt at that?

https://huggingface.co/EssentialAI/rnj-1

piyh•21m ago

https://docs.mistral.ai/models/ministral-3-8b-25-12

dfajgljsldkjag•1h ago

Interesting they are releasing a tiny (30B) variant, unlike the 4.5-air distill which was 106B parameters. It must be competing with gpt mini and nano models, which personally I have found to be pretty weak. But this could be perfect for local LLM use cases.

In my ime small tier models are good for simple tasks like translation and trivia answering, but are useless for anything more complex. 70B class and above is where models really start to shine.

dajonker•1h ago

Great, I've been experimenting with OpenCode and running local 30B-A3B models on llama.cpp (4 bit) on a 32 GB GPU so there's plenty of VRAM left for 128k context. So far Qwen3-coder gives the me best results. Nemotron 3 Nano is supposed to benchmark better but it doesn't really show for the kind of work I throw at it, mostly "write tests for this and that method which are not covered yet". Will give this a try once someone has quantized it in ~4 bit GGUF.

Codex is notably higher quality but also has me waiting forever. Hopefully these small models get better and better, not just at benchmarks.

latchkey•1h ago

https://huggingface.co/unsloth/GLM-4.7-GGUF

This user has also done a bunch of good quants:

https://huggingface.co/0xSero

dajonker•1h ago

Yes I usually run Unsloth models, however you are linking to the big model now (355B-A32B), which I can't run on my consumer hardware.

The flash model in this thread is more than 10x smaller (30B).

latchkey•1h ago

There are a bunch of 4bit quants in the GGUF link and the 0xSero has some smaller stuff too. Might still be too big and you'll need to ungpu poor yourself.

disiplus•1h ago

yeah there is no way to run 4.7 on a 32g vram this flash is something that im also waiting to try later tonight

a_e_k•29m ago

When the Unsloth quant of the flash model does appear, it should show up as unsloth/... on this page:

https://huggingface.co/models?other=base_model:quantized:zai...

Probably as:

https://huggingface.co/unsloth/GLM-4.7-Flash-GGUF

homarp•21m ago

it'a a new architecture. Not yet implemented in llama.cpp

issue to follow: https://github.com/ggml-org/llama.cpp/issues/18931

dumbmrblah•21m ago

One thing to consider is that this version is a new architecture, so it’ll take time for Llama CPP to get updated. Similar to how it was with Qwen Next.

WanderPanda•40m ago

I find it hard to trust post training quantizations. Why don't they run benchmarks to see the degradation in performance? It sketches me out because it should be the easiest thing to automatically run a suite of benchmarks

behnamoh•50m ago

> Codex is notably higher quality but also has me waiting forever.

And while it usually leads to higher quality output, sometimes it doesn't, and I'm left with a bs AI slop that would have taken Opus just a couple of minutes to generate anyway.

eurekin•1h ago

I'm trying to run it, but getting odd errors. Has anybody managed to run it locally and can share the command?

bilsbie•1h ago

What’s the significance of this for someone out of the loop?

epolanski•1h ago

You can run gpt 5 mini level ai on your MacBook with 32 gb ram.

You can get LLM as a service for cheaper.

E.g. This model costs less than a tenth of Haiku 4.5.

baranmelik•55m ago

For anyone who’s already running this locally: what’s the simplest setup right now (tooling + quant format)? If you have a working command, would love to see it.

pixelmelt•29m ago

I would look into running a 4 bit quant using llama cpp (or any of its wrappers)

esafak•37m ago

When I want fast I reach for Gemini, or Cerebras: https://www.cerebras.ai/blog/glm-4-7

GLM 4.7 is good enough to be a daily driver but it does frustrate me at times with poor instruction following.

montroser•36m ago

This is their blurb about the release:

    We’ve launched GLM-4.7-Flash, a lightweight and efficient model designed as the free-tier version of GLM-4.7, delivering strong performance across coding, reasoning, and generative tasks with low latency and high throughput.

    The update brings competitive coding capabilities at its scale, offering best-in-class general abilities in writing, translation, long-form content, role play, and aesthetic outputs for high-frequency and real-time use cases.

https://docs.z.ai/release-notes/new-released

pixelmelt•29m ago

I'm glad they're still releasing models dispite going public

montroser•22m ago

> SWE-bench Verified 59.2

This seems pretty darn good for a 30B model. That's significantly better than the full Qwen3-Coder 480B model at 55.4.

infocollector•14m ago

Maybe someone here has tackled this before. I’m trying to connect Antigravity or Cursor with GLM/Qwen coding models, but haven’t had any luck so far. I can easily run Open-WebUI + LLaMA on my 5090 Ubuntu box without issues. However, when I try to point Antigravity or Cursor to those models, they don’t seem to recognize or access them. Has anyone successfully set this up?

CSS Web Components for marketing sites (2024)

The Microstructure of Wealth Transfer in Prediction Markets

Apple testing new App Store design that blurs the line between ads and results

GLM-4.7-Flash

Show HN: Pipenet – A Modern Alternative to Localtunnel

Iterative image reconstruction using random cubic bézier strokes

Folding NASA Experience into an Origamist's Toolkit

A decentralized peer-to-peer messaging application that operates over Bluetooth

Radboud University selects Fairphone as standard smartphone for employees

Luxury Yacht is a desktop app for managing Kubernetes clusters

Cows Can Use Sophisticated Tools

Dead Internet Theory

Robust Conditional 3D Shape Generation from Casual Captures

Ask HN: COBOL devs, how are AI coding affecting your work?

Raccoons break into liquor stores, scale skyscrapers and pick locks

Nepal's Mountainside Teahouses Elevate the Experience for Trekkers

"Anyone else out there vibe circuit-building?"

Flux 2 Klein pure C inference

The Code-Only Agent

Two Concepts of Intelligence

I set all 376 Vim options and I'm still a fool

Self Sanitizing Door Handle

MTOTP: Wouldn't it be nice if you were the 2FA device?

Nvidia contacted Anna's Archive to access books

Provide agents with automated feedback

Fluid Gears Rotate Without Teeth

What it's like to be banned from the US for fighting online hate

AVX-512: First Impressions on Performance and Programmability

Wikipedia: WikiProject AI Cleanup

American importers and consumers bear the cost of 2025 tariffs: analysis

GLM-4.7-Flash

Comments

CSS Web Components for marketing sites (2024)

The Microstructure of Wealth Transfer in Prediction Markets

Apple testing new App Store design that blurs the line between ads and results

GLM-4.7-Flash

Show HN: Pipenet – A Modern Alternative to Localtunnel

Iterative image reconstruction using random cubic bézier strokes

Folding NASA Experience into an Origamist's Toolkit

A decentralized peer-to-peer messaging application that operates over Bluetooth

Radboud University selects Fairphone as standard smartphone for employees

Luxury Yacht is a desktop app for managing Kubernetes clusters

Cows Can Use Sophisticated Tools

Dead Internet Theory

Robust Conditional 3D Shape Generation from Casual Captures

Ask HN: COBOL devs, how are AI coding affecting your work?

Raccoons break into liquor stores, scale skyscrapers and pick locks

Nepal's Mountainside Teahouses Elevate the Experience for Trekkers

"Anyone else out there vibe circuit-building?"

Flux 2 Klein pure C inference

The Code-Only Agent

Two Concepts of Intelligence

I set all 376 Vim options and I'm still a fool

Self Sanitizing Door Handle

MTOTP: Wouldn't it be nice if you were the 2FA device?

Nvidia contacted Anna's Archive to access books

Provide agents with automated feedback

Fluid Gears Rotate Without Teeth

What it's like to be banned from the US for fighting online hate

AVX-512: First Impressions on Performance and Programmability

Wikipedia: WikiProject AI Cleanup

American importers and consumers bear the cost of 2025 tariffs: analysis