GPU-unlock: FP/BF16 instruction issue rate on Nvidia 3000's cards

24•fragebogen•5h ago

Comments

Tiberium•4h ago

I've tested myself and the results are legit, this is incredible, even if a little risky (it relies on exploiting firmware): https://github.com/Zagitta/gpu-unlock/blob/main/docs/How%20A...

> Unfortunately for Nvidia the booter is vulnerable to a classic buffer overflow that writes straight into the stack which gives us the ability to execute code via Return Oriented Programming (ROP).

Repo for RTX 4090 and 4070 Ti Super: https://github.com/JustAResearcher/gpu-unlock-ada

jacquesm•51m ago

This one is real, but there are some caveats:

- you will need to run your card hotter than you might be comfortable with

- the real pay-off requires increasing the maximum power to 450W

- This is well above what the stock cooler was designed for so a bit more cooling would help

What's nice is that you can stack this mod on top of (or rather, below of, this one loads first) the p2p mod that has been out for a while, they don't seem to interfere with each other.

Tiberium•33m ago

> - you will need to run your card hotter than you might be comfortable with

Can you explain this one? On my GPU I didn't have to change the power limit. Obviously the card might spend more time at max power usage, but it's still within the limit.

jacquesm•27m ago

If you increase the power limit on the card you will be able to get much more mileage out of this modification. At 450W the difference is much, much larger but that is not something I'm comfortable with doing for longer runs. Note that the stock settings are already pretty extreme for that cooler, and that ambient temp and free airflow are going to be impacting the resulting GPU temp in a very direct way. I have my 3090's in an external box (with its own separate PCIe backplane) so they can talk p2p and have 'fresh air' rather than in the enclosure.

Tiberium•26m ago

Yes, but you can get up to a 40-50% increase in some workloads even with the same power limit, and some cards have it locked anyway. I just felt like it was a litle misleading for you to mention the power in your original post, since the unlock works regardless of that.

jacquesm•23m ago

Interesting, so I have 7 3090's here of different plumage and they all require a power increase to get > 25% gain with this modification, would you mind sharing the type of card you have?

Obviously, if there are 3090's out there that respond differently then I will be happy to retract that.

augment_me•32m ago

Scene_Cast2•45m ago

Really interesting.

On the other hand, this only applies to a limited set of workloads (training is often memory bandwidth bound for example).

I also wonder how this affects power consumption.

kingstnap•13m ago

Training is compute bound.

If its memory bandwidth bound you need to raise your batch sizes to leverage more parallelism.

superkuh•41m ago

Uh, no. This headline is wrong. It's FP32 unlocked from being halved, not 16 bit ops. And to be clear, over the last week I've looked many places and I've not seen any reports of this unlock being applied to any 3xxx card except the 3090/3090ti series. Even if it might apply more generally in theory.

augment_me•34m ago

The repo has fp16 numbers.

Also this one for 4070s shows that its for fp8 as well:

https://github.com/JustAResearcher/gpu-unlock-ada/releases/t...

Tiberium•32m ago

The repo has a lot of reports of people applying it to other 3xxx card, see https://github.com/Zagitta/gpu-unlock/issues

There's at least 3080 Ti, 3070 Ti, 3070, 3060

Zagitta•11m ago

It's the FP/BF16 FMA to FP32 instructions.

And also it does work on most 3000 series cards as they all run the same GA102 booter that's vulnerable :-) There are some reports of cards with weird vbios layouts having issues but i'll try to get that fixed

Zagitta•7m ago

Hey this is my work, although the exploits etc aren't my ideas by any means. This is just a way to wrap the whole thing up in a much less annoying package than having to patch the nvidia driver. I originally got inspired by the CMP90/50 windows unlocks that run as standalone UEFI modules that chainloads into the OS.

Happy to answer questions!

Google illegally cut down 250K trees in Finland for data center

TornadoVM 7 Enables GPU-Native Workloads from Java Applications

OBS Studio 33

Around a dozen banking apps began specifically permitting GrapheneOS

Node2Shell evolves to HTTP2Shell, blast radius affects many HTTP libs and apps

I Don't Understand Quantum Physics (2018)

Kimi K3.1 Pro and K3.1 Pro Highspeed

Physicists trap a single antimatter particle for a month

Today, we hacked what we'd set out to hack: the iPhone

Appreciating the beauty of all your body's scars and scratches

Twelve years of distributed SQLite

Local Wikipedia: Wikipedia, but only for things a half hour walk from you

Apple Music 404 Not Found

How to run a city builder on 16 MHz

How close can pure functional programming get to C?

A practical guide to AI image generation, from diffusion to agents

Bringing uv and ty together

Show HN: Autonomous root for Tcl Android TV via CVE-2022-22706

Dessa: Conversation Model trained only on speech

GPU-unlock: FP/BF16 instruction issue rate on Nvidia 3000's cards

AI-Assisted Paper on Z-Library Cites a TorrentFreak Article That Doesn't Exist

Canadians are running out of time to oppose Bill C-22

I built a neural network using DOOM games

Hacker reverses Samsung TV, finds on-board AI listening OpenAI account suspended

The Chalk and the Airplane

Apple Security Research Device

Tiling Noise – Procedural GPU Patterns

The Real Kevin Bacon

CVE Tracker

C vs. C#: racing VLC's Gaussian blur

Google illegally cut down 250K trees in Finland for data center

TornadoVM 7 Enables GPU-Native Workloads from Java Applications

OBS Studio 33

Around a dozen banking apps began specifically permitting GrapheneOS

Node2Shell evolves to HTTP2Shell, blast radius affects many HTTP libs and apps

I Don't Understand Quantum Physics (2018)

Kimi K3.1 Pro and K3.1 Pro Highspeed

Physicists trap a single antimatter particle for a month

Today, we hacked what we'd set out to hack: the iPhone

Appreciating the beauty of all your body's scars and scratches

Twelve years of distributed SQLite

Local Wikipedia: Wikipedia, but only for things a half hour walk from you

Apple Music 404 Not Found

How to run a city builder on 16 MHz

How close can pure functional programming get to C?

A practical guide to AI image generation, from diffusion to agents

Bringing uv and ty together

Show HN: Autonomous root for Tcl Android TV via CVE-2022-22706

Dessa: Conversation Model trained only on speech

GPU-unlock: FP/BF16 instruction issue rate on Nvidia 3000's cards

AI-Assisted Paper on Z-Library Cites a TorrentFreak Article That Doesn't Exist

Canadians are running out of time to oppose Bill C-22

I built a neural network using DOOM games

Hacker reverses Samsung TV, finds on-board AI listening OpenAI account suspended

The Chalk and the Airplane

Apple Security Research Device

Tiling Noise – Procedural GPU Patterns

The Real Kevin Bacon

CVE Tracker

C vs. C#: racing VLC's Gaussian blur

GPU-unlock: FP/BF16 instruction issue rate on Nvidia 3000's cards

Comments