Next-Gen GPU Programming: Hands-On with Mojo and Max Modular HQ

https://www.youtube.com/live/uul6hZ5NXC8?si=mKxZJy2xAD-rOc3g

44•solarmist•1y ago

Comments

solarmist•1y ago

I'm really hoping Modular.ai takes off. GPU programming seems like a nightmare, I'm not surprised they felt the need to build an entire new language to tackle that bog.

mirsadm•1y ago

GPU programming isn't really that bad. I am a bit skeptical this is the way to solve it. The issue is that details do matter when you're writing stuff on the GPU. How much shared memory are you using? How is it scheduled? Is it better to inline or run multiple passes etc. Halide is the closest I think.

solarmist•1y ago

What are you skeptical of? I believe the problem this is solving is a framework that's not CUDA that allows low level access to the hardware, makes it easy to write kernels, and is not Nvidia only. If you watch the video you can write directly in asm if you need to. You have full control if you want it. But it provides primitives and higher level objects that handle common cases.

I'm a novice in the area, but Chris is well respected in this area and cares a lot of about performance.

pjmlp•1y ago

There are already plenty of languages in CUDA world, that is one reasons it is favoured.

The problem isn't the language, rather how to design the data structures and algorithms for GPUs.

solarmist•1y ago

Not sure I fully understand your comment, but I'm pretty sure the talk addresses exactly that.

The primitives and pre-coded kernels provided by CUDA (it solves for the most common scenarios first and foremost) is what's holding things back and in order to get those algorithms and data structures down to the hardware level you need something flexible that can talk directly to the hardware.

pjmlp•1y ago

C, C++, Fortran, Python JIT from NVidia, plus Haskell, .NET, Java, Futuhark, Julia from third parties, and anything else that can bother to create a backend targeting PTX, NVVM IR, or now cuTile.

The pre-coded kernels help a lot, but you don't have to use them necessarly.

melodyogonna•1y ago

Yes, the problem isn't language, it is the entire stack. I think people focus too much on Mojo while ignoring the actual solution Modular has built, which is MAX. The main idea here is that MAX provides a consistent API for both library authors (e.g vLLM, Ollama) to target, as well as for hardware vendors to integrate with - so similar to LLVM.

Basically, imagine if you can target Cuda, but you don't have to do too much for your inference to also work on other GPU Vendors e.g AMD, Intel, Apple. All with performance matching or surpassing what the hardware vendors themselves can come up with.

Mojo comes into the picture because you can program Max with it, create custom kernels that is JIT compiled to the right vendor code at rumtime.

diabllicseagull•1y ago

It is a noble cause. I've spent ten years of my life using CUDA professionally, outside the AI domain mind you. Most of these years, there was a strong desire to break off of CUDA and the associated Nvidia tax on our customers. But one thing we didn't want was to move from depending on CUDA to depending on another intermediary which would also mean financial drain, like the enterprise licensing these folks want to use. Sadly, open source alternatives weren't fostering much confidence, either with their limited feature coverage or just not knowing if they will be supported in the long term (support for new hardware, fixes, etc.).

pjmlp•1y ago

Also while as language nerd I find Mojo cool, given NVidia's going full speed ahead with Python support in CUDA as announced at GTC 2025, to the point of designing a new IR as basis for their JIT, very few researchers will bother with Mojo.

Also what NVIDIA is doing has full Windows support, while Mojo support still isn't there, other than having to make use of WSL.

melodyogonna•1y ago

Why? Will the new Nvidia Python stuff work on AMD GPU and other non-nvidia accelerators?

pjmlp•1y ago

It still remains to be seen how much that will happen to Mojo and MAX, while most researchers are using CUDA anyway, and best of all, it works on their laptops, which cannot be said for AMD GPU and other non-nvidia accelerators.

Naturally assuming they are using laptops with NVidia GPUs.

catapart•1y ago

My mistake completely, but I thought this was going to be something to do with a new scheme or re-thinking of graphics programming APIs, like Metal, Vulkan or OpenGL. Now I'm kind of bummed that it is what it is, because I got really excited for it to be that other thing. =(

pjmlp•1y ago

That is already taking place with work graphs, and making shader languages more C++ like.

ttoinou•1y ago

Seems like with it you will be able to compile and execute one code on multiple GPU targets though

ashvardanian•1y ago

There is a "hush-hush open secret" between minutes 31 and 33 of the video :)

refulgentis•1y ago

TL;Dr same binary runs on Nvidia and ATI today, but not announced yet

throwaway314155•1y ago

They desperately need to disable whatever noise cancellation they're using on the audio. Keeps cutting out, sounds terrible.

solarmist•1y ago

Yeah, the mic quality was terrible.

hogepodge•1y ago

This was the first time we ran an event in the office with this wireless mic setup. We're definitely aware of the problems, and will have them fixed for the next event.

Archit3ch•1y ago

> Other Accelerators (e.g. Apple Silicon GPUs): free for <= 8 devices

From their license.

It's not obvious what happens when you have >8 users, with one GPU each (typical laptop users).

threecheese•1y ago

This is covered by ARM which they consider CPU, and doesn’t fall into that clause. IOW no restrictions.

Security problems, what security problems? This is only the beginning

HN: AMPH – A keyboard-driven TUI for managing Arch Linux package sets

Thank you, but I prefer it my way

AEONKit targets TheA1200 and Amiga's partners with worldwide trademark grab

Turn a single illustration into an animated 2D avatar: Mesh Avatar Studio

Claude Cohen-Tannoudji (Physics Nobel Prize 1997) has died

Malaysian PM Anwar Bans Civil Servants from Using Israel-Developed Mapping App

Show HN: Jam – a platform where beginners can collaborate in music

Connecting ESP-Mosaico Board over ESP-Now to Run Amplifier Models in Parallel

October Counterfeit Block: What Happened and Where Every Coin Went

Farol: The anti-slop workspace for your agents

What Could Go Wrong

Glyphh Ada: Decision model that learns from your past decisions, no transformer

The most important thing today is that Starlink and Starshield operations

AI Assistance Reduces Persistence and Hurts Independent Performance

How AI Saved My Company From a 2-Year Litigation Nightmare

RFC 8949: STD 94: Concise Binary Object Representation (CBOR)

Not Invented Here

We are terminating Chris Degnan for unethical conduct involving Cognition

UX Research for AI Products: The AI-Native Research Lifecycle

All Abstractions Are the Same

What the U.S. Is Doing in Venezuela

Google illegally cut down 250K trees in Finland for data center

With recent eMudra case, learning Cryptography well is more important that

Show HN: TenseMem – deterministic memory for AI agents

BeNice Closing the Inference-Memory Gap of Looped Models with KV-Sharing

I created this because I wanted to

Group in San Francisco graftng fruit-bearing branches onto street trees

Under a Cloud: How was slavery justified?

Show HN: Firefighter.sh – an insane solution to the current CVE apocalypse

Security problems, what security problems? This is only the beginning

HN: AMPH – A keyboard-driven TUI for managing Arch Linux package sets

Thank you, but I prefer it my way

AEONKit targets TheA1200 and Amiga's partners with worldwide trademark grab

Turn a single illustration into an animated 2D avatar: Mesh Avatar Studio

Claude Cohen-Tannoudji (Physics Nobel Prize 1997) has died

Malaysian PM Anwar Bans Civil Servants from Using Israel-Developed Mapping App

Show HN: Jam – a platform where beginners can collaborate in music

Connecting ESP-Mosaico Board over ESP-Now to Run Amplifier Models in Parallel

October Counterfeit Block: What Happened and Where Every Coin Went

Farol: The anti-slop workspace for your agents

What Could Go Wrong

Glyphh Ada: Decision model that learns from your past decisions, no transformer

The most important thing today is that Starlink and Starshield operations

AI Assistance Reduces Persistence and Hurts Independent Performance

How AI Saved My Company From a 2-Year Litigation Nightmare

RFC 8949: STD 94: Concise Binary Object Representation (CBOR)

Not Invented Here

We are terminating Chris Degnan for unethical conduct involving Cognition

UX Research for AI Products: The AI-Native Research Lifecycle

All Abstractions Are the Same

What the U.S. Is Doing in Venezuela

Google illegally cut down 250K trees in Finland for data center

With recent eMudra case, learning Cryptography well is more important that

Show HN: TenseMem – deterministic memory for AI agents

BeNice Closing the Inference-Memory Gap of Looped Models with KV-Sharing

I created this because I wanted to

Group in San Francisco graftng fruit-bearing branches onto street trees

Under a Cloud: How was slavery justified?

Show HN: Firefighter.sh – an insane solution to the current CVE apocalypse

Next-Gen GPU Programming: Hands-On with Mojo and Max Modular HQ

Comments