but why use “~float16” for something that in its own documentation the authors describe as a float16* ?

"...made incremental improvements to CUDA" just doesn't get you promoted like "creator of GPU Kernel Programming language Tilus"

My impression of NVIDIA is that internally the teams are quite independent and there's not much in terms of broad strategy. So it could be these two efforts are not related.

programming w/ tiles means working w/ contiguous-ish blocks/squares of data, to minimise cache misses (often major bottleneck in gpu programming), so it's just a nod to the fact that optimisations like these are included in the language or some such...

I'd say the main difference is that in traditional GPU languages, the thread of execution is a single lane of a warp or wave. You typically work with ~fp32-sized values, and those are mapped by the compiler to one lane of a 32-wide vector register in a wave (or 16- to 128-wide depending on the architecture). Control flow often has to be implemented through implicit masking as different threads mapped to lanes of the same vector can make different control flow decisions (that is, an if statement in the source program gets compiled to an instruction sequence that uses masking in some way - the details vary by vendor).

In tile languages, the thread of execution is an entire workgroup (or block in CUDA-speak). You typically work with large vector/matrix-sized values. The compiler decides how to distribute those values onto vector registers across waves of the workgroup. (Example: if your program has a value that is a 32x32 matrix of fp32 elements and a workgroup has 8 32-wide waves, the value will be implemented as 4 standard-sized vector registers in each wave of the workgroup.) All control flow affects the entire workgroup equally since the ToE is the entire workgroup, and so the compiler does not have to do implicit masking. Instead, tile languages usually have provisions for explicit masking using boolean vectors/matrices.

Tile languages are a new phenomenon and clearly disagree on what the exact level of abstraction should be. For example, Triton mostly hides the details of shared memory from the programmer and lets the compiler take care of software pipelined loads, while in this Tilus here, it looks like the programmer has to program shared memory use explicitly.

It's like writing code directly for the GPU's DSP-like SIMD cores in assembly, instead of taking the CUDA model of targeting a single SIMD thread, from which the compiler figures out how to write assembly for the core itself.

Maybe this is my dad speaking through me when he got tired of answering questions and said "look it up" and pointed to our bookshelf but....

Copying and pasting your exact words above into an LLM (gemini/chatgpt) provided an answer arguably better than any of the human answers at the time of this post.

What’s bad for Nvidia is good for everyone else. The cuda lock-in needs to die.

After looking up Triton this seems quite different - Triton is a high-level CUDA competitor with Python-like syntax, and this seems to be a library aimed at generating GPU assembly for micro-optimizing kernels.

There are a lot of hardware features inaccessible from webgpu because its devs still have a lot of work to do for the current implementations—no browser is even shipping it on Linux by default.

Vulkan is closer, but CUDA still exposes more features.

WebGPU is a API that sits above Vulkan, DirectX and Metal. For CUDA, ROCm and oneAPI there is SYCL. But it's not used too much and it seems mostly just Intel is invested.

WebGPU is the lowest common denominator over ancient mobile phones and modern hardware; as well as Vulkan, DirectX and Metal; effectively making it an ancient-mobile-phone-graphics-API. It is at the time of its releas as outdated as WebGL was when it came out.