> Unit (thread in CUDA, invocation in Vulkan/Wgpu): the smallest execution entity performing computations.
> Plane (warp in CUDA, subgroup in Vulkan/Wgpu): a group of (typically 32) units executing in lockstep and able to share data efficiently through registers.
> Cube (thread block in CUDA, workgroup in Vulkan/Wgpu): a group of units that execute on the same SM, sharing memory and able to synchronize
It's already bad enough that the vendors themselves insisted on different names but why in the bejesus would you rename these concepts and diverge from literally all existing naming conventions when you're providing middleware. Ie when using your tool I'm still going to reference NVIDIA's or AMD's docs to understand how the hardware actually works. Like do you really think otherwise - that your thing is gonna be end of the line???
FYI the word warp isn't random techno babble but is actually a very clever pun that actually fits very well conceptually:
There you go you've hit basically two of 3 completely (AMD and Vulkan) and are close enough to CUDA that people would get it.
I have no idea what a plane connotes and a cube literally gives a distinct enough picture from block that I will be continuously reminding myself of the mapping.
What you did was pointless - you assigned new words to objects that you don't own and now your conceptual framework is askew from the actual underlying (true) conceptual framework.
> CubeCL to CPU
There is zero affinity between GPU programing models and multicore CPU programing models. If you don't believe me go ask the OpenMP people how they're doing supporting GPUs.
Congrats - I have no idea what this means lol.
It does come with some mental overhead, but let’s be honest, there’s no objectively “good” choice here without introducing bias toward a specific vendor API.
Learning the core concepts takes effort, but if CubeCL is useful for your work, it’s definitely worth it.
Just commenting to share, personally I have no naming preference but the hierarchal abstractions in general are incredibly useful.
Unit, plane (as vs train), and cube?
Or point, plane, cube (1d, 2d, 3d)?
If you have a measure of correctness, and a measure of performance. Is there a maximum value of correctness per some unit of processing that exists below a full matrix multiply
Obviously it can be done with precision, since that is what floating point is. But is there anything where you can save x% of computation and have fewer than x% incorrect values in a matrix multiplications?
Gradient descent wouldn't really care about a few (Reliably) dud values.
https://kolinko.github.io/effort/
For semi-random weights you cam get down to 20-30% multiplications/mem reads and maintain ~0.98 cosine similarity output between the approximated and full result.
As far as LLM inference goes, the speedup from removing multiplications is at best comparable to the speedup of quantisation (that is - you get at best similar KL divergence score whether you remove calculations or quantise).
It is... it's in GPUs lol
> first class in torch
It is
> costing a fraction of GPUs
Why would anyone give you this for cheaper than GPUs lol?
So really GPU v not GPU (e.g. TPU) doesn't matter a whole lot if you've got fundamentally the same memory architecture.
Typically, you are not doing a matrix multiplication for the sake of it, but as part of a broader algorithm (e.g. a simulation). Without fusing those other operations into the MatMul kernel, you are leaving performance on the table. How will the Burn devs address this?
raphaelty•6mo ago