I guess I am too risk adverse to bet $10,000 on an ASIC run (having no experience doing that and no desire to go that route).
If I can demonstrate a proof-of-concept in a reproducible research way, others can then advance to the ASIC level.
This is just a hobbyist experiment looking for other hobbyists who can afford a cheap FPGA and have some free time and interest.
My understanding (as a non-FPGA expert) is that currently FPGA beats generic hardware (CPU,GPU) for "small size algorithm" (i.e that do not need GB of weights), while enabling a certain flexibility vs ASIC.
My guess is that you cannot bake all the weights into the circuit topology, so you are still bound by the memory transfer speed (to be double checked).
wmf•6h ago
softwarewright•4h ago
The reason for this is, model weights do not need to be randomly accessed. So why store them in expensive RAM.
Cerebras and Qrok seem to be using a very different approach than NVIDIA to get orders of magnitudes speed ups. I'm trying to explore other alternative approaches.
wmf•3h ago
softwarewright•2h ago
I have the MCUs and FPGAs (in a drawer) and I am retired, and this is my idea of fun.
I am trying to generalize an approach to use large MoE models (with possibly small quants) to run many agents in parallel without spending more on more or bigger GPUs.
I am also doing some edge ML (bird species recognition near the camera) using NPUs (in design phase, yet untested). I have an electronics lab, and I've emulated soft CPUs and built software that runs on FPGAs and in my emulators.
Instead of assuming my approach won't work or is too expensive, I choose to be optimistic. Also, failures are educational. I'm trying to gain more FPGA experience.