How AI on Microcontrollers Works: Operators and Kernels

https://danielmangum.com/posts/ai-microcontrollers-operators-kernels/

83•hasheddan•7mo ago

Comments

Neywiny•7mo ago

In the same way "embedded" is relative, I appreciate the author's recognition that "edge" is relative. For some, AI at the edge means on-prem server farms. For some it means a mini-pc. For others, maybe an SBC. Here it's a microcontroller. Further still is AI within the sensors a microcontroller would talk to. That's probably just another microcontroller but still.

woliveirajr•7mo ago

Microcontrollers all the way down

astrobe_•7mo ago

There's "micro" and "micro". The microcontroller operating a simple coffee machine, or a simple washing-machine is probably 8 or 16 bits. This is what I would call "bare metal", as they don't run an OS, only off-the-shelf frameworks at best.

For "bigger" devices, it's usually a Cortex inside a system-on-chip or system-on-module, 32 bits single core and a few Mb of RAM for low-end (enough to run regular Linux distro instead of uClinux for instance), 64 bits multicore for high-end devices that deal with audio/video. That kind of business is often resource-hungry in every way.

I work with that kind of stuff, and to me these "microcontrollers" are just monsters that I hesitate to call "micro" when some of my coworkers work on much smaller chips with only a few K of RAM available.

Neywiny•7mo ago

I do wish sometimes they used the bigger micro, though. We have some power supplies that technically have an Ethernet interface. But when using it, even for SCPI over TCP (forget about the virtual front panel that takes a minute to update), it lags so bad the output enable button needs a few tries to toggle. I should practice yanking the positive wire for an emergency

amelius•7mo ago

Indeed. Is nVidia Jetson "edge" or not?

simgt•7mo ago

Of course it is. Edge mostly stands for network edge, Jetsons aren't meant to be deployed in data centers.

amelius•7mo ago

I don't understand why we need a separate "lite" format for microcontrollers.

Wouldn't it be advantageous if we used ONNX for everything? https://onnx.ai/

batuhandumani•7mo ago

tf-lite micro library has many advantages, and the first of these is the tensorFlow framework itself. You can train the model easily and then implement the same or a similar architecture on esp-32s without much effort. Another advantage is its optimization and you can easily intervene in various memory optimizations and even though it is not a large one, it does have a community.

Apart from these, for example, the author implemented the model the traditional way using C, but it is more convenient to use tf-lite micro on esp32s with the Berry script language.

However, since I have never used onnxin this kind of project, I cant speak to its advantages, so comparisons are difficult from my perspective. But as I said, tf-lite and offer benefits like easy integration, good optimization, and as the name implies, tensorFlow.

gozzoo•7mo ago

What about the so called NPUs which are present in some modern microcontroller chips?

SectorC: A C Compiler in 512 bytes

The F Word

Brookhaven Lab's RHIC concludes 25-year run with final collisions

Speed up responses with fast mode

Software factories and the agentic moment

Hoot: Scheme on WebAssembly

Stories from 25 Years of Software Development

OpenCiv3: Open-source, cross-platform reimagining of Civilization III

I write games in C (yes, C)

First Proof

The Waymo World Model

Show HN: A luma dependent chroma compression algorithm (image compression)

Al Lowe on model trains, funny deaths and working with Disney

Vocal Guide – belt sing without killing yourself

Start all of your commands with a comma (2009)

Reinforcement Learning from Human Feedback

Show HN: I saw this cool navigation reveal, so I made a simple HTML+CSS version

Selection Rather Than Prediction

Coding agents have replaced every framework I used

A Fresh Look at IBM 3270 Information Display System

The AI boom is causing shortages everywhere else

France's homegrown open source online office suite

72M Points of Interest

We mourn our craft

Unseen Footage of Atari Battlezone Arcade Cabinet Production

Where did all the starships go?

Show HN: Kappal – CLI to Run Docker Compose YML on Kubernetes for Local Dev

Learning from context is harder than we thought

Show HN: Look Ma, No Linux: Shell, App Installer, Vi, Cc on ESP32-S3 / BreezyBox

History and Timeline of the Proco Rat Pedal (2021)