Disclaimer: I work for a manufacturer of headphones and microphones. If you’re into Pro Audio, you have undoubtedly heard of or used our products, which are in widespread use throughout the industry. I spend my days in the lab with DSP engineers who are responsible for noise cancellation and other algorithms some of you may even be using right now, in fact. I myself work on plugins related to post-processing features to support our microphones, which are in wide use throughout the industry, on stage and in the studio, by amateurs and pros alike.

I found a lot wanting in this article. The emphasis on a lack of tooling seems very ill conceived. The #1 DSP tool in use in this industry is .. Matlab.

Yet, it’s not even mentioned.

Here’s how it goes: Matlab is used to model and test everything. Algorithms are developed, refined, tested and evaluated in Matlab, which has ample capabilities to do realtime audio with high performance. Once the algorithm has been demonstrated and proven, it is ported from Matlab to C, and then core parts are implemented in assembly.

This does not take years. It takes months and in some cases, with a great DSP team, even just weeks. Matlab has plenty of tools for doing this properly .. from visual tools, to optimization functions. There are very, very few professional DSP engineers who don’t work with Matlab in this way.

So this article really just felt a bit more like marketing copy designed to sell their product .. and as I got to the bottom of the article to see the mention of the JS event loop and the browser, I felt like the article was intentionally being disingenuous about the real issues. (The real issue with embedded DSP development is that hardware vendors are terrible compiler developers; but this has had the side effect of making great DSP guys into great assembly programmers. You absolutely must use assembly; but you must also absolutely use Matlab.)

And then, there is JUCE. JUCE is rapidly becoming THE tool to use for high performance audio - it dominates the desktop plugin market - but, as it matures, it has become more and more embeddable. There are devs out there who have made millions with JUCE plugins and are now porting their JUCE code to hardware and embedded platforms with a great deal of ease.

If you want to do embedded DSP properly, start with Matlab, get your ideas implemented, port to C++, integrate into JUCE, and then port your JUCE code to embedded, if your BOM budget supports a decent processor - or, port your processBlock() to assembly, if it doesn’t. This is the future of DSP platform development. Meanwhile, Matlab->C->Assembly is a very well refined workflow, and it is quite productive. One of the best reasons to keep those assembly language programming chops sharp.

Fun coverage of what all goes in to such a fascinating compelling topic, that so so so few people have any real idea about (computer × audio pipelines). I'd like to run through the major players as I see them right now: Sound Open Firmware and then the ever present widely adopted Cadence Tensilica Xtensa HIFI IP that's on a good number of systems-on-chip. Then gaze forward a little.

Based on the whose who of who contributors/users (Intel, MediaTek, Realtek, AMD (although it seems they dropped the hardware for it after Zen3?), my impression is that Sound Open Firmware is the 900 lb gorilla in this area. There's a ton of work & capabilities poured in here. No one wants to own this and differentiating yourself is extremely hard. So cooperation makes sense. So: SOF; started 2018 & quickly picked up by the Linux Foundation. With the 2.4 release (January 2023) they switched to Zephyr as the base embedded OS, which seems like an awesome win to offload development efforts & perhaps to prosper from subsystems like Bluetooth support. https://archive.fosdem.org/2024/schedule/event/fosdem-2024-2... https://thesofproject.github.io/latest/introduction/index.ht... https://www.phoronix.com/search/Sound+Open+Firmware

SOF is a software consolidation effort, and has done great. The somewhat wilder side is the hardware. Not my field but again my impression is that, at this point, Tensilica (acquired by Cadence in 2013)'s Xtensa IP is basically used by everyone, often the incredibly featurefully integrated DSP rocking HIFI 4/5/5s/1. Tensilica's Xtensa is famously used in Espressif ESP series (and now with RISC-V designs). But Tensilic (mostly "LX"?) cores are everywhere, almost always explicitly for audio (or vision, in the case of Hololense) processing. With nice chunky DSPs and now coming with 32 bit float, nicely parallelizing/auto-vectorising with Cadence's proprietary XCC compiler, this is everywhere. There's HIFI4 and HIFI5 (and now 5s) cores in a wild number of chips: almost all modern Intel laptops/desktops, to tiny little Cortex A7 chips to tiny little TWS systems. Qualcomm has some Xtensa but I'm not sure where. For example there's a great Defcon talk on hacking ath10k wifi, which is Xtensa based! https://media.defcon.org/DEF%20CON%2031/DEF%20CON%2031%20pre... https://www.cnx-software.com/2022/02/13/allwinner-t113-s3-du... https://semiwiki.com/forum/threads/wuqi-microelectronics-sel...

Cadence has the DSP market down right now, seemingly.

Thoughts on the frontier/future: I'm interested to learn what AMD did after walking away from TrueSound hardware (having a HIFI block on their designs). Ther was work they were trying to pursue GPGPU efforts, and maybe on APU's the latency wouldnt be an issue, but I sort of expect its mostly software now, and I'd love to know more about what that software.

SOF has had years of wanting to get more open. But that XCC compiler seems like a so-far holding moat, and I'm not sure who else is even trying to have more DSP like instructions or cores to target the market. SOF has been trying to get running on ARM or x86 or RISC-V for a while, but without some vectorization and nice low power parallel hardware it's hard to hope for too much. I keep hoping RISC-V RVV vectors might maybe do the thing. But how much success folks have had with the plans to get llvm doing vectorization is very unknown.