The UX in some cases required voice interactions, others needed a graphical user interface (GUI), and in one instance I needed camera for reading barcodes/QR codes.
Unlike desktop or mobile apps, where there are well-developed SDK and development environments like Android Studio, Xcode, etc (which separate the low level concerns for developers), there is no fully open source capable alternative for embedded linux.
So for example, if you are developing a hardware-integrated app on Raspberry Pi that requires audio/speech interactions, an embedded graphical user interface, interfacing with sensors, camera, and/or other on-board hardware, you typically have to start from scratch and integrate these elements directly in your application (instead of using platform APIs).
So I started Ubo projects a few years ago which aims to address this issue through a collection of open source building blocks (software and hardware).
I am exciting to release the latest stable version of Ubo app (v1.6) which abstracts system and hardware level interactions with a clean event driven API.
It allows developers to add GUI, audio, speech, camera/vision, LED ring, Infrared RX/TX, sensors, etc to their apps with a few lines of code.
You can run Ubo app on Raspberry Pi 4 and 5.
For hardware specific capabilities, you can either DIY your hardware as explained in README.md or obtain an Ubo pod (an open hardware companion for Ubo app).
Some of the hardware capabilities such as the embedded GUI and audio playback is currently also emulated in the browser.
For more information Ubo app, check out my GitHub repository.
Would love to hear your feedback and how something like this could help you in projects.
P.S. there are some experimental ai driven features enabled by pipecat-ai in the works to bring more interactive voice/vision interfaces to developers apps.