> I think as we look forward to the future, more inference will start happening locally for one reason or the other. This brings the distribution story front and center. In order to have more applications running inference locally, we need to make running inference easier.
This makes these projects so much more trustworthy and easier to approach:
> Were any of the words here written using AI? Nope. They came from my mouth or my fingers.
>> Were any of the words here written using AI? Nope. They came from my mouth or my fingers.
I have to push back on this a bit, as I believe (quite strongly) that we're shaped by the tools we use; text-to-speech LLMs are still LLMs, and generally their mistakes are shaped by the expectations inherent in their training. This, in turn, shapes the words that appear on the screen. For those who regularly use them, you then learn which word sequences are likely to be accurately transcribed, and this definitively becomes part of your thinking process. Over time, the LLM becomes tangled into your thinking; the use of AI, even in this way, very much can and often does shape the resulting words.
The M4 max has probably 10x the compute and memory bandwidth hahaha
Nemotron Streaming
Parakeet Unified
Voxtral Mini Realtime
If something you want is not supported, open an issue on transcribe.cpp!
Noob question: How do you think about funding from a foundation(i have no clue if you need it or not, I do hope you have a way to get paid one way or another because handy is amazing) for maintenance of this? if you did or were going to get paid by asking for maintaining such a project what might be the kind of organizations you would look for to get supported and how would you do it?
Certainly I am very lucky that quite a few people donate to Handy, and also some people and organizations who sponsor the work I do
To be honest I just love contributing to open source and wish to continue to do so. So anyone who supports this is good to me. Organizations which believe in OSS and push it forward are typically most aligned with me
Of course you can always email me (contact@handy.computer) and we can discuss in more detail
It's a nice reminder: You can use AI to slop cannon at maximum speed, or you can use it to scale your ambitions and build something more rigorous and lasting than ever before.
I'd build Transcribe.cpp into the apps I maintain, but I feel like this functionality should (generally) be integrated into the OS or "everywhere" via an app like Handy.
I agree this should be everywhere and I hope to distribute libtranscribe some day properly so it is more a system library! It will take time to stabilize but I think we can get there
Looks like it's using IBM's Granite-Speech-4.1-2B-Plus https://huggingface.co/ibm-granite/granite-speech-4.1-2b-plu... and/or MOSS-Transcribe-Diarize https://huggingface.co/OpenMOSS-Team/MOSS-Transcribe-Diarize
I’m not sure how many specific models will be supported as the library is more focused on transcription specifically. But the models which support diarization natively must be supported I think. And parakeet multitalker was the primary driving force for this change
The github page doesn't seem to go into depth on these orthogonal topics. May have missed it.
Nice. Here's the Python one: https://github.com/handy-computer/transcribe.cpp/tree/main/b... - looks like it's not yet available as a binary wheel on PyPI with the dependency included (the library on PyPI right now uses ctypes to call a separately installed library) but that's planned for a future release.
If there’s any issues or improvements on the bindings I would love help to make the DX the best it can be
You should be able to fix this by playing with the mic speech floor. It happens when to much ambient stuff slurps in.
It's actually gaslighting you, you don't say that many ums and ahs ;)
But the answer largely depends on you, the languages you speak, and personal preference. Whisper is still excellent and supported in transcribe.cpp
Cohere Transcribe is also excellent, but many of the new models are as well
I want to open an office document, for example, and start talking. And I want the software to continuously type what I am saying at the cursor with minimal latency. The continuous part is crucial. Many software will paste whatever I said after I have stopped recording, but that is not useful.
It looks like the rust bindings have streaming examples so hopefully there is a nice solution here.
I’m planning on having it as a first class feature of the app too just too many other issues to work on first
I often prefer to dictate everything in my head about a particular thing for 5–10 minutes and then go through it afterward. I find that much more useful because it doesn’t break my thought process the way continuous transcription does.
arikrahman•4h ago
zuzululu•4h ago