Hi HN — I’ve been working on a real-time conversational AI, and one of the biggest problems is speech latency. As I was experimenting with various TTS systems I stumbled upon Pocket-TTS by Kyutai. It runs on the CPU which made it a great candidate since it keeps speech synthesis away from GPU resources.

As I was using it I noticed that it didn't solve the latency issues fully due to leading silence and artifacts generation. So I started poking under the hood with the help of agents to try and improve it.

On m4 max, the new browser build runs around ~14x realtime on longer utterances, and around ~3.5x realtime on an iPhone 16 Pro. The native CPU path is faster still: roughly 32–33x realtime warm, with first useful audio around 30ms on my benchmarks. Other systems will vary; my Windows browser results were slower but still usable. This does not introduce a new model or claim better voice quality. The goal was to preserve the intended PocketTTS quality while making it fast enough for low-latency interactive scenarios.

I decided to open source it in case it's useful for anyone else, or at least as a demo of what things that are finally possible now that the barrier of trying stuff has been lowered.

EDIT: Keep in mind it downloads 67mb of data on load.