...if you care about voice cloning.

Maybe that's not so important?

That's because you're sleeping on things like Higgs Audio

A big caveat of this is that the author is just looking at a ranking of open models? It's just buried in a little sentence in there but makes a big difference to model quality. Kokoro in the overall rankings is only 15, so if #15 is what you consider the "best model" you need to be cognizant that you are leaving performance on the table.

I've heard a lot of Substacks voiced by Eleven Labs models and they seem fine (with the occasional weirdness around a proper noun.) Not a bad article but I think more examples of TTS usage would be more useful.

I guess the outcome is, open weight TTS models are only okay and could be a lot better?

It's very interesting to see that there actually are people who want to automatically create a "podcast" from their blog using their cloned voice. Is this just what tech bro culture does to someone? Or is it about hustling and grinding while getting your very important word out there. I mean over time one would certainly save up to 20 minutes for each article...

> Anything over 1000 characters starts hallucinating.

So just feed it batches smaller than 1000 characters? It's not like TTS requires maintaining large contexts at a time.

For local TTS for a podcast I'd try the quantized .gguf versions of Microsoft VibeVoice large in comfyui to clone my voice from a ~30 second speech sample the apply it to marked-up text of the desired podcast. But it'd be nowhere near real time and require dedicating a $300 GPU to it. And the quantized version often goes off the rails and loses consistency in voice tone or accent. So just one run often isn't enough and I you have to piece the good parts of many separate runs together. It's not set it and forget it.

I do a lot of desktop screen-reader and pdf/doc/epub/etc text to speech every single day. It's been 20 years and I still use Festival 1.96 TTS with voice_nitech_us_slt_arctic_hts voice because it's so computational cheap and just slightly a step about normal festival/espeak/mbrolla/etc type TTS quality to be clear and tolerable. In terms of this local "do screenreader stuff really fast" use case I've tried modern TTS like vibevoice, kokoro tts, sherpa-onx, piper tts, orpheus tts, etc. And they all have consistency issues, many are way too slow even with a $300 GPU dedicated to them, and most output weird garble noises at unpredictable times along with the good output.

Slightly off-topic but Why would a blog of all things have DRM content that I need to enable?

Superhuman TTS is well within the capabilities of the big AI labs. Even Google had voice indistinguishable from human back in 2017, but they deliberately kneecapped it because of the potential for misuse. Boomers and older folks are not culturally or mentally equipped to handle it - even the crappy open source voice cloning we had in 2019 got used to scam people into buying gift cards.

Because of the potential for abuse, nobody wants to release a truly good, general model, because it makes lawyers lose sleep. A few more generations of hardware, though, and there will be enough open data and DIY scaffolding out there to produce a superhuman model, and someone will release it.

Deepfake video is already indistinguishable from real video (not oneshot prompt video generation, but deliberate skilled craft using AI tools.)

Higgsfield and other tools allow for spectacular voice results, but it takes craft and care. The oneshot stuff is deliberately underpowered. OpenAI doesn't want to be responsible for a viral pitch-perfect campaign ad, or fake scandal video sinking a politician, for example.

Once the lawyers calm down, or we get a decent digital bill of rights that establishes clear accountability on the user of the tool, and not the toolmaker, things should get better. Until then, look for the rogue YOLO boutique services or the ambitious open source crew to be the first to superhuman, widely available TTS.

Commercial TTS mostly sucks too.

There’s flashes of brilliance but most of it is noticeably computer generated.

While it sounds like this blogger doesn't want to bother (and perhaps experimenting with AI is itself the appeal), I personally appreciate when authors read their posts instead of delegating the task to AI.

> After filtering by my stupid rule of open models,

That's a good rule.

> You must enable DRM to play some audio or video on this page.

Looks like `embed.podcasts.apple.com` isn't in the same spirit.

>However, like many models in this leaderboard - I can’t use it - since it doesn’t support voice cloning.

That's such a strange requirement. A TTS is just that. It takes a text and speaks it out loud. The user generally doesn't care whose voice it is, and personally I think TTS's sharing the same voice is a good thing for authenticity since it lets users know that it's a TTS reading the script and not a real person.

You want your voice to be reading the script, but you don't want to personally record yourself reading the text? As far as I'm concerned that's an edge case. No wonder that TTS's can't do that properly since most people don't need that in first place.