The phonetics work quite well for vowels - you can try making a sound and then seeing if the predicted tongue prediction matches where your tongue was. It's less accurate for consonants; it doesn't do plosives or nasals (I tried).
How the machine learning works:
It's a speech autoencoder in which the decoder is Pink Trombone. A couple of issues: PT is not differentiable, so I use Gumbel softmax to get a differentiable approximation. PT also generates audio sample-by-sample (apparently it's called a "Kelly-Lochbaum vocal tract model"), which would make training extremely slow. To do that, I approximate it using FIR filters that change each 10ms. The fundamental frequency is not computed by the model, but by a method called pyin. Now that the gradient can flow through the model, the question is what loss to use. Matching the spectrum/MFCC coefficients gets you to something that sounds like speech, but isn't intelligible. What does work is trying to match the original and generated audio in the space of a self-supervised audio representation model - I used wav2vec2. There are a couple of tricks on top of that like encouraging the model not to move the parameters too quickly, and balancing the Gumbel softmax between exploration and exploitation.