My struggles are as follows:
1. Voice to voice is promising but is not quite there in quality. Not sure what kind of model is being leveraged underneath but responses are worse than 4o
2. Have not used Livekit but seems very popular. Though not sure why it is needed
3. Interruption handling: Did not come across a model or system that handles these well. Even 4o gets highly confused after around 2 minutes of talking and one single interruption in my experience