It's built with the Realtime API, WebRTC (via LiveKit), and Three.js, Next.js plus a custom lipsync animation system. The biggest challenge was performance — running smooth 3D and real-time audio in-browser is brutal. I ended up designing a lightweight 30MB character model (down from 500MB) to make it usable on the web.
It’s desktop-only for now. Mobile browsers don't have enough performance for heavy 3d models. If browser version gets stuck, just reload the lesson.
Would love feedback on: – Whether this tool can find real product–market fit in such a crowded space – How you’d approach bringing this to mobile (iOS especially)? In my experience Apple libraries don't work well for heavy hyperrealistic models. Unity seems the only reliable option rn. – Better ways to optimize or rethink this stack for 3D + realtime audio? Realtime API has random disconnects that I haven't figured out.
Try it here: https://www.anditutor.com