Demo: https://mikapo.vercel.app
How it works:
MediaPipe extracts 3D landmarks from your camera feed Custom hierarchical solver converts landmarks to MMD bone rotations Real-time 3D rendering with Babylon.js
Technical challenge: Mapping MediaPipe's world-space coordinates to MMD's bone coordinate system while handling left/right asymmetries and maintaining smooth animations across frames. Features:
Face, hand, and body tracking Multiple input sources (webcam, video files, images) Live character rendering with physics
Built with Next.js, TypeScript, and runs entirely in browser. The pose solver handles the complex math of converting computer vision landmarks into proper character bone rotations. GitHub: https://github.com/AmyangXYZ/MiKaPo