Show HN: Real-time webcam motion capture for MMD anime characters

2 points

9 months ago

Built a web app that captures your movements via webcam and applies them to 3D anime characters in real-time. No special equipment needed.

Demo: https://mikapo.vercel.app

How it works:

MediaPipe extracts 3D landmarks from your camera feed Custom hierarchical solver converts landmarks to MMD bone rotations Real-time 3D rendering with Babylon.js

Technical challenge: Mapping MediaPipe's world-space coordinates to MMD's bone coordinate system while handling left/right asymmetries and maintaining smooth animations across frames. Features:

Face, hand, and body tracking Multiple input sources (webcam, video files, images) Live character rendering with physics

Built with Next.js, TypeScript, and runs entirely in browser. The pose solver handles the complex math of converting computer vision landmarks into proper character bone rotations. GitHub: https://github.com/AmyangXYZ/MiKaPo