Microsoft’s 3D Photo Realistic Talking Head

This was one of the technologies on show at Craig Mundie’s TechForum earlier this week. Zhengyou Zhang takes us through a demo of the talking head – usual approaches are a 2D talking head (used for tech support on many sites) or a 3D avatar. This approach takes the best of the two worlds, taking a 2D photorealistic video and paste this on to a 3D mesh model that is built from a Kinect sensor or webcam.

In the demo, Zhengyou Zhang uses text to speech (TTS) to “drive” the 3D head in real-time and explains that over time, the 3D head could recognize your facial expression and react in real-time to those.