Lifelike digital faces with motion capture tech

By Casey Frye, CCNN Writer

software face capture
Hao Li’s software can analyze many different facial factors to create realistic adaptive images.

When I first saw the movie Avatar in theaters, I was utterly impressed by the digital effects used to create the alien world of Pandora. The landscapes of lush trees and bioluminescent plants glowing in the night were literally breathtaking. What was most eye-catching however, was not the quirky organisms or the trees that stored memories, but the blue Na’vi people. Sure, they were about 10 feet tall and had tails, but the way they were digitally created was completely believeable! That’s not to mention how similar the normal humans looked to the actual actors voicing them in the movie. Inventor Hao Li felt the same way when he watched Jurassic Park as a kid, and he set his sights on improving digital technology to better record the faces of actors.

You see, when visual-effects artists want to create realistic human motions, they’ll place tags on the bodies and faces of actors to track any movement. The tags are recorded and then sent to a computer where the information can be translated into a digital representation. However, the technology isn’t foolproof. Oftentimes the tags cannot keep track of subtle human facial expressions, or they’re uncomfortable for an actor to wear.

Li’s digital invention uses cameras equipped with depth sensors, which are usually found in motion capture games that work with the Xbox Kinect. When the camera is pointed at an actor’s face, special software monitors the tiny changes that occur from frame to frame. This information is then used to create a more realistic digital representation of their features! “That moment of seeing something that didn’t exist in reality, but it looked so real – that was definitely the one that made me think about doing this,” said Li.

As awesome as it will be for lifelike fantasy movies (I can’t wait for Avatar 2!), the technological applications go far beyond the realm of entertainment. In fact, the medical field has already adopted Li’s invention to monitor tumors in cancer patients, and even developed software for a digital beating heart. His ultimate goal is to incorporate his invention into robotics and help our metal friends make more sense of the world! Imagine a robot placing a comforting arm around your shoulders the moment it reads sorrow on your face.

Images and video courtesy of Hao Li.