This AI Can Create Your 3D Avatar using just your Smartphone Camera!

Overview:

• This AI needs just one small video of you to create your 3D form digitally
• The average accuracy of the model was within 5 millimetres!
• Check out the awesome video demonstrating this algorithm below

Introduction:

If you’ve ever played the ultra popular video game The Sims, you will love what the latest AI has come up with. That concept of building your own character in front of your eyes is no longer a fantasy.

This AI has the ability of creating 3D versions of people’s bodies which can be used for a variety of purposes, like animated movies, retail shopping, virtual gaming, telepresence, etc. It requires one a smartphone camera or a computer webcam to identify the person’s features and replicate them in their digital avatar.

The algorithm is built in different stages as we have shown below:

• The system first analyzes a video of the person (a few seconds long). The results are best when he/she is moving in the video such that it captures all sides
• For each frame a silhouette is created such that it separates the person from the background
• Based on machine learning techniques the computer learns and roughly estimates the 3D body shape of the person and location of joints
• Further it “unposes” the virtual human created from each frame, making them all stand with arms out in a T shape
• It then combines information about the T-posed person into one so as to create a more accurate model
• Finally, based on the video recording, it applies color and texture to the model to make it look more realistic

The team then tested their model on various body shapes, clothing, backgrounds and environments. The results were mind boggling – the model had an accuracy within 5 millimetres! They will be presenting their findings at the Computer Vision and Pattern Recognition conference in Salt Lake City in June this year.

For a better understanding of this algorithm, take a look at the below video:

The researchers have published their research paper here.

Our take on this:

Why this research is significant is because so far, computer vision and pattern recognition tasks have required multiple camera angles and expensive equipment to gather the data before it ready for collection. With this AI, it reduces the cost and can be used in a vast number of fields – from virtual reality avatars, movie making, video games, online retail, fashion design, and even surveillance.

Subscribe to AVBytes here to get regular data science, machine learning and AI updates in your inbox!

Aishwarya Singh

An avid reader and blogger who loves exploring the endless world of data science and artificial intelligence. Fascinated by the limitless applications of ML and AI; eager to learn and discover the depths of data science.