Synthesia Research has just released its latest innovation – HumanRF. This technology captures human movement from multiple angles and allows playback from unseen perspectives. Thus, making it possible for a single camera to capture video from different viewpoints. HumanRF is a 4D dynamic neural scene representation that captures full-body appearance in motion from multi-view video input and enables playback from the novel, unseen views. Scientists expect this breakthrough technology to revolutionize how we capture and generate videos.
Also Read: FILM Model: Scale-Agnostic NN Architecture for Creating High-Quality Slow-Motion Videos
Addressing the Challenge of Operating at 12MP
While most research focuses on synthesizing at resolutions of 4MP or lower, Synthesia Research has addressed the challenge of operating at 12MP. To store these generated images, they have also introduced a new dataset called ActorsHQ. ActorsHQ is a novel multi-view dataset with 12MP footage from 160 cameras for 16 sequences with high-fidelity, per-frame mesh reconstructions.
The dataset, ActorsHQ, consists of 39,765 frames of dynamic human motion captured using multi-view video. Synthesia used a proprietary multi-camera capture system and an LED array for global illumination. The camera system comprises 160 12MP Ximea cameras operating at 25fps. The figures highlight the close-up details captured at this resolution. The lighting system synchronizes a programmable array of 420 LEDs with the camera shutter. We set all cameras to a shutter speed of 650us to minimize motion blur for fast actions.
Enhancing Video Quality
Synthesia, the text-to-video generation platform, is known for its capabilities. This new research is expected to enhance the video quality by making the avatars more active and less static. With HumanRF, videos can capture human movements from multiple angles, providing a more realistic and engaging viewing experience.
Future Applications
This breakthrough technology has various potential applications, including sports broadcasting, movie-making, and video game development. Imagine actively watching a basketball game from the ball’s perspective as players pass it between each other or experiencing a movie scene from various angles. Video game developers could also use this technology to create more immersive gameplay experiences.
Also Read: How AI Is Revolutionizing Game Testing in 2023
Our Say
Synthesia Research’s HumanRF, with its 4D dynamic technology, is a game-changer in video generation and capture. It allows single cameras to capture video from different viewpoints, providing a more comprehensive and realistic viewing experience. Moreover, with the introduction of ActorsHQ, researchers can now operate at 12MP. Hence, making it possible to capture close-up details that were previously impossible to capture. Experts expect this innovation to enhance video quality and revolutionize capturing and generating videos.
Also Read: Reversing the Video Using Computer Vision
A 23-year-old, pursuing her Master's in English, an avid reader, and a melophile. My all-time favorite quote is by Albus Dumbledore - "Happiness can be found even in the darkest of times if one remembers to turn on the light."
