Disguise and Move.ai partner for AI-powered markerless mocap

Disguise and Move.ai have partnered to integrate the latter’s markerless mocap technology with the former’s graphics processing for film and episodic TV, broadcast and extended reality studios.

The pair are developing an AI-powered technology based on real-time markerless motion capture software, Invisible, that aims to remove the need for mocap suits. The technology works by extracting natural human motion from video using advanced AI, computer vision, biomechanics and physics to automatically retarget the data to a character rig and create a virtual character that can mirror human motion in real time. 

The motion capture data from Invisible will be directly integrated into creative workflows in the disguise Designer software. Meanwhile, disguise’s RenderStream protocol will transfer skeleton data across the disguise Unreal Engine rendering cluster, allowing for greater synchronicity of content and tracking data across the production workflow and the merging of the physical and virtual world.

This hopes to end challenges such as latency, sensor drift, interference and extensive data cleanup. Examples of its use include avatars in metaverse experiences; digital characters in virtual productions and AR players in broadcasts; realistic shadow casting for talent onstage; gesture-triggered 3D graphics and scene changes; and movement-triggered particle effects, such as smoke and fire. 

You can watch the technology in action below.

Disguise chief product and technology officer Raed Al Tikriti said: “When it comes to virtual characters and real-time effects, movement needs to be translated into data. Motion capture and recreation are key pieces of the puzzle. We want to make this technology as accessible and scalable as possible, enabling our community of partners to enhance shared experiences, entertainment and storytelling.”

Fernando Kufer, Disguise CEO, added: “We’re excited to be working with Move.ai, who have undoubtedly been making waves in this space for the past year. As we continue to explore the intersection of physical and virtual here at disguise, a partnership with a team as forward-thinking as the Move.ai team, and as innovative and accessible as their product, is a no-brainer. The future of entertainment is on the horizon and together, we can play a big part in delivery.”

Niall Hendry, Move.ai head of partnerships, commented: “Creatives and technologists use disguise technology to deliver the most spectacular virtual and live productions. This strategic symbiosis will add the noiseless, real-time, markerless motion capture technology of Invisible to their arsenal. We can’t wait to see what users create with these tools.”