At the edge of the metaverse: LED Virtual Production, Live motion capture + high quality characters



Throughout 2021 Final Pixel (www.finalpixel.com) has been honing their LED wall virtual production workflow and pipeline through experience on multiple large scale commercial shoots for a range of clients – including ABC, Discovery and Shutterstock. The end-to-end nature of their production approach has meant they have been involved from initial concept to final post on many large scale virtual productions, and have developed a reliable workflow for shooting using high-end LED walls and cameras.

Throughout this period their virtual art department (VAD) has been developing technologies that allow them to incorporate the use of digital animated humans or creatures into their virtual production shoots. For example, they have experimented with Epic’s metahumans and pre-recorded motion-capture animations as background digital humans on various shoots for Discovery. Ultimately they never made the final cut. In Final Pixel’s assessment, this typically stems from two key factors:

1. The realistic and believable look of the character
2. Realistic movement of the character

As a company specialising in virtual production for film, tv and advertising, they are excited by the opportunities working in real-time game engines can provide for the creative process when everything can be captured in-camera while shooting live-action. They’ve already seen how virtual production is moving elements of a traditional VFX pipeline with green screen or compositing into pre-production, and even being done ‘live’ on stage – like a show. The next evolution of this technology is to look at the elements which are still considered too heavy or complex to move out of the post-production workflow. Creature work is a big area for this, and also extremely important for storytelling narrative. Having live interactions between real-life actors and creature or character animation cements the creative process.

For this reason, they have been watching closely the development of real-time motion capture, which in recent years has reached new heights of fidelity, control and quality. So in Autumn 2021, they embarked on a research project, a ‘proof of concept’ to incorporate live-action motion capture of a detailed creature animation into their current successful virtual production workflow.

This R&D exercise was in partnership with Target3D, who led on the motion capture workflow and collaborated on the on set virtual production alongside the Final Pixel team.

The project was shot at the Digital Catapult’s Virtual Production Test Stage (VPTS) a joint venture with Target3D the UK. VPTS is a full end-to-end VP studio aimed at research, development, access and skills for SMEs, practitioners and startups. VPTS is part of StudiosUK, a cluster of interconnected research facilities leveraging 5G and future networks capability to enable the exploration of convergent production technologies – such as the use of high density LED walls, motion-capture, real-time rendering engines and new forms of real-world capture.

source

Leave a Comment