Talking Augmented Reality Workflows for Live Production 
Augmented reality (AR) has become a popular storytelling tool among production professionals to deliver a more engaging audience experience. Konstantin Ganzhela (KG), Creative Director at Carrot Broadcast, the developer behind the Carrot Engine, a toolset for producing AR graphics and virtual sets, and RT Project Supervisor Roman Nikitin (RN), one of Carrot’s clients, recently sat down with us to talk about the trend and what it takes to produce an AR project. 

Why is AR attractive to broadcasters? 
[RN]: Capturing the attention of modern audiences has become increasingly challenging due to the  overwhelming volume of content available and digital distractions. AR is attractive because it’s allowing traditional broadcasts to better engage audiences for longer periods of time by using interactive visuals.

Which kind of productions is AR best suited for? 
[KG]: One of Roman’s latest projects, an immersive virtual set celebrating the 60^th anniversary of Cosmonaut Yuri Gagarin’s historical outer space orbit, is a great example of the type of production that is ideal for AR. With coverage to commemorate the event planned, his team thought about how it could reimagine a traditional broadcast with a unique visual experience. By developing a virtual set, they were able to expand the studio’s boundaries to create outer space in AR. You can check it out here. 

What does a typical day look like on an AR project like this? 
[RN]: In recent months, more of our work has taken place remotely due to pandemic restrictions. Initial concepts, drawing and assembly are typically completed from our home offices, and virtual set tests are conducted in the studio. Based on the test results, our team returns home to work on the rest of the project, using online meetings to stay connected, collaborate and iterate until we’re satisfied with the final result. 

How does AR production typically differ from traditional production?
[KG]: The main difference we’ve seen with clients working in AR is that it requires more team work on-site rather than from a workstation. In my experience, the designers we work with, like those on Roman’s team, tend to enjoy this part of the work, as they’re not tethered to their desks. 

Describe the challenges that AR productions pose. 
[RN]: While working on the project commemorating Yuri’s journey, we encountered a range of new challenges. The project required linking broadcast and virtual cameras, and creating a virtual set that included an outer space environment, 3D digital spacecraft and a physical news studio. We had to find an AR solution that could drive the virtual set and ensure smooth operations from start to finish. Thankfully, Carrot Broadcast’s solutions work out-of-the-box and combine the best computer graphics technology used in TV, cinema and game development. Using Unreal Engine and Carrot Broadcast’s solutions with integrated AJA Corvid I/O cards allowed us to leverage the full functionality of the engine and camera and mask system to bring our concept to fruition.

What are the key things to keep in mind when working on an AR project?
[KG]: Start simple, and make sure that you’re constantly checking and monitoring the output on-site. AR production combines pre- and post-production techniques. It’s not enough to just shoot once and then draw everything. Taking an iterative approach is crucial, as the team must draw, review the site, adjust, and repeat until the right look is achieved. 

[RN]: A reliable, high-quality engine and I/O card that will deliver photorealistic renders with low-latency are critical and can ensure minimal latency between your image and tracking data. Using Carrot Broadcast’s real-time solutions with AJA Corvid 44 and Unreal Engine makes it easy for our team to work with graphics and virtual sets. Together, these solutions provide a powerful arsenal of tools for live AR production. 

Is there anything else you’d like to include or mention?
[RN]: In the process of developing our latest project, the ideas, discoveries and workflow that we’ve built provide a solid foundation for future projects. It seems we’ve opened a real Pandora’s box on the possibilities for AR development. 

For more details about Carrot Broadcast, visit: https://www.carrot.software/eng.