VR Post Production Challenges

VR Post ProductionVR Post Production Challenges

VR post production is growing rapidly. There are many challenges that not only face physical production video collaboration tools(see 360 Degree Camera: VR Challenges), but now that narrative content is being considered, VR post production eventually will become something more for editors to become familiar with. As technology continues to advance, my friend Neil Smith continues to move right along with it, designing systems that can keep with the pace. In this special post, Kahlani Leon interviews Neil, who provides us with an interesting perspective of the VR scene. We hope to give you footage from E3 2015 and even more notably, Oculus Rift in the coming weeks so keep us on your radar.

— Stu Brown

Post-production Challenges for Delivering Narrative Content in the VR Age

By Kahlani Leon KL_mail_photo.small

Los Angeles. June 2015

As we all know ‘Content is King’ and in the wonderful world of Virtual Reality narrative this is even more so.

Story is story, but the means by which we tell those stories keeps changing. From charcoal drawings on ancient cave walls to cinematic vistas projected onto giant IMAX screens, the technology of story keeps evolving and changing.

Post-production professionals have always been at the forefront of technological innovations. From splicing off flatbeds to multi-cam editing on NLEs, from SFX to VFX to CGI. From mono to stereo to surround sound. From black-and-white to color to 3D to VR, post-production has always braved the waters of innovative technology to keep visual story telling at the leading edge of consumer experience.

In the last year the art and craft of visual story telling has taken a revolutionary turn. Virtual Reality (VR) is threatening to not only revolutionize the video gaming industry but also dramatically impact narrative movie making. VR is coming to Hollywood! But new technologies demand new practices and here at Nonlinear Post we are always interested in tracking how new technologies impact the art and craft of digital content creation and distribution.

VR Post ProductionRecently, I had the opportunity to interview Neil Smith, CVO (Chief Visionary Officer) at Rift Valley Studios based in Los Angeles, California. Neil is a thirty-year veteran of the computer and movie industry. He first experienced VR in the mid 80s and now in 2015 he is steering Rift Valley Studios towards becoming a world leader in VR post-production and cost-effective VR workflow.

I interviewed Neil at the Rift Valley Studios in Los Angeles.

KL: Virtual Reality has come and gone several times over the last few decades – what’s different about this current wave of VR?

NS: In one word – PRESENCE! This new generation of VR technology from companies like Oculus Rift/Facebook and Samsung Gear generate a sense of ‘presence’ in the minds of the viewer that has never been achieved before.

KL: Presence? What exactly do you mean by ‘presence’ and why does it matter?

NS: When we watch 2D or 3D content on a movie screen or TV or PC monitor, we’re always aware that we are watching the story unfold before us on some kind of proscenium, some kind of stage or two dimensional screen that we are passively watching. The screen frames the contents and causes a sense of separation between us and the story. If we’re lucky the ‘willing suspension of disbelief’ takes over and we’re transported into the story world but there is still a frame that bounds the content we are watching. There’s always a separation between us, the viewer and that stuff over there, the content.

In the VR experience there is no screen between us and the content, no invisible fourth wall that frames the story. VR is a frameless viewing experience. When you’re in the VR HMD (Head Mounted Display) you are IN the content. You’re not watching the story but EXPERIENCING it directly on a deep neurological and visceral level. It is as if you are in the story, not watching from the sidelines but standing in the room with the characters. You can look around the room like the characters can, you can see behind objects. You can look up and down, round and round, behind you. You are no longer constrained by the 2D frame of the camera image. You are in the 360 degree spherical world just like you are in your real life.

Presence is the key that unlocks the power of VR.

KL: Wow. Sci-Fi writers and filmmakers have been talking about VR immersion for decades. Think ‘Total Recall’, ‘Minority Report’ or ‘The Matrix’. Why is it only now that you think this wave of VR is different to what has come before?

NS: The confluence of several technological and economic vectors that have produced the ‘perfect VR Storm!’

KL: What do you mean?

NS: Firstly, video gaming technology has reached a high degree of technological sophistication at an incredibly low price point. Gaming engines nowadays can produce near cinematic content. Secondly, Moore’s Law has caught up with VR technology. For $400 you now buy a HMD that would have cost thousands of dollars twenty years ago. And thirdly, Palmer Luckey and the Oculus Rift team at Facebook have really captured the imagination of the young gaming generation and us old geezers in Hollywood. Silicon Valley has come to Hollywood in a big way.

Oculus Rift based VR gaming is going to be huge. Already there is more money spent on video gaming than on TV and Movies combined! The fact that Mark Zuckerberg purchased Oculus Rift for $2 billion dollars last year is a testament to the huge potential that VR gaming and VR mobile communications has to offer in the years to come.

KL: So what has VR gaming got to do with traditional Hollywood based movie making?

NS: A year ago, I would have said not much. But in twelve short months most of the Hollywood Studios have started to invest serious money and effort into experimenting with narrative VR storytelling. Several studios have already produced VR content to go along with their traditional movies. Fox Searchlight Pictures produced a short VR trailer of ‘Wild’ starring Reese Witherspoon. It’s a great example of how VR can take you into the story in a way that the 2D movie can’t. Samsung with their Gear VR mobile headset have a dedicated VR app where you can watch VR movie trailers and shorts in a VR cinema.

KL: Interesting … so what do you think all this VR technology will mean for post professionals used to working in traditional 2D story telling space?

NS: Much pain and suffering! (Loud laughter).

But seriously, that’s a great question. Quite literally, everyone will have to start thinking outside the box – there is no longer a box! VR is a frameless media. There is now a 360-degree sphere. Fundamental assumptions like ‘front and back’ are going to have to be re-thought. We’ve all been trained over this last 100 years of cinema watching to think in terms of sitting in front of a screen and passively watching the story unfold before our eyes and ears. This is no longer the case. Editors are going to have to re-train their brains and start thinking in terms of spheres rather than rectangles. Cuts will be different. Framing will be a challenge. Where we think the viewer might be looking will be uncontrollable. The viewers will not necessarily be looking at the gun on the table when the bad guy enters the room. No more use of wide shots and close-ups to direct the viewer’s attention to what the Director thinks is important to the story at that point in the action. It is a whole new set of challenges for production and post-production.

Kl: What kind of challenges?

NS: All sorts. From where you hide the production crew and the equipment whilst filming to how you light a spherical scene. A 360-degree camera rig sees everything. You can’t have a boom mike above the actors’ heads. You can’t have a bounce board in the foreground. The amount of digital data that VR generates is enormous – handling that securely, stitching the video streams together, cutting spherical video, color timing and titles are all different in VR. Cool sound design for spherical viewing is a huge challenge.

Here at Rift Valley Studios we’re re-engineering the entire VR workflow to see where things can be simplified and made easier. Nothing is easy about cinematic VR but everyday we are finding new ways to help the creative community tell immersive and compelling VR stories that appeal to the imaginations of VR viewers around the world.

The main lesson we’ve learned over this last year is this – though VR post is the last thing you do, it should be one of the first things you think about. Start at the end and work backwards. Don’t just go out there and shoot a ton of VR footage and hope for the best. It won’t work. VR delivery is the most important thing to get right. We’ve had to develop a special mobile storage and stitching system to cope with the vast amounts of data that multi streams of video produce and for post-production reviews, we’ve developed the RVS ‘Iteration Engine’, a high-powered storage and compute platform that allows VR Directors, Producers and post-Artists to tweak and iterate through different versions of their content in real-time.

It’s only by continual iteration and checking of the content inside a VR head-set that you can really make sure that you’re achieving ‘presence’. If you take viewers out of presence for just a moment you take them out of VR experience just like that (clicks fingers). The RVS ‘Iteration Engine’ allows everyone to quickly and easily try out to different variations until they find the combination that works perfectly.

VR production and post is still in the early days of development. By the time that the consumer version of the Oculus Rift HMD ships in early 2016, we’ll have already moved onto new VR workflows and techniques. Every week brings something new. VR really is a paradigm shift in visual story telling.

KL: Exciting times! Where can readers go to find out more about Rift Valley Studios and your VR production and post-production services?

NS: Check out our website at www.RiftValleyStudios.com or feel free to drop me an email at Neil@riftvalleystudios.com

KL: Thank you for the fascinating overview. Anything else our readers should know about?

NS: Well next week (June 16th through 18th) here in LA is the annual E3 Expo. All the major video game companies will be there but there’s also going to be a strong showing of the latest VR technology. Oculus will have a large booth and though the emphasis will of course be on VR video games, a lot of the latest technology will have implications for the media and entertainment industry. Hope you’ll be there.

KL: Of course. Stu and I have already packed our gear and are heading over to downtown LA. It’s going to be an amazing week for sure. We’ll keep everyone posted!



You can be the first one to leave a comment.

Leave a Comment


CommentLuv badge


Follow me on Twitter

Facebook login by WP-FB-AutoConnect