5 things you should know about VR Video?
So you want to shoot a video for VR? That’s wonderful! Before you rush out to record your latest opus, here’s some advice from a guy who spent the last few years in the VR trenches.
1. VR Video is not video, it is Virtual Reality.
The old rules of film and television developed over the last century need to be tossed aside when creating for VR. That may make some people uncomfortable to hear, especially someone like me who has spent the vast majority of my professional career mastering the art and craft of filmmaking. Don’t panic. Film & television are not going away anytime soon. Just as people still read novels and listen to music, these visual mediums will exist alongside VR. In fact they will most likely be adopted inside VR. But I am getting ahead of myself. I am sure you are asking, “who the hell do you think you are with these prophecies of doom?” Who am I? I am your guide to VR Video, that’s who. So let’s start your journey by recapping mine.
First Steps Into a Larger World
My first experience with VR Video was in 2018 when I was contracted by tech companies like Microsoft, Facebook & WhatsApp to shoot customer research videos around the world. Since it was impractical to fly an entire engineering staff from silicon valley to each global market to experience the culture, they sent a team of researchers and documentarians to conduct interviews and capture footage of each location. It was my job to document the “slice of life”.
So while my team was off interviewing dozens of people, I was huffing around (mostly on foot) with my Sony A6500, an assortment of lenses and an Insta360 One X, filming markets, public parks, cafes, places of worship & monuments, often zig-zagging around town in tuk-tuks, taxis, electric scooters & motorcycles.
One of my primary responsibilities was to shoot & edit 360 videos recorded on the Insta360 One X for the Oculus headset. I ended up producing 40 short videos in over 7 countries. It was 2018 and the 360 Camera technology wasn’t nearly as far along as today. The camera resolution was 4K, which sounds impressive until you realize that the viewing space is 360 degrees. Spread 4K resolution in a VR space and the quality drops exponentially. It was a novel way to transport the engineers into the physical spaces of these communities, but because the resolution was so low, it still felt like watching a video.
It was an invaluable experience for me on how to shoot & edit in VR but I didn’t see these videos as anything more than a gimmick. Maybe they were useful for real estate or tourism but I didn’t see much appeal beyond these industries. After the contract was up I didn’t even purchase my own Oculus. I returned mine to the client and put the experience behind me.
Pivot
Fast forward to 2020. Covid-19 just kicked off and all of my work had vanished. Now that travel is out of the picture, I needed to figure out the next evolution of my career. That’s when my friend Peter sent me an Oculus Quest 2 and had me watch VR 180 videos in 8K. The video was a short behind-the-scenes documentary of a music video shoot. What I saw blew me away. This was not the same technology I experienced 2 years prior.
The view was cut in half (180 not 360) but the image was crystal clear: no artifacting or pixelation. And since it was shot stereoscopically — two camera lenses mimicking the distance between human eyes — everything in the frame looked real. Like really “real”. The host appeared in 3D, as if she was in the same room as me. That’s when I saw the potential, and the challenges ahead.
2: Why Is 3D VR Different? Because there is no way to cheat.
My entire approach to filmmaking was to focus on what the camera captured in the frame. Anything outside of the frame was irrelevant. It allows filmmakers to hide lights, boom poles, tripods, sound blankets, etc. That approach does not fly in VR. The camera sees all.
The host had professional hair & makeup for the music video. I am sure she looked stunning in traditional 2D. But in 3D VR, she appeared caked in makeup. She was not a flat image manipulated with light and shadow. 3D VR is completely unforgiving.
3: Editors Beware
No more quick cutting. Since the viewer is grounded in a space, if the image cuts quickly it is extremely jarring and takes a few seconds to orientate oneself. Out of necessity I have leaned on dissolves and fades when transitioning between shots because the experience can be so off-putting. The role of the editor will be sidelined and maybe even obsolete in VR.
4: Sea Sickness
After I unwrapped my Quest 2 I eagerly explored Oculus TV (now Meta TV), the built-in video channel on the headset, to see what other creators were doing with VR Video. I was underwhelmed. Maybe because the camera technology to capture 8K video is expensive and difficult to find, but the offerings were very limited.
One of the most striking limitations of the medium is the lack of camera movement. This is done strategically because no matter how smooth the dolly track or camera pan, the motion can make the viewer instantly nauseous. Of course some viewers are more hardy than others, but immersion in a VR world tricks the eyes and ears that we are present in another space, and when that space is moving but our inner ear thinks differently, the effect can make you feel similar to riding on a boat in rough seas. Flying through the Grand Canyon in 3D is an incredible experience although when I took off the headset it took me a few minutes to allow the dizziness to pass.
5: Interactivity
Feeling present in a space is so integral to VR that a passive experience like watching a video, no matter how engaging the storytelling, leaves the viewer somewhat unfulfilled. My business partner and I purchased the K2 Pro Camera with iZugar fisheye lenses, basically two Z Cams synced together with a case built around them. It’s an impressive piece of camera technology that can capture 2 x 4K video images and record natively with Apple Pro Res HQ codec. That’s why the image is so clean, because the video is not compressed until mastering the H265 file for the Oculus headset.
One of our first clients was Portland Seed Fund. For their 10 Year Anniversary I interviewed CEO’s about their Entrepreneurial journey and asked them to bring an artifact to show off to the camera. At the Anniversary party we brought 5 headsets and let the guests watch the videos. Almost every instance where a guest tried out 3D VR Video for the first time, they would either try to step into the virtual space or reach out to grab a virtual object in front of them. VR demands interactivity with the environment. It’s what sets the experience apart from passive, 2D video.
What Now?
My prediction is that VR video in the near future will be combined in a hybrid form with more game technology. LIDAR will be used to scan 3D wireframes of a subject and the images captured will be mapped back onto those wireframes. Now the viewer can “walk-through” or interact with objects and people. Not only does this create more interactivity possibilities, but the rendered experience will take a fraction of the storage required for high resolution video. The video file size challenges with traditional 8k video (and beyond) are very real. A short video project can take up to a terabyte of hard drive space. And that’s just for 8K. We are already seeing 12K cameras on the market. Video games are vector renders and take 100 x less storage and can scale up to future, higher resolutions. It’s a new frontier with more collaboration between software developers and 3D artists, but the experience will be breathtaking. The “Orson Welles” of this new medium has not made “Citizen Kane” yet, but it’s coming. A new, more technical breed of directors will lead video to the next evolution of storytelling and participation.
In Conclusion
I hope I inspired you, dear reader, into exploring the world of VR video. We are witnessing the birth of a brand new medium and I am thrilled to be a part of it. I will continue to post excerpts from my journey. I recommend you check out some of my VR videos on our Oculus Channel https://creator.oculus.com/community/372168154364296/ or our YouTube channel https://youtube.com/playlist?list=PLIGzxO4-C1BUwCIAqceS9g-RZ2kEGj2Bo
And if you’re curious what writing will be like for this new medium, check out my previous post on Storytelling in the Metaverse.
