Virtual Reality and Storytelling: What Happens When Art and Technology Collide?

Tech giants Oculus and Valve have declared 2016 the year of Virtual Reality (VR). In the past six months, both companies stormed into the consumer marketplace, offering the first two high-quality — and highly functional — mass-market virtual reality devices: the Oculus Rift and the HTC Vive. Although VR sits poised for a mainstream explosion, it’s far from the new kid on the block; inadequate technology has thwarted the ever-pending VR revolution for 25 years.

When it comes to placing VR in homes all around North America, The Oculus Rift and HTC Vive do raise some red flags. Both pieces of hardware are prohibitively expensive, require high-end computers to run efficiently, and nobody looks cool stumbling around in a VR headset. VR’s high cost of entry and outright physical dorkiness mean it will be a while before VR has its Pokémon Go moment. Even with the proliferation of cheaper, more accessible options (The Gear VR and Google Cardboard), VR still has the potential to tumble back into obscurity.

VR does have one clear mega-weapon in its arsenal, though. The technology finally works as intended. Modern day VR works like gangbusters. The technology has come a long way from its Lawnmower Man days. One needs only spend a few moments immersed in a VR world to understand the medium’s potential. Sadly, unless you’re a VR enthusiast with several thousand dollars to burn, it will be a while before most people have their own high-end VR headsets.

This summer, the TIFF Bell Lightbox team are bringing VR to the public. POP, which takes place at TIFF Bell Lightbox, “is a three-part virtual reality and immersive media pop-up installation series showcasing some of the best virtual storytelling in the world.” The POP series is divided into three groups: POP 01: VR + Music + Art (June 24 to 26), POP 02: VR + Empathy + Real-World Storytelling (July 15 to17), and POP 03: VR + Experimental Film (August 19 to 21).

In addition to each pop-up installation, TIFF is featuring POPx on Saturday evenings. POPx provides attendees a chance to sit in on panel discussions featuring VR content creators.

To the uninitiated, VR is a glorified video game peripheral (or world’s greatest gift to pornography). With POP, TIFF is offering the public an amalgamation of art and technology. Through POP, artists, musicians, and filmmakers introduce lovers of movies, music, and gaming to a new medium, one that proves that pixels and MHz are just as important as paintbrushes and canvases. The installations offer the added benefit of planting seeds of inspiration in the minds of future content creators.

TIFF invited PopMatters out for the POP 02 media day last month and I was fortunate enough to test out several cutting edges experiences that blended art and technology.

Irrational Exuberance by Ben Vance (HTC Vive)

The HTC Vive currently offers the market’s most unique VR experience. In most VR rigs, users must sit down (preferably in a swivel chair). The Vive enables users to physically move around the room — think Star Trek: The Next Generation‘s holodeck. Vive users also hold a controller in each hand which renders their hands inside VR. Watching your virtual hands move in precise visual-spatial relation to your real life hands tricks your brain into a deeper level of immersion. Putting on a VR headset is akin to looking through a window into another world. Walking around and moving your hands in VR is like free-falling down a rabbit hole into a new dimension.

Irrational Exuberance takes full advantage of the Vive’s unique hardware. The result is an immersive and surreal experience unique to VR. Upon entering Irrational Exuberance‘s world, I found myself encased in rock. I soon realized that swinging my hands like virtual hammers broke chunks away from my stone prison. As I flailed around in 360 degrees, I felt like a 6’3” chick breaking free of a giant stone egg. To anyone in the room (aside from the handler walking me through the experience), I must have looked like an oddball, less an adept VR enthusiast than someone doing their version of the Elaine dance.

As I finally broke free of my stone cocoon I realized that I was hurtling through space on an asteroid. The moment was breathtaking. I felt like the Star Child in 2001: A Space Odyssey, a newborn entity gazing upon the wonders of the cosmos. A giant planet took up a huge portion of my field of vision. When I say a huge portion, I mean impossibly huge. It was like standing next to an IMAX screen. I had to physically turn my body around to see past it. The best way to describe the scale of objects inside VR is to compare them to looking straight up at a skyscraper. Put it this way: while watching TV you are a giant staring at a screen, in VR you are an ant riding atop of a locomotive.

As I stood, mouth agape, marveling at the cosmic beauty all around me, I passed through a debris field. The chunks floating my way were some type of pinkish crystals. As I moved my arms in the real world, swatting the debris away in VR, the crystals made shrill little pings every time that they made contact with my hands. I knew these crystals only existed in virtual space, but I could feel a physical difference between swatting crystals and hammering the rock. It’s as if my mind filled in the gaps for the tactile senses that it thought I was using.

My experience inside Irrational Exuberance was beautiful and serene. Afterwards, I couldn’t help but wonder how we may use this tech to reduce tension and address anxieties. Despite being in a loud room filled with journalists, I exited Irrational Exuberance feeling calm and centered. Calm and centered is the complete opposite of how I felt lining up to enter an early morning media session an hour earlier. I left the experience thinking that someone could come home after a stressful day, slip into Irrational Exuberance and find relief. I can see myself inside VR listening to some psych-rock while meditating on an asteroid.

Never Forget: An Architecture of Memory by Nicole Del Medico (Oculus Rift)

Never Forget offered one of the day’s more surreal experiences. During Never Forget, users wander around the memories of 93-year old Anna van Vliet. It’s a haunting experience that feels like passing through someone else’s dream. I traversed through Anna’s mind with no clear objectives, markers, or waypoints. Everyone who enters Never Forget will have a different experience; which is exactly the point.

Never Forget’s designer, Nicole Del Medico, was on hand to discuss her creation. As a lifelong video game player, Nicole was excited by the artistic opportunities VR presented.

“The real driving factor is that I do see video games as an art form,” she told me. “Not to say that necessarily the piece I created is a video game. But I find that the opportunity that VR affords creators to create new narratives in a way that can be architecturally explored [is something] I find really fascinating and engaging. And I think that’s a really worthwhile investigation.”

Del Medico on the challenges of creating a narrative in a 360-degree environment:

It’s very challenging, the way I addressed it in my piece in particular. Because I’m dealing with memory, I just considered how we experience memory. Sometimes we experience memory in a linear fashion, other times we recall memory in different orders. So because this is a piece that’s dealing with the memories of a 93-year-old Dutch women, I treated it as a relatively linear experience but there are side branching narratives and memory spaces. Each space is essentially a memory and you are navigating through them and you don’t have to navigate them in any specific order. So some people visiting the installation might actually be traveling backwards in time and experiencing memories in different orders than they would be in a traditional narrative.

Del Medico on the mainstream accepting VR as an art form:

I think with events like this [POP], it will come into that light more. Hopefully, as time goes on the accessibility will increase as far as hardware and software. As soon as that becomes more successful, we’ll see more indie developers getting their hands on it, and I think they’re the ones that are going to drive the medium forward rather than the triple-A developers because I feel like the level of risk is different for them. I feel like we’re more willing to take the risk.

Del Medico on linear narratives versus environmental experiences:

I think the interesting thing with VR is, as I’ve noticed with some of the artists here and going to the other conventions, is everyone’s approach to VR is different. There’s really no right or wrong formula because the medium is so young. With my piece, I feel it is a hybrid piece. Definitely, there is a narrative and it’s driven by the narrative, but at the same time, it’s exploratory. So it’s kind of a hybrid piece, and I feel that there is a few pieces in here that are hybrid. You have to grapple with that, and I don’t feel that there is a right or wrong with that.

Henry by Oculus Story Studio (Oculus Rift)

If there was one display that spoke to the medium’s immediate future, it was Henry on the Oculus Rift. Henry is an 11- minute animated feature with stunning production values. The Henry “experience” has the candy-coated sheen of a Pixar movie, is narrated by Elijah Wood, and engineers at Skywalker Ranch handled its audio.

The experience offers the user an up close look at the story of Henry, an anthropomorphic hedgehog who is celebrating his birthday. I received a front row seat to a party inside Henry’s quaint little tree-house home.

His world is a technical marvel. It’s filled with rich and vibrant colors that make the three-dimensional environment pop with a real sense of depth. Think of the best 3D movie that you ever watched and multiply that by 10. Henry’s high fidelity 360-degree sound is so crisp that you can close your eyes and follow where inside the room the action is taking place. What does all this technobabble mean? At times, my mind believed I was inside a cartoon. While Henry’s cutting edge technology made for an engrossing other-worldly experience, there are two specific moments that hammered home what makes VR such a game changer.

As the experience began, Henry stepped into the tree-house, turned his head, and met my gaze. Later on, during the stories’ emotional low point, a depressed Henry, once again locked eyes with me. On television, when a character stares into the camera, it breaks the fourth wall, destroying immersion. In VR, Henry’s eye contact had the opposite effect. It affected me on a visceral level. I tuned in. Not only did I lock in emotionally, I also felt a sinking feeling in my stomach for this sad little character. The closest thing that I can equate it to is sitting front row at the theater and having Lady Macbeth lock eyes with you during a soliloquy. When Henry looked me in the eye, he demanded my attention, and I couldn’t help but give it to him.

VR is the future, the only question is how soon. As an early adopter, I’ve used every available opportunity to share the tech with the people around me. VR has wowed everyone from my pre-teen nieces and nephews to my technophobic aunt that still uses a VCR. If the proliferation of LCD TVs and smartphones are any indication of VR’s evolution, third and fourth generation Vives and Rifts will come out packing some Hulk-sized punch at Ant-Man sized prices.

Right now, hardware developers and content creators are in the same newbie boat. Developers are still learning how to program fully textured worlds while figuring out the new rules for level design. Even filmmakers have to adapt to a new cinematic language — how does a DP go about composing shots in 360-degree environments? How does one block the actors in a scene, position extras, or even light a set that encompasses everything?

After two decades of intrigue, VR tech is finally catching up to its artistic ambitions. Now it’s a matter of getting VR into the public’s hands and allowing it to grow. This fall, Google, Samsung, and Sony will all have relatively affordable VR in the homes of their large user bases. Factor in cultural institutions like TIFF educating the public on VR’s artistic potential and mainstream dominance feels inevitable. After all, where else can you spend a morning riding comets, walking through dreams, and empathizing with a cartoon hedgehog?