In 2016, when Google Spotlight Stories’ Pearl became the first VR piece to land an Oscar nomination for Best Animated Short, many animators took notice. “After watching Patrick Osborne’s Pearl I did two things,” says Jorge Gutiérrez, director of the Golden Globe-nominated feature The Book of Life and the co-creator of Nickelodeon’s Emmy-winning series El Tigre: The Adventures of Manny Rivera. “First I cried, and then I said, ‘Can I watch it again, please?’ I’d connected on an emotional level that I’d never felt before. If we can make people feel things, that is the best special effect. I knew I wanted to do one of these.”
Gutiérrez got his wish, completing his first VR piece for Google Spotlight Stories, Son of Jaguar, which puts viewers inside the POV of a Mexican wrestler. “When wrestlers put on a mask they become someone else. That’s what we do in VR. When you put the headset on, you become someone else. It also made me feel like a ghost, because I’m watching things that can’t interact. That led to the idea of a Mexican wrestler ghost who on the Day of the Dead comes down to visit his family. That’s where the inspiration came from.”
Gutiérrez tackled the project alongside Reel FX in North Hollywood, Calif., his collaborators on The Book of Life. Gutiérrez admits, “VR was a lot harder and crazier than I ever imagined. I was very skeptical of the medium, but I can say that we jumped out of the plane and made the parachute as we were falling!”
The talented Mexican-born director recalls that the real-time rendering in VR was hard. “But the audience still wants it to look beautifully textured. In animation, we’re used to controlling everything—especially cutting and framing. In VR, you lose both of those things. I said I wanted to be able to do those things. So we figured out how to cut and figured out a way to… not ‘frame’ things, but to make you look at certain things.”
“The big ‘aha’ moment for me was thinking about it like theater,” Gutiérrez notes. “A theater director directs the audience with light and sound and by telling the actors to walk to a certain place. And wherever you’re sitting in the theater, you’re going to get your own version of the story. So instead of looking to the future, I looked to the past.”
Working in Abstract Environments
Notably, a similar aesthetic was followed by Scot Stafford, who co-directed the dreamlike VR short Sonaria with Kevin Dart of Chromosphere. “We’re transporting you through 10 totally different environments, which is hard to do in VR,” explains Stafford. “We thought, ‘What if we approached it like theatrical lighting and scene changes?’ Chromosphere took that idea and ran with it.”
Stafford, through his company Pollen Music Group in San Francisco, has worked on the sound for several Google Spotlight Stories, including Pearl and the Aardman Animations mobile piece Special Delivery. Sonaria was Stafford’s first co-directing effort, but he’s been involved in VR since 2012. He likens that to “dog years,” saying it’s felt like 35 years.
Sonaria was new territory, however, for the animators at Chromosphere, notes Kevin Dart. His L.A.-based studio, which has done animation for clients including Disney and the television series Cosmos, hadn’t tackled a complex VR challenge before. “Getting a project from Google that was meant to be an experimental art piece was a dream,” says Dart. “That’s something we usually do on our own time.”
The collaboration between Stafford and Dart actually was facilitated by Google Spotlight’s tech lead, Rachid El Guerrab. “Rachid showed me Kevin’s film Forms in Nature,” recalls Stafford. “I was thunderstruck. It was a really interesting exploration of forms and shapes.”
When the directors began communicating, Sonaria evolved into an abstract exploration of diverse environments, from deep-sea dives with phosphorescent creatures to high forest canopies and bat caves. Stafford says this required extensive audio tool development. “We needed to have full spheres of sound both above and below.”
Audible Delights
To achieve this, Pollen Music worked not only with Google engineers but also UC San Diego’s Sonic Arts Department. “Sound plays a huge role in VR. Sound can tell you where you are and what you’re looking at. You can only see a small minority of the world at any given time, but you can hear all of it,” notes Stafford. As Google’s El Guerrab explains, “We pushed the idea of real-time filtering, and blending between different 360 audio fields.”
Watching Sonaria on an HTC Vive headset allows the viewer six Degrees of Freedom. Stafford remarks, “You can stand up or sit down or step to the right and have a much more immersive experience.” But a naive viewer can also end up feeling dizzy. And for animators coming from a traditional background, 6DOF means they can’t animate to camera. Stafford observes, “There are new challenges that you would never see in film.”
Even the traditional way of storyboarding a linear piece is not applicable in VR. So Chromosphere had to develop a new way of doing a 360 storyboard. Dart also notes, “We had to figure out how to preview animation from an animator. They haven’t quite invented a pipeline yet to do effective VR animation review.”
That’s one of several reasons why El Guerrab calls these projects experiments: “Every show allows us to explore our tech a little bit more.” Which is why Google Spotlight Stories has worked on as many projects that don’t ship as those that do. Both Sonaria and Son of Jaguar have been seen at festivals like SIGGRAPH, Annecy, Pixelatl’s El Festival, and the Future of Storytelling conference.
Field of Vision
The growing network of VR animation artists might arguably be as crucial to the development of this medium as the technology is. Google has already worked with legendary 2D Disney animator Glen Keane (The Little Mermaid, Beauty and the Beast, Tangled) to produce the mobile 360 short Duet, and Disney Animated Short Oscar winner John Kahrs (Paperman) is currently working on his Spotlight Stories VR project. Add to this that Spotlight’s creative leadership includes Jan Pinkava (Animated Short Oscar winner for Pixar’s Geri’s Game) and you can see the seriousness of Google’s efforts.
El Guerrab adds, “We hope to showcase to creators the many kinds of things that are now possible in VR, and why they should create content for this medium. They can do things that aren’t possible in a theater with surround sound.”
“It’s great to have a patron like Google without any expectations other than driving the technology forward,” says Stafford. It is surely in the self-interest of deep-pocketed Google to jump-start the VR movement. As El Guerrab puts it, “This is an age when a lot of digital companies are becoming content companies. We can’t afford not to do it. I think companies that have the ability to invest in this are the ones who should.”
It’s interesting to observe that 2D animators are welcomed in Google’s VR fold, a reflection of agnosticism about traditional forms. While Sonaria is primarily 2D animation, Dart asserts, “We’re not trying to rescue 2D styles.”
Gutiérrez also reminds us, “I come from 2D. I had El Tigre on Nickelodeon. I think 2D people will love working in VR. I’m now waiting for stop-motion people to join the movement.”
Google Spotlight Stories’ latest short, Son of Jaguar is now rolling out exclusively on Pixel 2 via the Google Spotlight Stories app, and has pushed the boundaries of what can be achieved visually and sonically on the Pixel 2. The piece will soon be viewable on Daydream, and will arrive on Steam for HTC Vive at the end of October. It will be released for the Google Spotlight Stories app on Android and iOS later this year.
For more info, visit atap.google.com/spotlight-stories.
