VR at SIGGRAPH: Reliving Moments of Truth

***This article originally appeared in the August ’21 issue of Animation Magazine (No. 312)***

With the SIGGRAPH confab underway August 9-13, we thought it would be a good time to spotlight some of this year’s buzzy VR titles and catch up with the brilliant creatives who have been raising the bar of this innovative medium with each new project. Not surprisingly, personal and more socially aware topics are dominating the field this year:

Replacements
Directed by Jonathan Hagard
Produced by Andreas Hartmann, Nova Dewi Setiabudi
Co-production of Kampung Ayu, Ossa Film, Suwe Ora Jamu

Jonathan Hagard’s Replacements, which won top honors both at Annecy and the Venice International Film Festival, is a follow-up to his 2009 film Time Lapse, which chronicles the changes experienced by a small village in Jakarta. After the Islamic parties won the 2017 elections, the country went through more radical urban and cultural changes, so Hagard decided to cast his net wider and capture how a small community evolves through four decades.

“My grandmother once told me, it’s important to remember your roots to understand the future,” says Hagard. “My response to the 2017 event was to make a new story inspired by this universal and human philosophy to remember and appreciate an Indonesian typical alley and how it changed in four decades. On one hand, I was inspired by my own family’s story and their house in central Java to add more emotions and nostalgia to the main protagonists. I also used the results of more serious urban researches I made in Jakarta’s alleys for about 14 years, to enhance its documentary and objective aspects.”

In 2018, after making several trips to Indonesia, Hagard completed an initial version of the project and his first VR prototype, which convinced him that the medium was a perfect way to tell this particular story. “That was the beginning of two years of intense production,” he recalls. “In the first year, I mainly focused on recreating the eras between 1980 and 2020 and the environments based on personal memories and actual archives. During the second year, I was more dedicated to the animation, post work and on deciding and shaping the final scene, which was actually different from the initial storyboard, due to the evolution of Jakarta’s situation and a certain pandemic.”

The filmmaker first created digital paintings of the village throughout the decades as wide panoramas without focusing on one main element, which he compares to Asian traditional paintings or a 360-degree Japanese ukiyo-e. “Although I first thought about creating the animation in CG, I finally decided to make it in 2D to enhance its handmade and blurry, memory-like qualities,” he notes. “It was a mix of traditional 2D animation and rotoscope on top of 360 videos of actual people and actors in Jakarta.” He also decided to switch from stereo to ambisonic (full-sphere surround) spatial sound to add more depth to the VR experience.

Hagard says he was attracted to the VR medium because of the freedom of exploration it offered him. “Replacements is a linear film, but it is full of small stories and elements that you cannot watch at the same time,” he explains. “Just like real life, you have to choose which element to focus on and which element to skip before it fades away in time. Just the fact of being able to turn your head and discover new stories and people —sometimes by choice, sometimes by accident— can be a great storytelling technique. Like one of Jakarta’s inhabitants, the viewers are challenged by their inability to focus on everything at the same time, they share their moods, routines and frustrations in front of the complexity of the surrounding space and the speed of the passing time.”

The filmmaker tells us that he counts great Japanese masters such as Satoshi Kon, Isao Takahata and Hayao Miyazaki as three of his biggest sources of inspiration. He adds, “I am also very inspired by illustrators such as Nicolas de Crécy and, of course, Moebius. Outside the animation world, Jacques Tati is one of my biggest inspirations for the attention to details, the multiplicity of small stories and the absence of dialogue. In the VR world, Patrick Osborne’s Pearl is my favorite: It is not just beautiful, but it fully takes advantage of the 360 environment and is still until now one of the few VR pieces that almost made me cry.”

Hagard has a crucial piece of advice for future VR filmmakers. “I would say don’t use an idea that you could make in a conventional film and then convert it to a VR film because it would be too much work and it won’t add much in terms of storytelling. I think VR storytelling should be detached from other storytelling mediums. Think about why your story is the right one to explore in VR, more than any other storytelling methods.”

And what does he predict for the future of the medium? “Unfortunately, VR is still currently not so accessible,” he responds. “I hope with the arrival of lighter, easy to use VR sets, this technology will be more democratized. VR can be not only a way to immerse ourselves into a fictional world, but I hope it can also help us to relearn truths about our actual environments.”

Website: cargocollective.com/jonathanhagard/replacements

Namoo
Directed by Erick Oh
Produced by Larry Cutler, Eric Darnell, Maureen Fan, Kane Lee
Baobab Studios

Acclaimed animation director Erick Oh’s new VR project Namoo (Tree) is the perfect example of how the medium can be used to tell more personal, intimate and stylized stories. The short, which was produced by Baobab Studios (Baba Yaga, Crow: The Legend) and developed both as a regular theatrical project and a VR experience, was inspired by the passing of his own grandfather. The short played at Sundance and Tribeca earlier this year, and will be part of SIGGRAPH’s program in August.

“Namoo is deeply inspired by the loss of my grandfather about 10 years ago,” says the Korean-born writer-director, who was nominated for an Oscar for his short Opera earlier this year. “It took me a while to finally put this together in an art piece because I wasn’t ready to share this idea with the world yet. It was actually a sudden goodbye, so we weren’t prepared and experienced this deep grief. That experience led me to think about our lives and the meaning of it all. But even after my grandfather’s passing, I still feel his presence. I feel I’m more spiritually connected to him now. He’s still my guiding star and that led me to come up with the core idea of Namoo, which is about the celebration of life. There’s beginning and there’s always the end. but the ending connects back to another beginning.”

Oh, who has worked at Pixar and Tonka House, says he embraced all the possibilities the virtual reality medium had to offer his narrative poem. “VR brings you deeply into your story,” he explains. “You become a part of the story on a very personal level. From the first moment I decided to bring this idea out to the world, I wanted this to be a very poetic spiritual experience and VR was the perfect platform for it. In order to make both the theatrical and VR versions successful, I had to understand the strength and characteristics of each medium deeply. That was the biggest challenge — but also the most exciting part of the production.”

“We have known Erick for a long time and love his previous works at Tonka (Dam Keeper),” says the short’s exec producer Larry Cutler, who is also Baobab’s CTO. “When he approached us about this project, we were very excited because he wanted to create this piece both as a 2D film and VR piece, using the Quill animation tool. Namoo covers a man’s life from birth to death through a poetic use of a tree, which represents his life and longings. It really ripped our hearts in a way, and we know we wanted to make this project with him. Our goal at Baobab has always been to bring in great filmmakers and give them the opportunity to tell their story in different ways.”

Oh points out that while VR projects have a tendency to be a bit techy and rigid, he wanted to bring a handmade and warm feeling to his short. “That’s why I thought of Quill, the VR animation software developed by Oculus,” he explains. “It lets the artist be able to paint, draw and animate intuitively in virtual space. Then we were able to coordinate a small but amazing team of Quill artists from all over the world. We were sending progress and notes to each other in VR, and that was an amazing experience.”

Looking back at the experience of making Namoo, which took less than a year to complete, Oh is optimistic about the possibilities of the medium. “It was challenging to make both 2D and theatrical versions from the same story and the world,” he points out. “But by doing that, we were able to learn so much about both mediums. In VR, you invite the audience into your world and let them explore. This is incredible, but it also means that you can’t control the camera the way you would in a traditional format. Understanding this difference actually made me think much more deeply about not only VR but also cinema. I definitely feel like I became a better storyteller and filmmaker after Namoo, and I can’t wait to apply what I learned to my next projects!”

Websites: erickoh.com/namoo
baobabstudios.com/namoo

The Changing Same
Directed by Yasmin Elayat, Michèle Stephenson and Joe Brewster
Produced by Scatter and Rada Studio

It’s easy to see why New York-based studio Scatter’s follow up to its first VR experience Blackout picked up the Best Narrative Prize at the 2021 Tribeca Film Festival: Their powerful new project The Changing Same allows participants to travel through time and space to witness the connected experiences of racial injustice in America.

Director Yasmin Elayat first met her collaborators Michèle Stephenson and Joe Brewster five years ago when they were working on their documentary, The Changing Same. “The documentary chronicled the town of Marianna in the Florida panhandle, which was the site of the Claude Neal lynching in 1934, which is considered one of the worst spectacle lynchings in the U.S.,” notes Elayat. “That film, this town and the story of Claude Neal is the origin of the virtual reality series. Over the years and in the co-creation process it’s turned into ‘hybrid speculative fiction,’ where the story is based on real events that have become abstracted.”

Brewster mentions that his own personal experiences informed this important project. “This journey began as our own personal journeys,” he notes. “Before this project, not only did I struggle with obstacles related to class and race as a Black middle class male, but I struggled to understand my privilege and the reasons why some of my friends ‘made it’ while others were destroyed by systemic racism and injustice. The Changing Same is my narrative and like most ‘makers,’ I share my story and hope that it inspires and informs others around me.”

According to the directors, the inspiration behind their project was that history is cyclical and that racial injustice hasn’t really changed, but has evolved. “Our goal with this project is for the audience to acknowledge our shared history, to bear witness to the past in order to heal and move forward,” says Elayat. “It was important to memorialize the unmarked sites of the Claude Neal lynching, for example, and it was also important to showcase Black joy, daily life and celebration. Part of the approach to the world building and the creation of the virtual worlds was capturing the actual sites in the real world. We sent a team to Marianna to do a reality capture shoot, and using photogrammetry we captured the landscape and the sites. We then re-contextualized these real-world places and re-imagined them.”

The project brought together a team of international designers, technical artists, engineers, a reality capture team and volumetric specialists, Depthkit experts, lighting environmental artists and others. Episode one went into production last year, and it premiered at the Sundance Film Festival earlier this year and was part of Tribeca’s Juneteenth curation as well. “Every concept, the characters, the environments and the mechanics we designed with intent to leverage the medium,” says Elayat. “With VR, you have multiple experiential tools to tell a story. You can completely restrict and confine the user’s space, and you can be completely expansive with camera moves and physics to add a whole other experiential dimension. For example, our time travel mechanic where you travel through this abstract symbolic space spanning 400 years of Black History, is meant to feel awe-inspiring and infinite. It has many layers and symbols and Easter eggs to discover each time you travel through it.”

Brewster and Elayat are both pleased that they were able to get their project off the ground during such a difficult year. “We resisted changing the content to make the story less painful,” says Brewster. “We resisted decreasing the complexity and size of the build. We made the story more painful, and that required more joy — a solution that makes the narrative stronger. We increased the complexity and the original build was close to 7GB, but we were again advantaged by this. I am most pleased that we remained patient and we are still happy artistically, as well!”

Adds Elayat, “I am quite proud of the strong storytelling, the uncompromising subject matter, the technological innovation and the beauty of the world’s we’ve designed.”

Website: scatter.nyc/productions/the-changing-same