Virtual reality (VR) and augmented reality (AR) will, one day, change the face of cinema. Despite the technological limitations of the mediums, they offer a profoundly immersive experience with the potential to reinvigorate the film industry.
With movie theaters just beginning to reopen from COVID-19 closures, the theater industry needs to find creative ways to boost attendance. While reopening and encouraging social distancing, “Theaters will operate at 30% capacity or less,” noted Catherine Henderson in the Denver Post.
Though VR, particularly, requires tech that’s years away from being available in major theaters, small venues have offered viewings for VR films via computer rooms. As the scalability improves, it’s something the industry will be watching closely.
Regal Entertainment is already offering AR experiences, recently debuting new AR games via their Regal app. They’ve been working with Moviebill to drive engagement through AR activations. Regal has doubled down in response to the pandemic, using gamification to keep their brand visible to their fan base.
To learn more about VR films and the artistry that goes into their creation, Innovation & Tech Today spoke with Disney’s first VR animated film director, Jeff Gipson. He debuted two AR short films this year: Cycles and Myth: A Frozen Tale. Both are now available on Disney+.
Innovation & Tech Today: You created Disney’s first VR short film Cycles. How did that opportunity come about?
Jeff Gipson: Disney has a program where anyone can pitch an idea to make a short film. So I pitched Cycles in the end of 2016 or early 2017, and was given the green light. It was funny because we had never made a VR short film at all. So when I put it in the idea, I was like, “If you don’t ask, the answer is no.” And then we were given the green light, it was kind of like, “Oh no! We’ve got to figure out how we can make it and get a team who knows how to work in VR.” I had never worked in VR before, so it was a gamble.
I&T Today: As the first production of its kind, you must have run into some major challenges.
Jeff Gipson: Yeah. I think that’s kind of what draws me, as well as a lot of the crew, to it — is that VR is such a new medium in terms of storytelling that it’s still kind of the Wild West.
There’s not a defined way of, “This is how you should make a film; this is how you can pull this shot; these are all the techniques…” So, trying to think about how we get those to work in VR was really exciting.
But, also finding the team. We had one person, our Head of Technology Jose Gomez, who was also in Boulder. He had worked a little bit in VR and we made some tools for the studio, several tools that the animators can actually be in VR and animate characters and explore sets early on.
He was excited to just dive in and figure out how we can actually create a pipeline and make a film. So he was key, through his leadership we got some great tech people together and managed to pull off the film. We had about four months from start to finish to create Cycles because it was in between Wreck-It Ralph or Moana and Ralph Breaks the Internet.
So we had a short little timeframe, but it was amazing just to create the whole workflow and then try to figure out how to tell a story, too. It was a very cool time.
I&T Today: How does directing a VR animated feature differ from a traditional animated feature?
Jeff Gipson: I think it always goes back to story. It’s always story first, whether it’s our feature films or a short film that you’ll see on TV. So the same idea was key for me.
The technology is secondary — we really focused on the story.
We fleshed that out, similar to how we would always do, but then trying to figure out, “How do you direct the audiences’ eye?” VR is really, really tricky because you can look everywhere.
So, we have a frame. To be on the screen, you have shots that are framed beautifully and the audience knows where exactly to look.
So, we tried to use similar techniques of color and lighting, and then design the set to help compose each story moment. And we also have an effect called the “Gomez effect” that Jose wrote, where it kind of darkens and saturates if you weren’t looking in the right area.
In the Cycles house, the story beats were all composed in a little area of the house. The house itself is designed to reinforce the composition of each one of those story beats.
I&T Today: Telling a story in four dimensions is such a departure from what we’re used to. When I watched Cycles at the Boulder International Film Festival in early March, I was surprised how organically my eyes were drawn to areas where the action was taking place.
Jeff Gipson: It’s very tricky. When we start our animated films, we always storyboard out the story first — each moment and rough animation, just to get a feel of how the scene flows. So we tried to do that similarly for Cycles and Myth where we would storyboard things flat, but we’re not going to see our film flat. So there’s a painting program called Quill that’s actually a VR painting program. So one of our artists is an amazing Quill painter, as well as animator, and he’s able to be in VR space and paint these characters.
What’s great is they’re actually volume metric. You get to see them from all the way around and it adds so much appeal and really gets a sense of how we can stage out the space, the flow of how close the characters come to us, and how far away they go — that’s another part of the language of VR.
Especially in Cycles, we start the characters really far away until it’s almost like they spiral closer and closer and closer until you are one — at the very end of Cycles. We have this very choreographed flow of how characters come close and far away and using space and scale. So that VR painting program and trying to storyboard things out really helps answer those questions early on.
I&T Today: That really came through. It was like you were getting to know the character in the beginning and you have that emotional distance. Then it felt very natural to find yourself in that first-person perspective at the end.
Jeff Gipson: Yeah, and that’s something special to VR . It’s something you wouldn’t be able to achieve on a flat screen. Those are also things that we try to think about, “Why are we telling this story in VR and how can we use this medium to enhance the storytelling?”
So for Cycles, that’s where you get the proximity to those characters, but then you become one of those characters. And then for Myth it’s still all about that space and all about the scale of those characters.
In Myth, we have some interaction where it’s not controller-based interaction, but it’s more headset based.
So if you come close to a character, this little salamander… if you’ve seen Frozen 2, there’s a little fire salamander… and if you go close to it in Myth it actually scurries away and will light on fire and kind of track with you and follow you.
So little things like that, just trying to enhance storytelling but not take you out of the film.
I&T Today: Would you elaborate a bit on how VR helps to create a stronger emotional connection? With VR, do you feel it’s more about eliciting an emotional connection from your audience?
Jeff Gipson: I think it’s like using the strengths of VR. When you’re in VR, you’re totally immersed in that world. So what’s cool about it, even with Myth, is the spatial sound.
We can place the sound in 3D space, so it sounds like it’s coming from your left or it feels like it’s coming from behind you. That’s a mechanism to help guide the eye, but it kind of adds to really being immersed in this space.
So I think if we were to create a film that only happens, say 15 feet from us in VR and we’re standing there, it’d feel very different than a film that uses the total VR space.
It gives you a feeling of being with the characters, and I think that’s making sure you’re using all the awesome things that VR has to provide using this total space.
It’s almost like theater in the round in a way, an inverted way where you can bring all the characters to you and this play happens around you. I think it’s kind of cool and unique to think of filmmaking and storytelling that way.
I&T Today: I wonder if your architecture background helped you to create that “theater in the round” experience in your films.
Jeff Gipson: Yeah, I totally give a lot of that credit to architecture and just feeling space and knowing if you go into a big atrium where the ceiling is 30 feet up above you, you get an emotional response from that versus those tight little coffee shops that have little nooks and warm wood, you get a different emotional response from the that versus the sterile lighting of a hospital versus the warm lighting of a nice restaurant.
There are all these things that our bodies react to in the background that are cool to try to work with in VR. So it’s definitely something I try to tune into.
I&T Today: When I think of a director, I suppose I think of more traditional film directors — but what does a day in the life of a VR animation director look like? How do you direct a team in virtual reality?
Jeff Gipson: I was telling my friend yesterday about just filmmaking, when we make a traditional film or short film or animated film, you get to see it roughed out in the editorial process where you’re storyboarding. But in VR, you’re hoping it’s all going to come together at the end.
So, there’s a lot of trust that I think the crew had on these projects. As the pipeline progresses, maybe there will be ways that we can see the whole film earlier on, but I think it’s really similar to how we would do an animation film.
We start from the story and then we design our characters and then we move into modeling, where artists model these characters in the environments in 3D. And then we have a department called “Look” that adds the textures.
They make chairs that look like wood have that wood grain, or those properties of skin, like the way skin reacts to light or leaves or flowers.
And then we have our character TD department that actually adds the bones to our CG characters, almost allowing the joints to move. They’re making the skeleton for the animators to actually move.
And then we have effects artists that add the explosions, the smoke, and the rain, plus lighters that do the final touches and make it feel like it’s day or night. So it depends where we are in the process of what I’d be doing day to day.
But a lot of it is just communicating with the team or answering questions and always making sure that they are all on the same page in terms of story or what the visual goal is for the day.
I&T Today: Viewing options are limited for these films, how are you finding new avenues for people to view the work? And what are Disney’s goals for the medium?
Jeff Gipson: For Cycles, it’s cool because we launched Disney+, so Cycles has a flat version that’s on Disney+ currently.
And that’s the beauty of the technology that we’re using to make Cycles and Myth, they allow us to create a VR film, but also the augmented reality portions, and we did several of those for Cycles as well as for Frozen, we created the AR poster.
But then, we can even do flat versions of these films. So maybe down the line there will be a flat version of Myth that can go on Disney+.
But, I think what’s so great right now, and I feel so fortunate to be at Disney animation right now, is that our studio leadership, Clark, Spencer and Jennifer Lee, they’re really forward thinking.
And right now, making these, the VR films isn’t really about monetization.
It’s more of, how can we continue to push storytelling and bring audiences into our world and be with our characters in new and different ways?
But even showing some of the executives at Disney company, it’s cool to see them be in these films and start to think about, “Wow, how can this affect my part of the business?”
Whether it’s the theme parks or whether it’s through publishing or whether it’s through events or cruises or whatever it might be. It’s cool that it’s starting to spark ideas of how this technology could inform different parts of the company. So it’s cool to be on the front edge of it all.
We’re trying to figure out how we release [these titles], because we want people to see them how they’re designed. It’s great to have a flat version, but it’s also, “How do we get them to our audiences in one of the VR stores or other methods that would allow them to have these films at home?”
So, we’re still working on some things and hopefully we’ll have some announcements soon on that.
I&T Today: I think it’s something a lot, or maybe all, of VR is running into — how do they get their content into the hands of regular consumers?
Jeff Gipson: I think it’s tricky, too, because a lot of VR is seen as just gaming, but… take my mom for example, because she was kind of hesitant at first, thinking “Oh, it’s just gaming.”
But the moment she got into the films and was in these worlds, it’s cool to see her like, “Whoa, I get it now!” This is something really powerful to experience in a film.
And so, even just general audiences, as we showed people at Boulder Film Festival, there was this elderly lady, she was probably 75 and she had never done VR and she [was watching] Myth.
That film is more of a visual poem, kind of a Fantasia type film, and she was just smiling and almost dancing a bit.
It’s cool to have that, for those in the general audience to have that experience with VR and to see the potential of it.
I think the more and more people that see the potential can then bring it to a bigger market.
I&T Today: Cycles was definitely a work of art. Do you think that our perception of art will change as technology becomes more embedded into the art form itself?
Jeff Gipson: For me, it’s the moment that the technology can start to fade away and you just buy into whatever the story or the film or the music is, technology becomes secondary. That’s where I find the sweet balance is, that sweet spot of what VR and even AR really can do, is letting the technology fade away.
So it’s tricky, because a lot of the things out there right now, for both AR and VR, are these cool tech projects that highlight the capabilities of technology. But, trying to find that art side, the storytelling or the film or the music side of it, I think that is really important.
So, hopefully, as things progress, people will get more and more comfortable working in VR and there will be better and better films. It’s almost like when we started filmmaking back in the day, it was kind of this novel thing, but then as people got more and more into it, it just became a standard thing. So, I think something similar maybe will happen with VR and AR.
I&T Today: That will be an exciting time to go to the theater, for sure.
Jeff Gipson: I know! Well, the thing is the headsets are kind of clunky and they’re not the most comfortable things and some of them have cords and some don’t. I think if you had a set of glasses that were as lightweight as your sunglasses or something, that would be more achievable and more comfortable to watch the films.
I&T Today: When I think of animation I think of children and how they’ll react to the medium, and more so how they could one day become part of the industry themselves. Did STEAM, like science, tech, engineering, art, math — did those areas play a big role in your early education or did that come into play more in college?
Jeff Gipson: It’s funny, I grew up in Trinidad, Colorado, so super small town. We didn’t have any STEM programs per se, but I was always really drawn to math and science. My dad, he’s a machinist and so he always had me doing science fair projects from an early age.
When I was in kindergarten, it was kind of our yearly father/son thing to do, and he always encouraged me to be hands on and try to make things. So I kind of lend that gravity towards making things or even architecture and science to my dad.
I think that what’s cool about animation, digital animation in general, is that it’s this balance between art — where you have to really design these beautiful worlds and characters, but then there’s all this technology that comes together with that to bring the film to life.
So it’s really cool to be at Disney, working with some of the best artists, but also some of the most amazing computer scientists that write the software that make it possible for us to do what we do. So yeah, for kids it’s this cool sweet spot. They’re interested in art but also technology and it’s a cool outlet for them.
I&T Today: Do you have any advice that you would offer young people who might be interested in pursuing a similar career field?
Jeff Gipson: I think persistence for me has always been a big key in my professional career. I applied to Disney I think four or five times before I finally got a yes. And I applied to countless other studios — so whether it’s at school, not getting a problem, not giving up and always being persistent and trying again and again and again.
And the same as you go out into the world applying for jobs, you’ll probably get tons of rejection, but being persistent and asking for feedback to try to do better, whatever the skill set is, is something that was key for me.
So even now as we make films, there are a lot of times the ideas don’t work, so try to think about how you can reimagine this idea and try it again and try it again until it does work.
