ILM and Trixter spearhead the complex mo-cap, animation and effects wizardry that required 19 facilities to make Marvel’s Avengers: Age of Ultron.
The stakes are higher, the action bigger and the visual-effects challenges off the charts as Marvel Studios re-assembles its A-list team of superheroes for Avengers: Age of Ultron.
The second Avengers feature sees the team facing Ultron, an artificial intelligence-based global-defense system, created by Tony Stark (Robert Downey Jr.) and Bruce Banner (Mark Ruffalo), that becomes conscious and rebels against its creators. Played by James Spader via motion capture and animation, Ultron morphs from an amalgam of Stark’s Iron Legionnaires into an imposingly huge robot bent on destroying humanity.
In resisting Ultron, the Avengers are joined by the sibling threats turned heroes Quicksilver (Aaron Taylor-Johnson) and Scarlet Witch (Elizabeth Olsen), in a massive showdown in their native European country of Sokovia.
Again written and directed by Joss Whedon, Avengers: Age of Ultron was a hugely complicated movie that required 19 visual-effects companies to complete.
The bulk of the effects — some 800 shots — were handled by Industrial Light & Magic, which ILM senior visual-effects supervisor Ben Snow says was one of the few facilities Marvel’s production visual-effects supervisor Chris Townsend trusted to deliver on the movie’s most complicated and difficult sequences.
Among the sequences ILM tackled are the Avengers’ opening battle with Strucker; Iron Man’s dream sequence of the Avengers defeated; Ultron Prime; the freighter battle; the Hulk, including the Hulkbuster sequence; and the final battle in and over Sokovia.
Created by writer Roy Thomas and artist John Buscema and debuting in 1968 in the pages of Avengers #54 and #55, Ultron’s first on-screen appearance was handled by Munich-based Trixter, which had done animation work for Marvel previously on Iron Man 3.
Trixter visual-effects supervisor Alessandro Cioffi says they had an advantage by starting on the project early, having been asked in early 2014 to work on Ultron Mark I in time for the character to debut in a presentation at last year’s Comic-Con.
“We were presented with an original design from Marvel for Ultron Mark I, but … it was rather sketchy and needed some translation into the real world,” says Cioffi.
Origins of Ultron
Ultron starts the story as a software system and assembles a physical form from various parts of battle-damaged Iron Legionnaires and old Iron Man armor. Trixter worked on an asymmetrical figure with moving parts, hanging cables and leaking oil to create a creepy look for the character.
Then Trixter added Spader’s performance, which was recorded by Andy Serkis’ London-based Imaginarium motion-capture studio.
Trixter’s animation supervisor Simone Kraus Townsend says they delivered an early version of Ultron Mark I with an animation rig to Imaginarium, which set up a mo-cap stage near London at Shepperton Studios, where the movie was shooting. With a few weeks to work exclusively on the motion capture, Spader came in to test out the rig and use it to explore the character.
“He started with the whole mo-cap setup and he could see himself on screen and how he looks as Ultron Mark I — a sort of digital mirror,” says Townsend.
Spader started experimenting with the idea that Mark I was Ultron’s not-entirely-successful first attempt at a body.
“We ended up testing some restraining devices on him,” says Townsend. “His idea was the arm doesn’t work, so we put it in a sling. He would drag one of the legs behind him, so we would put weights on that leg and put several other weights on one shoulder to have this asymmetric pose … and try to develop a unique James Spader Ultron Mark I body language, which was really important because he doesn’t have any facial expressions.”
The motion capture was not directly translated to screen, but provided a solid base for animators at Trixter and other studios to follow. “We used it as a base for almost every shot,” Townsend says. “Some of them are key-frame animated, but since we knew the character was defined in how he moves, it was much easier to fill in the gaps.”
The animators also listened to a recorded interview with Spader talking about his vision for the character. “I found it really inspiring for the animators just to hear him, how he thinks about he character — why he does what — to get a good starting point,” says Townsend.
Townsend was on-set for the shooting of the party sequence in Avengers Tower, which includes the introduction of Ultron Mark I and his first battle with the Avengers, to understand each shot and ensure consistency.
In addition to Ultron Mark I, the sequence required 2D and 3D face replacements; a CG ceiling in nearly every shot to cover up stunt and camera rigs; and a CG New York skyline to be seen through the tower windows.
“It’s a very dense sequence,” says Cioffi, who says Trixter had at crunch time about 100 people working on about 300 shots that made the final edit, and about 400 shots overall. “We defined it sometimes as a visual-effects playground.”
As Ultron evolves from Mark I into Ultron Prime, he became part of ILM’s purview. “When he’s Ultron Prime, he’s this giant 8- and then 9-foot robot, so its quite different from a normal-size person,” says Snow.
ILM animation supervisor Marc Chu began by applying Spader’s performance to the character, though, again, it was used more as a guide. “We were using all of the data that we had captured as a guide and then animation would hand match that by eye,” says Snow. “I think that process worked out perfectly for it because I think it really captured Spader and the way he was moving in his performance. And having it be something the animator was involved in was a good way of getting the target as satisfying as possible.”
Ultron Prime has the most complex hero-character rig ILM has ever done, with Snow saying it was about 10 times the size of a standard rig used on the Transformers movies with 2,000 nodes — 600 in the face.
Improving the Hulk
ILM has history with the Hulk, having worked on the character’s movie appearances back to Ang Lee’s 2003 feature Hulk. “He was definitely one of the attractions of doing (this movie). Can we improve on it?” says Snow.
The facility started with the Hulkbuster sequence, in which Scarlet Witch has made the Hulk even more crazy and savage than usual. That forces Iron Man to break out the Hulkbuster armor to contain the Hulk in a massive battle in the streets of Johannesburg, South Africa.
Snow says Whedon described the Hulk’s state in this scene as akin to a junkie in need of a fix — bloodshot eyes, veins popping, etc. So ILM began by adding more skin details to the Hulk and ended up recreating the character from the inside out. “We added a lot of detail into the texture maps and … then when we went back into the other scenes to do the regular Hulk we kept some of that,” says Snow. “It was dialed back, but it was there.”
One goal ILM had for this film was to reduce the amount of corrective sculpting done on the shapes of muscles in CG characters.
“We have a great team of sculptors and modelers who are great at sculpting these shapes in both the face and the body, but the effect has always been some slight compromise on the animation’s intent can creep in with that sort of work,” says Snow.
So instead of starting with an outer skin and building muscles underneath it, Snow says ILM looked into recent research on muscle tension and consulted with medical professors to build a more accurate model.
“We built a skeleton, put the muscles on top and then completely simmed the skin over the top of it,” he says. “In the past, we’d tend to hand animate in some of the tension and stretches. … This was much more realistically done in this incarnation of the Hulk, and I think you get a lot of secondary stuff that — even though the audience may not notice it — adds believability.”
Snow also says he was surprised at how much time these changes saved. “It meant our creature-dev guys were stretched less thin, and we were able to spend more time plussing things out rather than fixing problems,” he says.
It also gave more accuracy to both the motion capture and animation performances in the final scenes. “Sometimes you find that you do this corrective work on the face and the performance has changed — his smile is not the same,” he says. “It makes it more true to the real performance.”
Strange Siblings
For Quicksilver and the Scarlet Witch, Trixter again got an early start by working on the characters for the post-credits scene in Captain America: The Winter Soldier and continuing to develop them through production on Avengers: Age of Ultron.
“There’s an obvious coherence on the vision of things,” Cioffi says of working in the Marvel Cinematic Universe. “We always try to improve the look of every asset we work on but having the history with some of the assets is definitely helping us as we can use the experience done on previous movies and just try to improve what we’ve done.”
For Quicksilver, a digital double was used for many scenes, particularly ones in which the character was running through a scene at super speed. “In other situations, we used very high-speed photography to capture as much material from the actor and then speed it up in compositing to create a very photographic type of speed effect,” Cioffi says. On top of that, there were layers of other effects like CG trailing to get the final look.
Scarlet Witch was a completely different challenge, with Trixter doing roto on the most-subtle movements of her hands and wrists.
The final battle sees Ultron attempt to wipe out humanity by raising the capital city of tiny Sokovia high enough into the air that dropping it would create an extinction-level event akin to the meteor strike that wiped out the dinosaurs.
Since Sokovia is fictional, Snow says a look for it had to be developed. Using locations in Italy and sets like a decommissioned police training facility in London, a look approximating a former Soviet bloc nation was created, mixing older buildings with 1970s-style modern architecture.
The city was built using a library of plates shot on location and new digital assets like streets and buildings. Snow says he wanted to push the earthquake effects of Sokovia’s liftoff with more detail, referencing footage of building demolitions and cliff subsidences.
The resulting asset included interiors with furniture in them that could be seen when the building shook apart and was so large that a workaround had to be figured out to work with it.
Pipeline Creativity
Snow credits technical director Ian Roth with figuring out a way to take the asset out of 3ds Max, where it had been built, and put it into Houdini to break stuff apart and refine it by hand. Then it went back into ILM’s proprietary Zeno software for rigid surface simulation and adding dust. Finally, it went back into 3ds Max for rendering.
“It saved us time because we didn’t have to reproduce all of the models in our traditional pipeline, but we were still able to put in things like the skyscrapers that fall right off the edge of the cliff right in your face,” says Snow. “The net result was something that I felt did have a bit more detail than we’ve been able to put in some of these things before.”
The number of facilities working on the film required a lot of cooperation and, where possible, sharing of assets. “We can’t usually share rigs and materials because there’s a lot of proprietary stuff in those, but you would definitely get the riggers from the other companies ringing our guys — and vice versa — and saying, ‘Hey, how does this work?’ ‘We can’t tell you exactly but here are some hints … ,’” Snow says.
Avengers: Age of Ultron VFX Facilities
- ILM
- Virtuos
- Double Negative
- Trixter
- Method Studios
- Lola VFX
- Animal Logic VFX
- Framestore
- Cantina Creative
- Soho VFX
- Luma Pictures
- RISE Visual Effects Studios
- Zoic Studios
- Blur Studio
- The Secret Lab
- Black Ginger
- Capital T
- Crafty Apes
- Technicolor VFX
