[Mild spoilers ahead!]
British supernatural TV series The Innocents is the latest original series on Netflix with incredible VFX work. Key to some of the more impressive digital human work and transformation sequences is that of principle vendor Jellyfish Pictures, who not only worked on 150 shots in the series but were also in charge of the shapeshifting scenes. We spoke to Dave Cook, head of 3D and Jason Hayes, lead compositor about how it all came together.
3DA: What was it like for Jellyfish to take on principle vendor role for The Innocents?
JH: Jellyfish is a very adaptable company, working on a myriad of both smaller and larger projects. We quickly assembled a team that had the experience of working on feature film standard work such as Solo: A Star Wars Story, The Last Jedi and Kingsman: The Secret Service, whilst also having worked on high-end television such as Outlander, Black Mirror and Legion.
DC: It was extremely exciting to be entrusted with realising the distinctive vision of a show that was not only high profile but also had a very unusual and ambitious creative requirement.
3DA: Was there any pressure being the principle vendor for the VFX for the series?
JH: Jellyfish is dedicated to always provide our clients with the highest quality of work possible, and with that comes a lot of pressure. The feedback we had from Netflix let us know that we were hitting the level they expected, which helped alleviate that pressure.
In some ways it helps to be the main vendor as you get a better look into the project, techniques being used and feedback overall.
DC: It is the principal vendor’s role to realise the vision of the writer and director. Whenever we are the principal vendor there is some extra responsibility to develop and maintain a high creative standard, not just in one shot but across all the episodes and sequences. But that is where all the good original work is, so it’s where we like to be!
3DA: I know that Jellyfish had a team of 40 working on the series, could you run through how the team member numbers broke down per department?
JH: Our compositing/2D team comprised of about fifteen people. And then of course you have our production team, which was around three people as well as our senior team overlooking our work, which included creative director, COO and our CEO.
DC: For 3D there were around five matchmove/tracking artists, three animators, three hair and groom TDs, three model/texture artists, two simulation artists and about eight lighting artists who also handled a lot of the secondary animation.
3DA: The Innocents is not Jellyfish’s first foray into working on VFX for a Netflix show thanks to Black Mirror. Did that help with getting the principle vendor job for The Innocents?
JH: With every new project there is a new pitch to demonstrate we can give the client the results they’re hoping for and more. Black Mirror definitely helped prepare us for the level of expectation from Netflix. Hopefully this also provided Netflix with an assurance that we were a tried and tested team whom they could trust to deliver and exceed expectations.
3DA: What was it like working with Netflix again?
JH: We were prepared for both their technical specification and their way of working. Namely working in 4K for example and providing very quick feedback. Netflix is excellent at providing a focused direction with what they think works and what doesn’t.
DC: Netflix are a great client to have, because they are committed to delivering innovative and brilliant original content. That can mean a lot of revisions until everyone is happy, but the end results are worth it!
3DA: How did Jellyfish pitch the creative process and how much time was spent on pitching?
JH: COO Luke Dodd and Creative director Tom Brass came up with the concept idea for the skin ripple. VFX Supervisor Jonathan Cheetham worked on designing the look of morphing from one face to another. We made an initial in-house test shot and then I worked with 3D on a key shift shot on episode 1. Once the clients were happy with the direction, we took what we learnt and spent some time building a more complex and detailed version.
DC: The original concept was pitched by pointing a leaf blower at the faces of two of Jellyfish’s directors, and creating shifting transitions in comp. The idea was to create a kind of high frequency intense deformation that could sell the idea of the shifts and ground it in reality rather than making it appear as a magical transformation.
3DA: Is being able to pitch a creative process quite normal for you guys or was it a good opportunity to really flex your creative muscles?
JH: The pitch stage is a great opportunity for Jellyfish to show off our creative brilliance, presenting the ideas we can bring aboard while helping the client visualise what visual effect would work best for the project. Most projects we work on require a pitch, and this is where Jellyfish is constantly pushing the boundaries of VFX.
DC: It’s great to get in as early as possible to develop ideas. We like to think that we can always find an inventive way to misuse garden equipment!
3DA: What kind of references did you use for the shape shifting transformation parts?
JH: It was crucial to our clients for us to stay true to our original pitch and that we made this look as real and believable as possible. We tried to give the shifts a fresh and original look while also building upon the performances from the actors.
DC: This varied per shot. Sometimes for some of the larger areas of the body – arms and legs for example – we might think about sequential waves or the interference patterns between waves as a reference. Other times – certainly with some of the facial transformations – we would think about a hand sculpting in clay or putty, but moving a little faster and more consistently through the form.
3DA: Did you look at previous examples in film and TV or come into it completely fresh?
JH: We had of course worked on and seen morphs in film and television projects. Part of the creative process is finding ways to go beyond this and show the viewer something they’ve not seen before. There are also a lot of little details included in our shifts that help elevate it as a whole, such as the slight skin rash when hair moves through skin, subtle vein detail and interaction the shifts have on the performers’ clothing.
3DA: Could you run us through the scanning process of the actors and how you worked with the scans to turn them into CG heads?
DC: Our scans were done by a specialist company in Sheffield called Ten24 with a standard 40 camera setup. This provided not only geometry but also a good starting diffuse map for our texture artists. We took the cleaned up .obj files and retopologised them so that they shared topology and UVs and could then be made into a morphing rig. Custom attributes drove the shift between characters via bones, blendshapes and mixed materials and displacements. After that there was a second pass of deformation to add the rippling effects.
3DA: There’s a lot of talk nowadays about the uncanny valley and creating CG humans and making them look natural. What was the most important aspect for Jellyfish in making the CG heads and the transformations look as natural as possible?
JH: As we had the actors’ performances it was important we seamlessly blend as much of it with the CG that used the scanned footage. One technique was to re-project the performance plate back onto parts of the CG to tie them together and trick the viewer in believing that what they are seeing is real.
DC: We were fortunate in that the characters are doing something unnatural, of which there is no obvious equivalent of: in a way the unsettling effect of the uncanny valley where the viewer is uncertain of a resemblance or the nuance of an expression worked to our advantage. That said we were able to reuse elements of the different actors’ performances and synchronise them with the CG renders.
3DA: Could you tell us about the challenge of grooming hair, and how it was overcome by Jellyfish?
JH: Hair is one of the most difficult parts to make look believable. We have a back and forth creative process to view the hair and find ways to improve its look and animation in 3D, while in 2D we try to integrate it using the performance hair as a guide.
DC: The transformation between very different hairstyles (such as tight dark curls to long blonde and back again) was extremely difficult. Especially in view of the dark tone of the dramatic context – there was no room for it to look comic, and yet the effect had to convey the different characters that the shift was passing through. We used Yeti in combination with some custom tools and workflows to animate different grooms towards one another. The longer hairstyles were harder owing to their different dynamic properties. The hair gives a lot of the characters recognisability, so it was important to include enough of it to allow the audience to see who was being ‘shifted into’ without necessarily going all the way to anyone’s complete hair style.
Check out Jellyfish Pictures’ breakdown reel for The Innocents below!