With years of experience in the games industry, Justin Manning, executive producer, and Sam Birley, lead developer, have recently joined REWIND VR to explore the future of immersive and creative technologies.
We spoke to the Justin and Sam to find out more about the transition, what to expect at a VR studio and the studio’s latest work including on HBO’s Silicon Valley: Inside The Hacker Hostel experience.
What’s so exciting about VR for you?
Justin: VR is a disruptive technology that has the potential to be part of daily life in so many ways beyond pure entertainment. At REWIND we work across VR, AR and MR with a variety of clients across a broad range of businesses. The possibilities are endless; it is exciting to think you could be shaping the future and having a real impact on people’s’ lives.
Artistically, it is a chance to immerse people in places they could experience in no other way – many people automatically try and recreate reality with VR but that feels too limiting; those experiences that take you out of reality and let the user experience something that never could in real life, are much more stimulating for the person in the headset.
Sam: I think the appeal of VR is obvious from the moment you put on an HMD for the first time. After taking the headset off, most people just get it. The possibilities and potential are huge, and whilst the technology is still at a nascent stage, people are already creating applications for more disciplines than I can count.
For me personally, I am tremendously excited about social VR and where that might take us. A lot of my family are scattered across the world, VR enables me to hang out with them in a shared virtual space, watching TV or playing games together; and this is just with first-generation hardware. The next few iterations will bring us technology such as fovea and facial tracking, which will take socialising with others in a virtual environment to a whole new level.
Why did you make the move from games to VR?
Justin: The reason I worked in games for the last 20+ years was that it was a constantly moving and changing business that posed challenges and made me think about new ways of doing things. When VR came along it was obvious that games, and entertainment in general, were part of a much bigger picture. I immediately saw its potential and started thinking about the design, creative, and technical challenges that VR posed. I was drawn by the limitless possibilities.
It was somewhat serendipitous that it was at that point I came across REWIND and all the exciting projects they were working on. I was interested in the fact they weren’t purely limited to PSVR, VIVE, and Oculus, but were using technology creators on the cutting edge, worldwide. The fact REWIND was hardware and software agnostic was very appealing to me.
Sam: I had gotten lucky twice in my career already, working on large-scale AAA VR gaming experiences. After we shipped the last project I worked on, I decided that I didn’t want to wait for lightning to strike three times; I wanted to be involved in shaping the future right now.
This is such a great time to be involved in VR, as there’s still so much we don’t know. Game development is a relatively mature industry and a lot of concepts in games have reached a pretty formalised state. Player health should be managed this way, third-person controls should be handled that way, and so on. By contrast, nobody knows what the best way of moving the player around in VR is yet, and people are shipping titles with all sorts of configurations. There’s just so much to discover.
The Silicon Valley: Inside The Hacker Hostel experience was a collaboration between Silicon Valley, HBO and REWIND.
What skills have you been able to transfer from games? Presumably, a lot of the core assets are still created the same way and using similar engines?
Justin: During my career in games, I gained a great deal of knowledge about how to run cross-functional teams, consisting of both creatives and technologists; all this knowledge was immediately applicable when I joined REWIND. The types of processes used in games, to take an idea through concept, prototyping, and to a final implementation, were also very relevant. Almost everything we do in VR – from content creation, pipelines, workflows, tools, and technology – were practices that I could bring across from games. The exciting part for me, or for anyone with experience in making games, is how we can take all that knowledge and apply it in a different context with different clients and different creative outcomes. It is helpful that the tools we use are the same so that we can focus on what we do with them.
For art, Maya, Max, ZBrush, Substance, and Houdini are used to create content but what we do with those assets in VR is different. For example, we still build the characters and rig them in the same way but what we expect them to do and how they behave according to our actions is more important than ever before. We can no longer just play a cutscene to tell the story, as the person in the headset is part of the story, therefore character behaviours need to respond to player action. Walk away from a character who is talking to you, they should get annoyed, stare at someone too long, they might get uncomfortable. This means we need to start creating systems on top of the old tools and pipelines to deal with behaviour.
Sam: The main difference between developing for VR and games is the interaction paradigms. For example, in typical games non-interactive props are everywhere and that’s OK. The language of games has evolved to the state where the player understands the environment is largely non-interactive; they only expect to be able to use items of direct relevance to the plot/gameplay.
By contrast, in VR a user will generally expect everything to be interactive. As developers, we are trying to allow the user to completely suspend their disbelief that they aren’t actually somewhere else, every time they encounter an object that doesn’t react to them, that suspension is broken. The assets themselves must be much more efficient to render (due to VR requiring a far higher refresh rate than that found in most games), so polygon counts and number of draw calls are more important than ever. Both the interaction paradigm shift and the focus on asset runtime efficiency are cultural changes, though. The same engines, the same asset-creation pipelines, and even the same team role-structures are just as applicable to VR.
This real-time experience takes fans of the show inside the home and workplace of the main characters.
What have been the core differences for you so far, working in VR compared to working for games?
Justin: At REWIND, the biggest differences are the length of the projects I now deal with and the variety of creative challenges I am faced with each day. In gaming, very often you could be working on a yearly iteration of a game or be on projects with a minimum length of 15 months. In VR, we have projects that can be as little as four weeks long and as long as 15 months. How you apply the tools and processes that I talked about earlier to a four-week project vs. a 15 month one is very different indeed – it makes you think about the whole process of creating something, viewing it from a different angle. It challenges what you know.
What I have really enjoyed is the diversity of the projects. I have been at REWIND for three months and have worked on five completely different projects. Some are for automotive brands, some for businesses that want to learn/teach something, others have been pure entertainment or pure technology R&D projects.
VR as a medium also offers some differences; in traditional gaming we still rely on the language of cinema, for example, we use the camera to frame things in order to create emotion; we use scored music to create tension and drama. All of these things have to be re-thought when the player becomes the camera, where music can’t just appear out of the ether. When you want to frame action in VR you have to do so with architectural design and lighting – in some ways it is like building a theatre set. If you want one character to be lower than another in shot to reflect their relative power, well you have to build something that raises one above the other. How we do interactive design and how that works with the art creation pipeline has to be re-thought.
Sam: Prior to joining REWIND, my typical expectation for the duration of a project was counted in years; now it’s months. The fast-iteration times, the breadth of projects coming in thick and fast and the company-wide pursuit of new technology and what can be done with it are just some of the reasons I took the plunge. I couldn’t be happier. It’s similar to game development teams in many ways, but the culture is very different – perhaps a reflection on the VR industry in general, there’s a lot of openness and collaboration on what the best way to do something is.
There are 756 interactive objects inside this experience!
What have you completed most recently? Can you tell us any challenging parts to that project and how you may have solved it?
Justin: Since starting at REWIND, one of the projects I have worked on is the launch of the Infiniti QX50 concept which was showcased at the Pebble Beach Automotive Week in the US. The narrated VR experience placed the user in the driving seat of the car where they could trigger the VR experience with their actions. The creative involved the car exterior, interior and engine parts disassemble and reform around the user. For this effect, the car was split into 342 parts and 50 animations were created and used.
The sheer number of parts to create at the right resolution for VR and then to animate was a challenge; then choreographing all of the parts in a cohesive and fluid animation during both the ‘explosion’ and reforming was a huge task. The team did a great job considering all this was achieved in a quick timeframe.
Sam: Our most recent project ‘Silicon Valley: Inside the Hacker Hostel’ allows the user to enter and explore the famous home and interact with the show’s characters. It was a lot of fun to make, recreating the tone of the show in a VR experience. The main challenge for us was performance. The hostel is full of interactive clutter, so each of those meshes needs to be rendered individually. There’s interactive monitors being rendered out each frame, tons of video playback and a phone with a pass through camera (rendering the whole scene again).
To get this all in our 90fps budget, we used the same methodology you would apply to optimising a game. We used UE4.16’s forward renderer path, with a very straightforward postprocess pipeline. We disabled UE4’s DBuffer feature, and used only one or two decals throughout the project. Good LODS, at as close a distance to the camera as we can get away with. No crazy-tessellated meshes. Minimising draw calls where possible (by hiding entire sections of the level we knew the player couldn’t see from certain vantage points – useful when GPU occlusion culling only gets you so far). Scalability options in the pause menu to allow users on older hardware to reduce graphical fidelity in favour of performance. They’re all standard tricks of the trade in the games industry, and they’re just as relevant in VR, if not more so.
Justin Manning spent the past five years at Supermassive Games and 12 years at Electronic Arts before that, joined us as executive producer. He has 23 years of experience in the games industry working on AAA titles at the very highest level. His game credits include Until Dawn, Need for Speed, F1, and the Harry Potter franchises.
Sam Birley started in quality assurance and was a level designer at Electronic Arts, before studying computer graphics at UCL. After graduation, Sam worked as an engine programmer at Headstrong Games, before moving to Creative Assembly to lead the development team for Alien: Isolation. He got his first taste of VR working on Alien: Isolation and this passion grew and led him to Rocksteady Studios, where he worked on Batman: Arkham VR. Sam has joined REWIND as a lead developer working on real-time content, with a particular focus on Unreal Engine projects.
For more great interviews, news, reviews and much more, why not get a 3D Artist subscription this Christmas? You can save up to 36%, just click here!