Most experienced gamers will have at least some inkling of how a game’s visuals are put together thanks to an abundance of in-depth reviews and articles on the subject in the press, but the art of creating a soundscape for a game is not as well documented.
How do you go about creating a 3D environment in sound using only a set of headphones and more pertinently, how do you make it robust enough to sound convincing when the player is free to move in every direction? We caught up with Ashley Read (aka CCP Sine), the man responsible for all the ear candy in EVE: Valkyrie to find out what he was up against.
Hi Ash. Sound design for games, then. Bit of a niche career. How did you end up where you are?
Luckily, I knew I wanted to work in game audio at an early age and around the same time I got my first guitar. From there both passions just grew together. By the time I was at Uni studying Sound Design and Engineering at the School of Sound Recording in Manchester I was already cramming as much information as I could find on game audio middleware and development engines! I then tried all sorts from looking up local game studios to joining LinkedIn and messaging a lot of people for advice. I eventually got invited to a local studio for a tour, played their game and got a two-week work placement which landed me my first job and the rest is history.
Tech savvy folks are familiar with home cinema surround sound set ups and the like, and can maybe get their heads around the idea of front and rear speakers. What technology is involved in creating positional sound through headphones?
The solution has been around for a while but there was once an issue in game engines where if the sound was hard-panned, the listener wouldn’t hear anything in the opposite ear. To naturally locate sound sources we rely on the time difference between the sound hitting both ears. Nowadays you will get the late reflections in the other ear resulting in a more immersive experience. To help with front and rear positional sound it can be a simple case of processing it slightly differently, for instance boosting the bass if positioned behind the player.
Was it a steep learning curve going from regular sound architecture to that required for VR?
Not necessarily, however the main challenge with VR audio is dynamic mixing. Choosing which sounds have priority, which sounds should modulate, which need to be positioned, what volume you should hear them depending on where you’re looking. It’s a lot of trial and error when testing the mix in VR. It does take time but it is so worth it when something works! Every little detail helps and if you manage to get in as much as possible into a VR game it helps the experience massively.
What are the differences when it comes to designing sound for VR?
The big difference is that you pretty much have a virtual ‘space’ to fill. Putting a shot sound on the end of a gun works generally but in VR it can feel flat and lifeless, so you split the fundamentals of that gun sound like shot, crack, bass and foley rattles and position them on different parts of the weapon. The results are often much more satisfying. You also need to pay attention to the player’s surroundings and how they would affect the sound. In EVE: Valkyrie’s case the player is in a cockpit so a lot of sounds were made to resonate to make it feel like the place is almost vibrating. Another thing is bass management in general. Low bass frequencies have longer wavelengths so positioning them would have little effect, so a few times, when something with significant bass content is played we fade in a stereo rumble to make the positioned sound feel powerful and it seems to do the trick!
Presumably, making a sound move from one ear to the other is the same whether you’re moving a control stick left or turning your head left, but in the case of VR the head can be tilted on its side and the whole body can be moved back and forth, albeit to a limited degree. Does this throw up unique challenges and what are they?
This was the first major challenge that audio programmers tackled when VR started out (this time around). Panning was a simple enough issue to fix but then you had the case of elevation – that is, sounds that can be above or below you. This is where head-related transfer functions (HRTFs) come in. As sounds move, different environmental elements and the shapes of our ears can affect the sound, making it easy to pick up where they are. In VR audio, a slight filtering is added to the sound so when it moves, the sound is also changing slightly. Our ears will pick up this difference and the source will be perceived as above or below you. Coupled with the late reflections mentioned earlier this can make crucial sounds (like enemy fire) much easier to locate.
We once heard a story about a TV show creating the sound of a roaring monster by recording someone doing a massive Coca Cola belch then slowing it down and over sampling it (if that’s the right term). Could you share some examples of the creative ways in which you’ve generated the sounds in Valkyrie?
That’s definitely one way to get a monster sound! For Valkyrie it can be pretty sci-fi heavy on the sound design side so a lot of processing and synths are used, but underneath that we still want the rawness of metal and mechanical elements. I have a car bonnet at the back of the office I can smash the hell out of if I need weapon/collision impact material. There’s a broken 32 inch flatscreen that’s waiting to be demolished for ship cockpit details like rattles, cockpit glass smashes and creaks etc. The office coffee machine has been used on a couple occasions. For explosions, we like to use firework samples to get that low punchiness. I’ve learnt in my career that sometimes some sources of sound effects are best not revealed. For example at my first job I made a new sound for a sci-fi pistol and told the director that most of the it was recordings of me brushing my teeth… He had an almost “I’m paying you to make sounds like that?!” face!
What are your inspirations when it comes to sound in the creative media?
I do play games more than I watch movies and I do enjoy taking in the sound. There are some incredibly good sounding games out there like the Battlefield series, Destiny, Call of Duty and many others. Whenever I feel I’ve hit a brick wall I just play a game to get inspired. The bar is set really high these days so it doesn’t take long to get the mojo back! It’s pretty important to set yourself goals with your work so you can continue to learn and develop yourself.
Thanks for that Ashley. We shall listen out for that bonnet-bashing next time we’re in New Eden.
If you want to find out more about how EVE: Valkyrie was created, check out our interview with CCP’s Emily Knox who tells us how the game’s maps are designed.