Mobile loudness: An adaptive approach
Producer Rob Bridgett explains why an intelligent solution to the issues associated with mobile game loudness is required.
Rob Bridgett, producer and audio director at Clockwork Fox Studios, explains why an intelligent solution to the issues associated with mobile game loudness is required.
As an audio director and mixer, having worked on the console side of game development since 2000, I had become acutely aware, through commissioning and implementing audio content, of the necessity for standardised output loudness recommendations (both average loudness and loudness range).
Fast forward to late 2012, and Garry Taylor and the ASWG (Sony World Wide Studios Audio Standards Working Group) made Herculean efforts to get pretty much everyone in the industry to agree on a loudness target via the ASWG-R001 document (http://gameaudiopodcast.com/ASWG-R001.pdf) finally published and updated in 2013.
The Sony document adopted the -24 LUFS standard, essentially the ITU-R 1770.BS-3 recommendations for broadcast, and, over the last few years, Microsoft, Nintendo and the Independent G.A.N.G. I.E.S.D. body have all subsequently fallen into line with the same recommendations and agreed that these loudness levels made great sense for games in the console-based home entertainment space.
In terms of mixing games, having these recommended levels to aim for has been really great. Prior to their arrival, it was mostly a guessing game in terms of overall levels. The landscape has changed dramatically for the better over the last few years, and ITU-R/EBU adherent metering is now not only available, but deeply embedded inside DAW workflows (Nuendo) and audio middleware monitoring workflows such as AudioKinetic’s Wwise.
While loudness has been taking centre stage inside console game development audio circles, another significant change has been happening on the side – the rise of mobile gaming via the success of iOS and Android devices as gaming platforms.
It is in the realm of the indie developer that loudness has not yet gained the traction that it has at the larger publishers and triple A developers. And it is easy to understand why. Loudness recommendations don’t seem to translate well to a mobile platform. It is likely very low on a list of priorities for an indie developer in getting their game to the market, for whom even audio integration may be similarly low priority. It is an area where there is great difficulty in pinning down a standard, but the biggest hurdle is that a single standard for mobile devices seems to make very little practical sense.
The idea of a single standard for home entertainment is, by comparison, relatively easy to pin down because the assumption, in the vast majority of use-cases, is that the listener/viewer will be consuming the content, via a television, somewhere inside their home.
With mobile, the gamer could be using either the device speaker, the device headphones, or maybe even mirroring the device’s content onto a larger home screen via Apple TV. Add to this the range of locations that a listener may be in – inside, outside, on a bus, on a plane, in a quiet café, a busy street, a classroom – and you have a range of loudness use-cases that one single loudness standard cannot ever hope to accommodate.
Mobile Loudness is not just an issue for video game content, but also mobile movie-player software, such as the Netflix player, and could benefit greatly from an adaptive approach to its loudness output.
This is where adaptive/intelligent loudness solutions need to take over – solutions that not only take into account the user’s current output method (detecting headphones, speaker or AirPlay mirroring) but going even further – solutions that dynamically monitor the levels of the background noise in the user’s environment, and adjust the LRA and levels at runtime to compensate.
Above: Audiokinetic's Wwise
Fortunately, the research and work already done in other areas of entertainment on loudness offer solid benchmarks for many of these different situations. Mirroring to Apple TV? You are now in the broadcast domain and -24 LUFS becomes the target. Using the device speaker? The iTunes ‘Soundcheck’ loudness of -16.2 LUFS becomes a legitimate benchmark.
Listening on headphones? Slightly trickier. In an ideal, mostly quiet setting, a -24 LUFS level will still work great – when the environmental noise gets louder, runtime master compression can edge things up to -18 LUFS (Sony’s Mobile target) or even -16.
But, what if you are a young child playing an educational game? Something a lot lower, safer and more comfortable is likely necessary around -28 LUFS (levels I am currently working with on our own ed tech products, and which still feel quite ‘hot’ for kid’s ears at full device volume) – this can be edged up to -24 LUFS with -5 LRA by the runtime compressor for noisier travel or classrooms.
Having the software switch (in the case of the device output selected) and blend (in the case of runtime LRA attenuation) seamlessly between these different standards is in reality fairly trivial to set up using today’s middleware and a Unity plug-in (which you’d currently need to write yourself!) to detect the output route of the device and use a device’s built-in microphone to monitor audio background levels when headphones are plugged in, but it is certainly not beyond the reach of any developer with a firm focus on quality audio.
With this technology in place, and with a range of targets defined, it is far easier to mix for mobile by getting everything to sound how you want it at the ‘idealised’ middle target, say -24 on headphones in a quiet room. Then, using the other states to simply tweak the mix at the other target levels through the various other outputs and simulating various noise conditions.
Rob Bridgett is audio director and producer at Clockwork Fox Studios in Canada. Inspired by the work of the Sony ASWG group and the lectures of sound advocate Julian Treasure, the Clockwork Fox team have been working on developing adaptive loudness switching and runtime LRA attenuation for child-friendly mobile ed tech products for the past two years.