If you have tried to build content for mobile web which requires audio, no doubt you have run into many issues. So called ‘HTML5 audio’ is not what it’s cracked up to be, and was massively oversold by Apple (when it sought to destroy the flash browser plugin). When the dust settled we were left with an underwhelming audio tag with a lot limitations. The situation got progressively worse on mobile Safari as Apple continued to separate itself from the standards being promoted by the W3C and other browser vendors.
Current Problems with the html5 <audio> tag
Single streaming audio (cannot play 2 sounds at once)
- No precise timing controls
No way to reliably pre-buffer a sound
No ability to apply real-time effects
No way to analyze sounds
When dealing with Mobile Safari, Qpple have decided to create several of their own rules, which really makes it difficult to make app like experiences. (Arguably intentionally so people continue to buy apps from iTunes).
No auto playing
audio streams cannot be auto played when a webpage loads, or when an automated action takes place, instead playing a stream must be directly linked to a user touch event.
Consider what this means for interactive micro-sites or games that rely heavily on audio
No Auto loading audio
As in the above, audio files can only be loaded after a user touch event, such as a start button.
some other limitations worth noting are:
There are a host of hacks to circumvent these deliberately set limitations, but as apple rolls out new versions of Safari, the hacks no longer work, making your web page/ web apps too difficult to maintain, not to mention the recent ios9 update seems to have caused all types of problems for the audio tag
So let me introduce you to the Web Audio API.
What is the Web Audio API?
The Web Audio API has been around for some time but only recently are we seeing the cross browser support we need to rely on it.
Here are some current penetration stats taken from caniuse.com
In a nut shell, Web Audio is an advanced native audio API that is now available across major browsers on both desktop and mobile without requiring an import.
It can be used to manipulate and synthase sounds in real time, such as creating bullet sounds without relying on an actual sound file.
This opens up a huge opportunity for people to deliver richer content than ever. As well as all the advanced sound actions it can do, it can also play simple music tracks and sound effects.
Key benefits relevant to html5 web /game developers
- can play multiple sounds simultaneously
- reliable cross platform between mobile and desktop
- real-time manipulation
Some of the other cool features include
- cross-fading sounds
- frequency response to create accurate real-time music graphs
- microphone support
- create basic tones and frequencies with an inbuilt Oscillator
- create procedural sound
- accurate timing model which can be used to create accurate musical rhythms
- Room effects
- spatialized audio in 2D which lets you position the sound relative to the listener
- It can be quite complicated to understand and requires a lot of code to even play a simple sound or music track.
- It’s not supported on some older browsers (but screw those guys right? To the future!)
To see how it can be used to play simple sounds and music see my post here: How to Play sounds and music with Web Audio
Get a feel for what is possible by checking out the following samples list on your mobile http://webaudioapi.com/samples/
Web audio really saved my ass on a new WebGL based mobile web app I’m tinkering with that requires rich atmospheric sounds.
Unfortunately the mobile safari specific limitations around auto playing and loading audio cannot be escaped with Web Audio. But that’s a whole separate discussion that Apple doesn’t want to have!