PreciseAudio
A browser-based API for more precise audio playback, scrubbing/seeking and gapless playback.
Background
When playing audio in browsers using the <audio>
tag,
seeking/scrubbing to different positions can often be inaccurate,
jumping to a particular point in time can often be off by seconds.
So when trying to synchronize events with playing audio,
like in synesthesia,
scrubbing can cause desynchronization issues
that don't occur when playing a track from start to finish.
This is due to browsers having to sometimes approximate where in an audio file a particular audio frame is, rather than knowing precisely. This is worse in certain codecs more than others.
This problem can be alleviated by using constant bit rate audio files, which allows browsers to more precisely jump to the correct audio frames, reducing the chance of discrepancy.
This however is problematic if you want to be able to consume audio files of numerous formats, with no bit rate guarantee.
This library aims to mitigate this,
and exposes an API similar to
HTMLAudioElement
.
For more background, read Terill Thompson's blog post: Constant vs Variable Bit Rate MP3 Encoding and Timed Text.
For testing this functionalty, we use this test MP3 by Terill, that was created for the blog post above.
Implementation
This library works by using the
Web Audio API
to decode and load the entire track into memory, as an
AudioBuffer
,
at which point scrubbing can accurately pinpoint the correct frame.
Note though that doing this goes directly against the advice on MDN:
Objects of these types are designed to hold small audio snippets, typically less than 45 s. For longer sounds, objects implementing the MediaElementAudioSourceNode are more suitable.
For applications such as synesthesia though, this precision is neccesary.
Gapless Playback
As this library use the
Web Audio API
to fully decode each track into an
AudioBuffer
,
we are able to schedule playback with accuracy down to each individual sample.
This makes it possible to schedule adjacent tracks to play
in such a way as to remove any audible gaps between tracks.
The library has been extended with an API for queueing multiple tracks,
and when used it will attempt to play these tracks gaplessly.
The NPM package
@synesthesia-project/gapless-meta
is used to try to determine any gapless padding
that has been added to audio tracks,
when it's not possible to determine padding,
it is assumed to be 0.
For instructions on how to use the gapless API, see Gapless API.
Usage
This library can either be used in a <script>
tag directly on your pages,
or imported and packaged using webpack.
In either case, you can obtain the package via npm by running:
npm install @synesthesia-project/precise-audio
Including directly in browser
-
Copy the file
node_modules/@synesthesia-project/precise-audio/dist.min.js
(and optionalydist.min.js.map
too) to an appropriate location for your project. -
Add something like this to your HTML:
<script src="/static/precise-audio/dist.min.js"></script>
-
The class
PreciseAudio
will then be available to use directly in the browser:const audio = new PreciseAudio(); audio.src = "...";
Using Webpack
If you are using webpack to bundle your client-side code, You can simply import the module and use it:
import PreciseAudio from '@synesthesia-project/precise-audio';
const audio = new PreciseAudio();
audio.src = "...";
Gapless API
To take advantage of the gapless playback functionality,
you need to queue multiple tracks at the same time using either
updateTracks
or updateUpcomingTracks
.
Example:
const audio = new PreciseAudio();
audio.updateTracks('track1.mp3', 'track2.mp3');
API
Full documentation on the PreciseAudio
api can be found
on the synesthesia project website.
TODO
- Allow specifying a threshold for maximum length of a song to fully decode, disabling gapless playback and reducing precision for songs outside that threshold by falling back to using HTMLAudioElement internally.