Bluish Coder

Update 2014-05-12: Matthew Gregan has a more recent post on Media Source Extensions support in Firefox Nightly.

There is a W3C editors draft for Media Source Extensions that provides a way for web applications to generate media streams for playback from JavaScript. This allows a number of new use cases on the web. A couple of examples might be:

• Media players that download encrypted content from a server, decode the data on the client using JavaScript, and send that data to the video player in a page.
• JavaScript implementations of adaptive streaming (for example, MPEG-DASH).

Work has been ongoing in Firefox to implement the specification. The majority of the work has been done by Matthew Gregan. The tracking bug for this is bug 778617. The dependancies of that bug list what needs to be implemented to complete support. Basic support is available in Firefox since bug 855130 landed.

That allows mediasource_demo.html to run in Firefox (that demo was originally from the URL mentioned in the bug report but hosted locally to have a stable source for testing).

The demo takes an existing VP8 encoded WebM file and splits it into five chunks, each chunk stored in a JavaScript typed array. The code looks like (reorganised from the demo for readability):

var mediaSource = new MediaSource();
var video = document.getElementById('video');
video.src = window.URL.createObjectURL(mediaSource);

var sourceBuffer = mediaSource.addSourceBuffer('video/webm; codecs="vorbis,vp8"');

var reader = new FileReader();
var file = new Blob([uInt8Array], {type: 'video/webm'});
var chunk = file.slice(startByte, startByte + chunkSize);

reader.onload = function(e) {
  sourceBuffer.appendBuffer(new Uint8Array(e.target.result));
  if (i == NUM_CHUNKS - 1) {
    mediaSource.endOfStream();
  }
}
reader.readAsArrayBuffer(chunk);

The createObjectUrl results in a blob URL that contains the media source data that is appended via source buffers. Playing the video using the standard HTML media API will result in it decoding and playing the binary data we've manually appended to the media source.

To run this demo you'll need to enable Media Source Extensions as it's currently switched off. Setting media.mediasource.enabled to true in about:config will turn it on. You'll want to use a nightly build to ensure you have the Media Source code and followup fixes.

Real world usage of Media Source extensions is more involved than that demo and there is much more to do. If you're interested in tracking progress you might like to track bug 778617. Support for a more advanced player is tracked in bug 881512. Completion of that bug should provide enough of the specification to run an MPEG-DASH Media Source demo where the MPEG-DASH code is implemented in JavaScript. This is one of the main goals at the moment.

Other browsers are also supporting Media Source Extensions. The MPEG-DASH demo currently runs in Chrome. The IE 11 preview implements MPEG-DASH support using Media Source Extensions. It'll be interesting to see what other uses web developers will find for this functionality once support becomes widely available.