Bluish Coder

Some time ago I wrote a dev-media post outlining the media formats we support and requesting discussion on future formats. This post summarizes the results of that and changes that have occurred since then.

In general Mozilla is trying to limit the proliferation of media formats on the web. One reason for doing this is to make it easier for a website to provide media that they know can be played by the majority of web users. Ideally the formats supported would be freely implementable by anyone in any user agent. This enables someone to share media on the web knowing that all users can watch it.

A counter argument is that limiting the formats to a small number restricts the producers of the media. They are unable to select the best formats for the type of media they are sharing that plays best on specific devices (for example, taking advantage of hardware acceleration on a device). The choice of what formats can be used would be left up to the operating system in this case and the browser would fall back to using operating system support when it wants to decode media data.

Taking the operating system codec approach would result in media on the web only being able to be played by a subset of users. If someone shares a wmv video file it will only play back on systems supporting that format. The list of all possible formats is huge and the intersection of formats supported across operating systems isn't great. We'd also like to avoid codec support on the web becoming a vector for malware. Evil sites could prompt users to install malware infected codecs required to play video from the site for example.

By building support for specific formats in the browser we hope to guarantee to media producers that their media will be viewable by all web users safely.

Firefox currently supports the following media formats where the decoding support is built into the browser:

• Opus audio in an Ogg container on all platforms.
• Vorbis audio in an Ogg or WebM container on all platforms.
• Theora video in an Ogg container on all platforms.
• VP8 video in a WebM container on all platforms.
• WAV audio on all platforms.

Opus, Vorbis, Theora and VP8 have the advantage of being open source and usable without paying license fees for generating and viewing content. Currently the decoding support in Firefox for these formats is not hardware accelerated. All decoding is done in software.

Support for the following formats in Firefox uses operating system decoder support. This means coverage is not available across all platforms. We are working towards this goal.

• H.264 video in an MP4 container on Firefox OS, some Android devices, and Desktop on Windows Vista and up.
• AAC audio in an MP4 container on Firefox OS, some Android devices, and Desktop on Windows Vista and up.
• MP3 audio in MP3 files on Firefox OS, some Android devices and Desktop on Windows Vista and up. Support for Windows XP is coming in Firefox 26.

These formats may use hardware accelerated decoding depending on operating system support. Support for these formats on other platforms is ongoing. On Linux we will use GStreamer. GStreamer playback support is already landed but not built by default. Enabling it is tracked in bug 886181. You can produce custom builds by using the configure switch --enable-gstreamer and setting the pref media.gstreamer.enabled to true.

Support for Mac OS X for H.264, AAC and MP3 is tracked by bugs 801521 and 851290.

Only a subset of Android devices are supported. For Android we are using libstagefright to provide hardware accelerated decoding. This uses an internal Android API that some device manufacturers customize which results in changes being needed to support specific devices. As a result we whitelist and blacklist devices that we know do or don't work. You can find the list of blacklisted or supported Android devices on the Mozilla wiki. Bug 860599 is a work in progress to implement a more reliable means of using libstagefright which should provide support on more devices.

All the formats listed above are what Mozilla has generally decided to support "on the web". That is, a web developer can expect to provide a video element with one of these formats as the source and expect it to play, if not now then at some point in the future.

There are some other formats that Firefox supports on specific operating systems or devices. These are primarily used in Firefox OS for system applications and to implement various mobile specifications. While supported on the device they are not guaranteed to be supported across all platforms or even on the web. They may only work on apps installed on the device. These formats are:

• 3GPP container on Firefox OS. This is used for MMS videos and video recording on the device.
• AMR audio format on Firefox OS. This is used for MMS and can only be played "on device" in privileged apps.

Other media formats may be supported in the future. VP9 and Daala for example are possibilities. For an official list of supported media formats, and the versions of Firefox they became supported, there is a supported media formats page on MDN. For discussion of formats and media in general there is the dev-media mailing list.