Bluish Coder

Building the Firefox video element backends

The git repository has multiple branches which means a little more git usage is needed to get at the Ogg (and soon, GStreamer) backends.

When you clone or pull from the repository you'll automatically pick up the commits for the new branches. You can see these branches with 'git branch -r':

$ git clone git://double.co.nz/git/video.git
$ cd video
$ git branch -r
  origin/HEAD
  origin/master
  origin/ogg

By default the 'master' branch has a checked out working tree. Building this produces <video> element support without any decoder. One approach to building the Ogg backend is to create a local branch with the commits from the remote branch origin/ogg and build that:

$ git checkout -b ogg origin/ogg
Switched to a new branch "ogg"
$ cd mozilla
$  cat >.mozconfig << "EOF"
. $topsrcdir/browser/config/mozconfig
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/obj-@CONFIG_GUESS@
mk_add_options MOZ_CO_PROJECT=browser
ac_add_options --enable-video
ac_add_options --enable-ogg
EOF
$ autoconf2.13
$ make -f client.mk build

When the gstreamer branch is added you can do similar:

$ git checkout -b gstreamer origin/gstreamer
Switched to a new branch "gstreamer"
$ cd mozilla
$  cat >.mozconfig << "EOF"
. $topsrcdir/browser/config/mozconfig
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/obj-@CONFIG_GUESS@
mk_add_options MOZ_CO_PROJECT=browser
ac_add_options --enable-video
ac_add_options --enable-gstreamer
EOF
$ autoconf2.13
$ make -f client.mk

You can do a build which contains both backends by merging. You'll need to fix any merge conflicts and change the ordering of the codec instantiation so that one is preferred over the other when they can both handle the same codec. A merge looks like this:

$ git checkout -b gst_and_ogg origin/master
$ git merge origin/ogg
$ git merge origin/gstreamer
$ cd mozilla
$  cat >.mozconfig << "EOF"
. $topsrcdir/browser/config/mozconfig
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/obj-@CONFIG_GUESS@
mk_add_options MOZ_CO_PROJECT=browser
ac_add_options --enable-video
ac_add_options --enable-ogg
ac_add_options --enable-gstreamer
EOF
$ autoconf2.13
$ make -f client.mk</pre>To update your branches with the latest code from the repository, fetch and merge:<pre>$ git fetch origin
$ git checkout ogg
$ git merge origin/ogg
$ git checkout gstreamer
$ git merge origin/gstreamer
$ git checkout gst_and_ogg
$ git merge origin/ogg
$ git merge origin/gstreamer

Once you've done a successful build you can run it from the obj-dir, or make an installer:

$ cd mozilla
$ make -f client.mk build
$ export NSPR_LOG_MODULE=nsVideoDecoder:5 
$ export MOZ_NO_REMOTE=1 
$ ./obj-i686-pc-linux-gnu/dist/bin/firefox -ProfileManager
$ make -C browser/installer

When the installer is built there will be a file in the 'dist' subdirectory of the object directory that contains the build. It will be a .tar.gz, .zip or .dmg, depending on the platform. The environment variables set in the above example enable logging of the video debug output and running Firefox even if an existing copy is already running.

Refactoring of Firefox HTML video element patch

I've making some changes to the patch implementing the HTML <video> element for Firefox.

Previously the patch attached to the bug contained the implementation of the video element and a Theora backend implemented using liboggplay.

This required pulling in a large amount of third party library code making it more difficult to get reviews for the patch and to get feedback on the changes required for the video element itself. To hopefully fix this I'm changing bug 382267 to only include the video element implementation with no video decoders.

Applying the patch from that bug to the Firefox trunk source and building with the --enable-video option will provide support for <video> but all videos will fail to load due to not having a supported decoder. This is obviously not very useful but will hopefully make the patch easier to get reviewed and landed.

I've added bug 422538 which will contain support for decoding Theora videos. This is the code taken out of the original bug. Applying the patch from this bug on top of the video element patch and building with --enable-ogg will give <video> element support that can play Theora videos.

As part of the refactoring I've made it easier for different video decoder backends to be implemented. There is an abstract class with pure virtual methods that can be inherited from. I have a 'work in progress' implementation of a GStreamer backend that I'll write about later and attach a patch to bug 422540. This backend allows playing any video format supported by GStreamer. I've tested it with the Fluendo H.264 decoder and it plays fine. Keep an eye on the bug if you're interested in this.

An advantage of this refactoring is that embedders of Gecko) can choose a video backend that suits their purposes (for legal or other reasons they may choose not to have support for a particular format), or implement their own.

Currently the interface for implementing video decoder backends is a C++ abstract class. I'm considering using XPCOM so that implementations can be built and dynamically loaded. This could allow extensions to be written containing video backends that will work with the <video> element. One of the questions asked of me at the W3C video on the web workshop was whether third party providers of DRM enabled codecs could leverage the <video> element. This could provide a pathway to doing that.

The git repository will be restructured so that the 'master' branch contains the <video> element support only. The 'ogg' and 'gstreamer' branches will contain the implementations of the backends.

Building the ActionScript compiler for Tamarin

Adobe have released the Flex SDK as open source. The trunk of the SDK repository contains the source to the ActionScript compiler and the GlobalOptimizer component that can be used for generating code used in Tamarin Tracing. To build the asc.jar required by Tamarin, first get a copy of the current source, change to the 'modules/asc/build/java/' directory and run 'ant' to build 'asc.jar':

$ svn checkout http://opensource.adobe.com/svn/opensource/flex/sdk/trunk flex-sdk
$ cd flex-sdk/modules/asc/build/java
$ ant clean main

The generated 'asc.jar' file will be in '/modules/asc/lib/asc.jar' and should be copied to the 'utils' directory of Tamarin Tracing to be picked up by the build tools.

Linux support for video element patch updated

I've updated the git repository to improve the support for Linux. I've uploaded a new build, which can be obtained from the test page or downloaded here.

This is the first build using the ALSA backend that I posted about previously. The sound playback is much improved over previous Linux builds, and so is the a/v sync - as long as you have a fast enough connection that the download doesn't starve. There is still no initial buffering in this build.

I tweaked the git repository import so that the details for each git commit includes the CVS commit messages for the CVS commits contained within it. That should help track down regressions or problems to their original CVS commit.

Video Element Progress

I thought it might be time for an update on the progress of the <video> element support for Firefox.

Unfortunately on Christmas Eve my Macbook hard drive crashed losing everything on it. Fortunately I push my repository to a remote server regularly. Until I get a new drive and the Macbook set up again I'm using another laptop dual booting Linux and Windows. This has given me the chance to work on getting the Linux version of <video> support a lot better.

The main problem with the Linux version was with the sound implementation. The Linux backend of sydney audio uses OSS and has problems with locking /dev/dsp, is unable to return information I need for a/v sync, and didn't have any volume control support. There is a partial implementation of ALSA support for sydney audio included with liboggplay but it was nowhere near complete, and used an older API interface.

I've implemented a new ALSA backend for the library and now have sound playing much better on Linux. Volume control works and sound playback is generally better. I'm adding support for returning the information required for a/v sync now. Once this is done Linux playback should be as good as that on the Mac and Windows builds.

After that I'll be updating the Windows audio backend so volume control and a/v sync information works with that as well.

I'm still using an older libtheora library. During my work on the theora playback a new version of the library was released. This has many improvements but does not have optimized assembler routines for the Microsoft Visual C++ compiler, which is what we use to build Firefox on Windows. Nils Pipenbrinck has submitted a patch to theora which adds MSVC compatible optimized assembler routines. Once that hits the main codebase I'll be moving to the new version and see how it goes.

I'll have a patch pushed to the git repository with the recent Linux changes as soon as I confirm it still builds and runs on the other platforms.