Bluish Coder

There has been a lot of attention lately around the WHATWG <video> specification recommending Theora and Vorbis as baseline codecs.

The issue seems to have gained some attention since the position paper Nokia submitted to the W3C Video on the Web workshop which made it clear they didn't want Ogg included.

The reference to Theora and Vorbis has since been removed from the WHATWG specification and replaced with the wording:

It would be helpful for interoperability if all browsers could support the same codecs. However, there are no known codecs that satisfy all the current players: we need a codec that is known to not require per-unit or per-distributor licensing, that is compatible with the open source development model, that is of sufficient quality as to be usable, and that is not an additional submarine patent risk for large companies. This is an ongoing issue and this section will be updated once more information is available.

This has also caused quite a bit of discussion around the web.

From what I can see the main objection to Theora is the submarine patent issue. Theora is not 'patent-free'. Rather all known patents relating to it have been released to the public. Submarine patents are those which are unknown. They refer to the practice of keeping quiet about a patent on a technology until some company with a large amount of money implements it. Then the patent holder surfaces and attempts to get large amounts of money for it.

The problem of submarine patents is not specific to Theora. All forms of software technology face the risk. H.264 seems to be the current popular technology for those that oppose the use of Theora in the specification. H.264 also has a risk of submarine patents (like any software) but companies have already exposed themselves to this risk because they ship H.264 based systems. By having to implement Theora they then open themselves up to a second avenue of risk, one which they didn't face before.

One could argue that implementing anything new contains this element of risk, so following that logic these companies won't be doing any new implementations of anything. That's not the case obviously, or nothing would be done. They weigh up the risk of patent issues vs the reward of implementing things. With the current perceived low usage of Theora they probably don't see any advantage to them for implementing it vs the risk. So that appears to be why Theora is not a desired choice but some groups.

I've seen questions asked in the discussions asking why don't we settle on H.264. The big companies are already using it. There is the open source x264 encoder and ffmpeg decoder. While these projects are open source my understanding is that any user of the product must pay the required license fees to the patent holders on the techology they use. In the case of H.264, this is the MPEG-LA. The summary of the terms lists the fees. There is a cap on the total amount a company should pay so why not just pay this amount and ship H.264 support?

For an open source project this creates some difficulty. If the source for an H.264 decoder is included then anyone who downloads and builds from source, forks the project, or otherwise distributes a build would seem to have to deal with the licensing issues seperately. That is, the cap wouldn't cover all usage of the source.

This would immediately limit the distribution of the browser and the ability to embed the engine in other products without removing the H.264 capability. With the capability removed you're back to the problem of what codec should be supported with <video>. To effectively implement the HTML <video> specification in a way that can play compatible video streams with other (closed source) browsers, you'd have to front up with that fee.

It might seem that the best approach is to not specify a baseline codec for <video>. This also has problems. The big advantage of a baseline codec is a content producer can provide video in a format they they know all conforming browsers on any platform can display. Without a baseline codec, content producers will upload video in formats specific to particular platforms and we have little advantage over the existing object/embed elements.

The issue of support for DRM as outlined in Nokia's position paper isn't that big of an issue. Even with a royalty-free baseline codec, implementors are still able to have other codecs supported. They can support whatever DRM specific codec is required by 'big media'. But it is important that a free baseline is available for those that just want to supply video in a convenient manner without having to pay any money. Robert O'Callahan mentions this in his recent blog posting.

The W3C Video on the Web workshop starts tomorrow (Wednesday). The subject of HTML 5 and codecs is going to be discussed there. I'll be there talking about Mozilla's position and taking part in the discussion of the codec issue.

For more good commentary on the issue you might like to read The HTML 5 Wars (and why you should avoid them). If you are passionate about the use of Ogg Theora and <video> one of the best things you can do is start using it.

Do compelling demos. Release video in Theora format. It may be easy to use a service that provides video for you in exchange for giving them certain rights but if you want your format to succeed, then increased usage is the way.