2010-11-24
Firefox video scaling during YCbCr to RGB conversion
When I changed to using the Chromium YCbCr color conversion code I didn't use the code that scales the video during the conversion as there was no infrastructure in place to do this at the time.
What this meant was we'd do the YCbCr conversion on the original image size and then later, during rendering, scale this using the browser's scaling routines. This was slow in the case of large videos being scaled using the 'width' and 'height' attributes on video elements.
YouTube's HTML 5 video user interface often scales down. By default it plays a good quality video scaled down to fit the standard YouTube user interface. In Firefox this results in a performance hit due to the slow scaling.
Bug 577843 was raised to address this issue. The fix was to implement scaling at YCbCr conversion time using the scaling code that already existed in Chromium's YCbCr code that we were using. This has now landed and will be in Firefox 4 resulting in better video playback when scaled.
As part of this work I also updated to the latest Chromium code. This was done in bug 583138. This gives more performant conversion routines and better quality. This has also landed recently and will be in Firefox 4.
For performance comparisons to see if there were improvements in playback I tested video playback using WebM, Theora and a commonly used open source flash-based video player playing an MP4 video. I compared dropped frame statistics using stats available from the flash player and the experimental playback statistics I wrote about earlier.
For a 1080p sized video, scaled down, the performance on my laptop improved by about 25% with these patches applied and was at least as good as the flash based player. For the Theora video performance was better - I suspect due to the lower CPU requirements to decode Theora video.
[update from feedback: Removed comment about YouTube's fullscreen not being hardware accelerated. Apparently Firefox trunk now accelerates it. Clarified that by 'flash player' I meant an open source video player written in Flash, not an open source Flash implementation]
