Bluish Coder

Programming Languages, Martials Arts and Computers. The Weblog of Chris Double.


2010-05-18

ATS - Applied Type System Programming Language

The release of Version 0.2.0 of ATS was recently announced on the ATS mailing list. From the ATS website:

ATS is a programming language with a highly expressive type system rooted in the framework Applied Type System. In particular, both dependent types and linear types are available in ATS. The current implementation of ATS (ATS/Anairiats) is written in ATS itself.

The Computer Language Benchmarks Game show's ATS as being similar to C/C++ in terms of performance.

Compilation from source follows the standard 'configure' and 'make'. Once compiled you need to set an ATSHOME environment variable along with adding the bin directory to the path. Instructions on what these should be set to are displayed after compilation is finished.

$ ./configure
...
$ make
$ ...
ATS/Anairiats has been built up successfully!
The value of ATSHOME for this build is "/home/foo/ats".
$ export ATSHOME=/home/foo/ats
$ export PATH=$PATH:$ATSHOME/bin

'atscc' is used to compile an ATS program to C and then compile the C code to an executable. A simple 'Hello World' ATS program is:

implement main() = print("Hello World\n");

Place that code in a 'hello.dats' file and compile:

$ atscc -o hello hello.dats
atsopt --output hello_dats.c --dynamic hello.dats
gcc ... -o hello hello_dats.c -lats
$ ./hello
Hello World

When running 'atscc' you can see the commands it spawns to do the compilation. It runs 'atsopt' to generate the C code and 'gcc' to compile the C code. The resulting executable on my Ubuntu Linux system is about 13Kb when stripped. The commands and options are explained in the ATS manual.

ATS feels like a hybrid of C and ML. Code is written in an ML style but you can embed C directly and call ATS functions from C or C functions from ATS. You can control memory at a low level, optionally include garbage collection, and use the type system to ensure that memory is safely accessed and released.

A quick example of calling C from ATS is below:

%{^
#include <ogg/ogg.h>
%}

abst@ype ogg_sync_state = $extype "ogg_sync_state"

extern fun ogg_sync_init (oy: &ogg_sync_state?): int = "#ogg_sync_init"
extern fun ogg_sync_clear(oy: &ogg_sync_state?): int = "#ogg_sync_clear"

implement main() = let
  var state: ogg_sync_state?;
  val r = ogg_sync_init(state);
  val () = printf ("ogg_sync_init: %d\n", @(r));
  val r = ogg_sync_clear(state);
  val () = printf("ogg_sync_clear: %d\n", @(r));
in
  ()
end

This can be compiled and run (assuming libogg is installed and the above is in ogg.dats) with:

$ atscc -o ogg ogg.dats -logg
...
$ ./ogg
ogg_sync_init: 0
ogg_sync_clear: 0

C code is directly included using '%{ ... %}'. A variant of this is '%{^ .... %}' which places the C statements at the top of the compiled ATS code. This is what I use for including the libogg header above. libogg has a C struct type called 'ogg_sync_state'. This is exposed to ATS using the '$extype "ogg_sync_state" definition. The C functions are defined with an ATS function definition matching the C definition and the '= "#ogg_sync_init"' at the end states that it is implemented in an external C library under the given name.

The argument passed to the C function is given as 'oy: &ogg_sync_state?'. The means the argument name is 'oy' and it is an ogg_sync_state object passed by pointer (The '&' part). The '?' at the end means it can be unintialized. The code that calls these routines looks very C like:

var state: ogg_sync_state?;
val r = ogg_sync_init(state);

The C code generated by this is actually:

ATSlocal (ogg_sync_state, tmp1) ;
ATSlocal (ats_int_type, tmp2) ;
tmp2 = ogg_sync_init ((&tmp1)) ;

This exactly equivalment to hand coded C. Creating the structure on the stack and calling the libogg function passing it a pointer to it.

The above is only one way of calling C routines. There is much more flexibility and you can use the type system to express an impressive amount of compile type safety checking. See the 'fgets' example at the end of this ATS for ML programmers page. Another good example is this Cairo tutorial which demonstrates using linear types to keep track of Cairo objects. It becomes a compile time error not to call the appropriate Cairo functions to release resources.

Some other interesting reads to learn about ATS:

Tags: ats 

2010-04-28

Fennec on Android

A pre-alpha build of Fennec, the mobile version of Firefox, is available for Android devices.

Over the last few months, we've made some great progress on bringing Firefox to Android. Michael Wu, Brad Lassey, Alex Pakhotin and I have been focusing on getting a build ready that's usable by a broader set of people, and we're now ready to get that build out there.

John Gruber comments:

It'll be interesting to see if Gecko can be turned into a worthy mobile competitor to WebKit.

Gecko already runs in a released mobile device. The Nokia N900 uses MicroB as its built in browser and that is based on Gecko. The web browser functionality of MicroB matches, if not exeeds, that of many WebKit based browsers on other mobiles which I think shows that Gecko can be a 'worthy competitor'.

Tags: mozilla  mobile 

2010-04-27

HTML 5 Video Seeking and Redirects

Bug 560806 has landed on trunk. Seeking and duration works for videos hosted on some servers where it didn't before.

Firefox's implementation of HTML 5 video allows seeking in unbuffered portions of the video as long as the server that hosts the video supports byte range requests.

Firefox needs to know in advance if it can seek the video so it can correctly display the 'progress' bar. It also needs this information so it can compute the duration (which requires seeking into the video to find the end time).

Most servers that support byte range requests send an Accept-Ranges header. If we see this on the initial request we know the server supports seeking.

Some servers support byte range requests but do not send this header. Amazon S3 is an example of a server that does this. To detect if these servers support seeking we make our inital request for the video include a 'Range' header that askes for the bytes from zero to the end of the file. Servers like S3 that support byte ranges will respond with a '206' response code meaning a partial request is returned. This is sent even though the byte range we asked for is actually for the entire file. Once we see the '206' we mark the resource as seekable.

A very few servers don't send an Accept-Ranges header and recognize our byte range request as actually being for the entire file and return a '200' response code. Unfortunately we don't recognize resources hosted on these servers as seekable. This means Ogg media on these servers don't have a duration since we can't seek to find it.

Another issue we came across, and exists in Firefox 3.5 and Firefox 3.6, is detecting seeking when the initial request for the media resource results in a '3XX' response code. This is a redirect to another URL. Current Firefox versions (and possibly other browsers) do not pass on the value of headers to the redirected URL. This means the 'Range' header is removed and we don't get a '206' response code on the redirected request. In servers like Amazon S3 that don't send 'Accept-Ranges' this means we don't support seeking.

This is why videos hosted on Amazon S3 and similar serving platforms that arrive there via a redirect don't get a duration, show progress or allow seeking. It's a fairly common practice to hide the fact that Amazon S3 is being used by having a redirect from a URL on the web site domain that redirects to the S3 location which results in this problem.

The following are two URL's that show the problem. The first is a direct link to the video. The second redirects to the final URL. In the first you can seek and see a duration of '1:00'. In the second you can't seek and no duration is shown:

This is Bug 560806 and has been fixed and is now on Firefox trunk and in the nightly builds. The fix was to explicitly pass the 'Range' header along to the final URL. This workaround exists in a few other areas in Firefox and Bug 311599 discusses a similar issue for plugins.

If your own videos are not seekable or showing a duration then you may be facing a similar issue and it might be worthwhile trying a nightly build and seeing if that fixes it. Other things to look at are the server configuration to make sure that byte range requests are supported and the Accept-Ranges header is being sent. If you use Amazon S3 you might want to try and get them to support Accept-Ranges. This forum post discusses the issue.

Tags: mozilla  video 

2010-03-12

Changes to Firefox YCbCr Color Conversion Code

I posted previously about colour conversion libraries. To replace the liboggplay conversion routines we decided to try the Chromium libraries.

Bug 551277 adds the Chromium code and integrates it with the new video layer painting code. The Chromium code uses a C fallback if MMX is not available but this is done at compile time. I modified it do runtime detection using Mozilla's SSE.h and moved the generic C routine from the Linux specific implementation to be used on all platforms that don't have MMX. Some other minor modifications were done (function name, namspacing).

I added an 'update.sh' script that can be used to update against the latest Chromium code. Given the path to the Chromium source it applies a patch for the changes I made.

Bug 551378 adds YCbCr 4:4:4 conversion (YV24). The Chromium code supported 4:2:0 and 4:2:2 (YV12 and YV16 respectively). Theora also allows 4:4:4 per spec. There aren't many 4:4:4 Theora videos out in the wild and the fix in this bug only adds support in the slower C implementation. I detect if the video is a 4:4:4 video and fallback to the C code.

At some point optimized MMX routines should be done for this. I've raised Bug 551844 for this. This would probably make a good first bug for anyone who wants something small and self contained to work on. It would be good to contribute this back to Chromium when done too.

Xiph has a test suite with example videos in the different formats.

Tags: mozilla  video 

2010-02-25

Vorbis Player implemented in Pure

After writing the preforking echo server example in Pure I wanted to try something a bit more complex. I've written Ogg players in various languages before so I decided to do one in Pure to compare. Here's some of the ones I've written in the past:

I wrapped the libogg and libvorbis libraries using the Pure FFI. A program 'pure-gen' is included with Pure that will generate FFI wrappers given a C include file. I ran this against libogg and libvorbis. The generated Pure wrappers look like:

extern int ogg_page_version(ogg_page*);
extern int ogg_page_continued(ogg_page*);
extern int ogg_page_bos(ogg_page*);

As you can see Pure FFI definitions look very much like the C definitions.

I decided to model an Ogg file as a list of physical pages. This is esentially a list containing elements of the libogg ogg_page object. I have a Pure function that will take this list of pages and return a list of packets for each logical bitstream in the file. For vorbis playback I take this list of packets for the vorbis bitstream and produce a list of decoded PCM data.

Pure is a strict language by default but provides an '&' special form to perform lazy evaluation. I use this to make sure all the returned lists mentioned above are lazy. Playing the lazy list of decoded PCM data never loads the entire file in memory. The file is read, converted into packets and decoded as each element of the PCM data list is requested.

For audio playback I use OpenAL. I queue the PCM data to an OpenAL source up to a limit of 50 queued items, removing them as they get processed and adding new ones when room becomes available.

Here's what some of the API to process the Ogg files looks like in Pure. First load the player code. This will also load the Ogg, Vorbis and OpenAL wrappers:

$ pure -i player.pure

From the Pure REPL, play a vorbis file:

> play_vorbis (sources!0) "test.ogg";

Load an Ogg file getting a lazy list of all pages in the file:

> using namespace ogg;
> let p = file_pages "test.ogg";
> p;
ogg_page (#<pointer 0x9e61cc8>,[...]):#<thunk 0xb7259b38>

Get a list of all packets and the serial number of the first logical bitstream in the list of pages:

> let (stream1,serialno) = packets p;
> serialno;
1426627898
> stream1;
ogg_packet (#<pointer 0x9dd98f0>,[...]):#<thunk 0xb725bce0>

Get the next logical bitstream:

> let p2 = filter (\x->ogg_page_serialno x ~= serialno) p;
> p2;
#<thunk 0xb725dc30>
> let (stream2,serialno2) = packets p2;
> serialno2;
629367739
> stream2;
ogg_packet (#<pointer 0x9cfcf08>,[...]):#<thunk 0xb725c988>

Note that all these results are lazily evaluated so the entire file is not loaded into memory. If we look at the original list of pages you'll see what has been read so far (two pages):

> p;
ogg_page (...):ogg_page (...):#<thunk 0xb725dd98>

Get all the streams in the file:

> let s = all_streams p;

Get all the vorbis streams:

> let v = vorbis_streams s;

Get the first vorbis stream. I ignore the serial number here (this is what the _ does):

> let (v1, _) = v!0;

Decode the vorbis header packets and intialize a vorbis decoder for the first vorbis stream:

> let decoder = vorbis_header v1;

Get a lazy list of all the decoded PCM data from the vorbis stream:

> let pcm = pcm_data decoder;

The mechanics of getting the decoding floating point data from libvorbis into Pure and then back to the OpenAL C API is made easier due to it being possible to easily convert to and from raw C pointers and Pure matrix objects.

For example, the data returned from the vorbis call to decode the data is an array of pointers pointing to an array of floats. To convert this to a Pure matrix with dimensions (c,n) where 'c' is the number of channels and 'n' is the number of samples:

vorbis_synthesis_read dsp samples $$ matrix m
when
  f0 = get_pointer data;
  floats = [get_pointer (f0+(x*4))|x=0..(channels-1)];
  m = map (float_matrix samples) floats;
end;

This code uses a list comprehension to return a list of 'pointer to array of floats'. One pointer for each channel in the audio data. It then maps over this list producing a new list containing a matrix of float's. This is list of matrices each of which contains the audio data for a channel. The 'matrix m' call converts this to a single matrix of dimensions (c,n).

OpenAL expects the data as a pointer to a contiguous matrix of floats with dimensions (n,c) so when writing to OpenAL the matrix needs to be transposed:

play_pcm source rate pcm@(x:xs) =
  ....
  queue_sample source rate (transpose x) $$
  play source $$
  play_pcm source rate xs;

This transposed matrix is later converted from floats to 16 bit integers and 'packed' so it is contiguous in memory, and a pointer to the raw memory passed to OpenAL:

buffer16 s = pack (map clamp s);  
...
data = buffer16 ...pcmdata...;
alBufferData ... (cooked (short_pointer NULL data)) ((#data) * (sizeof sshort_t)) ...;

The 'cooked' call means the C memory has 'free' called on it when nothing in Pure is holding onto it anymore.

One downside with lazy lists is it can be easy to accidently read the entire file and decode everything before playing. This will cause issues with large Ogg files. I managed to avoid that and files play in constant memory. An iteratee style approach might be interesting to try as an alternative to lazy lists. I'd like to try decoding Theora files and doing a/v sync in Pure at some point as well.

I haven't used Pure much but it seemed to be fairly simple to put this together. I'd be interested in feedback on the approach (using lazy lists) and Pure style issues in the code.

The code for this is at http://github.com/doublec/pure-ogg-player.

Tags: ogg  pure  vorbis 


This site is accessable over tor as hidden service 6vp5u25g4izec5c37wv52skvecikld6kysvsivnl6sdg6q7wy25lixad.onion, or Freenet using key:
USK@1ORdIvjL2H1bZblJcP8hu2LjjKtVB-rVzp8mLty~5N4,8hL85otZBbq0geDsSKkBK4sKESL2SrNVecFZz9NxGVQ,AQACAAE/bluishcoder/-61/


Tags

Archives
Links