Bluish Coder

Programming Languages, Martials Arts and Computers. The Weblog of Chris Double.


2006-09-27

Futures and Promises in Javascript

The Alice programming language has some nice concurrency features to facilitate dataflow programming. Futures and Promises allow performing computations and having threads block waiting for the results of those computations in an safe, predictable manner.

There have been discussions in the Narrative Javascript group about how to implement this sort of functionality on the browser client side. Given client side threads, it's fairly easy to implement. I'd done some work previously on implementing client side threads with Narrative Javascript but to try out these ideas I took a slightly different approach. Narrative Javascript changed the internals recently and I couldn't use the manually written CPS transformed functions like my previous threading implementation. This time I reimplemented things using the 'notifier' functionality that it has been replaced with.

For simple threading, instead of writing a scheduler I that has a queue of continuations to resume I just pass everything on to Javascript's 'setTimeout' function:

function Process() {
   this.pid = pid_count++;
   this.mailbox = new Queue();
   this.blocked = false;
}

Process.prototype.resumeAfter = function(func, ms) {
   var p = this;
   setTimeout(function() { self = p; func(); self = false; }, ms)
}

Process.prototype.resume = function(func) {
   this.resumeAfter(func, 10);
}

function concede() {
   var n = NjsRuntime.createNotifier();
   self.resume(n)
   n.wait->()
}

function sleep(ms) {
   var n = NjsRuntime.createNotifier();
   self.resumeAfter(n, ms)
   n.wait->()
}

function spawn(func) {
   var n = NjsRuntime.createNotifier();
   var p = new Process();
   p.resume(function() { func(); n(); });
   n.wait->()
   return p;
}

A 'future' is just a spawned thread that updates an object when it is complete. A process can block waiting for the result of the future. For this quick example I poll the object to see if it has completed, and give up a timeslice using 'concede' if it hasn't. In an actual framework I'd rewrite this to not poll, but instead to be notified when it is complete - I just wanted to prove the concept first:

function future(func) {
   var f = {}
   f.finished = false
   spawn->(function() { f.result = func->(); f.finished = true; })   
   return f;
}

function value(f) {
   while(!f.finished) {
      concede->();
   }
   return f.result;
}

An example of how this can be used using a 'fib' calculation:

function fib(n) {
   concede->()
   if(n==0) { 
      return 1;
   }
   else if(n==1) {
      return 1;
   }
   else {
      return fib->(n-1)+fib->(n-2);
   }
}

function test() {
   var f1 = future->(function() { return fib->(5); })
   var f2 = future->(function() { return fib->(6); })
   print(value->(f1) + value->(f2))   
}

Two threads are spawned as futures and then the results are used and printed. 'print' is a simple function to display the results in a div on the html page. If the threads have not completed then the process blocks until they are and the results provided.

Promises are implemented in a very similar manner:

function promise() {
   var p = {}
   p.finished = false;
   return p;
}

function fulfill(p, value) {
   p.result = value;
   p.finished = true;
}

function test2() {
   var p = promise();
   spawn->(function() { var v = value->(p); print("Thread 1 complete: " + v); });
   spawn->(function() { var v = value->(p); print("Thread 2 complete: " + v); });
   spawn->(function() { var v = value->(p); print("Thread 3 complete: " + v); });
   fulfill(p, 42)
}

Here I spawn three processes waiting on a single promise variable. When that promise is fulfilled the three threads resume and display the values.

You might notice that the process object has a 'mailbox' - this is for Erlang style message passing which I've also implemented. Here's how the lightweight message passing code looks:

function test3() {
   var p1 = spawn->(function() {
         var m = receive->();
         print(m[1]);         
         send->(m[0], "From p1!")         
      })

   var p2 = spawn->(function() {
         send->(p1, [ self, "From p2" ]);
         print("p2 sent to p1");
         var m = receive->();
         print(m);
      })

I think this implementation of threading using Narrative Javascript is a bit cleaner than my last implementation since it passes off most of the work to the browser's setTimeout function - and there's less code which is always good. It also uses the standard Narrative Javascript API rather than taking advantage of the internal structure of the generated code.

I'm rewriting the promise and future code not to use polling and tidying it up and I'll make the code and a live example available here over the next few days.

Tags: concurrency 

2006-09-26

More on Jwacs

Bill Clementson has a post summarising the lispvam meeting discussing jwacs. A great transcript of the presentation is also available.

Tags: continuations 

2006-09-19

Lazy file reading in Factor

I've added a couple of new lazy list routines in Factor to do lazy stream input/output. These are lazy counterparts to the standard contents and lines words.

The new words are 'lcontents' and 'llines'. The first returns a lazy list of all the characters in the file. The second returns a lazy list of all lines in the file. Both of these lazily retrieve from the stream as needed and can be used on files larger than the memory available to Factor:

"test.txt" <file-reader> llines [ . ] leach

While adding these I noticed the the mapping and subset operations didn't memoize their values. This meant the quotation for these operations were being called whenever the 'car' or 'cdr' of the cons was called. I added a <memoized-cons> type that wraps an existing cons and remembers the previous calls to 'car', 'cdr' and 'nil?'. 'lmap' and 'lsubset' automatically wrap their return type in a <memoized-cons>.

Now that they are memoized, the following 'fib' implementation works very fast:

: fib ( n -- )
  dup 2 < [
    drop 1
  ] [
    dup 1 - "fibs" get lnth swap 2 - "fibs" get lnth + 
  ] if ;

naturals [ fib ] lmap "fibs" set

5 fib
25 fib
60 fib

This creates a lazy list of fibonacci numbers. Retrieving the nth item from the list returns the nth fibonacci number. As the values are automatically memoized as a result of the lazy map operation, susbequent calls to 'fib' are very fast.

Tags: factor 

2006-09-17

Factor code to upload to the S1 MP3 Player

Futher to my post on using Factor to program the S1 MP3 Player, I got some basic code working to upload Z80 code and execute it.

It requires updates to the usb code in my Factor repository:

darcs get http://www.bluishcoder.co.nz/repos/factor

I've created a darcs repository to host the code. It's best to get the repository from within the Factor 'contrib' subdirectory so you can use the standard module system to load the code:

darcs get http://www.bluishcoder.co.nz/repos/s1mp3

libusb must also be installed. Once all that's done you can test things out by plugging in the MP3 player and navigating to the 'Update Firmware' menu option. The MP3 player then sits waiting to receive code.

The following will load a Z80 assembled file called 'test.bin', upload it to the MP3 player and start executing it:

"contrib/s1mp3" require
USE: s1mp3
"test.bin" s1mp3-run-file

The Z80 file must be assembled with an origin of 0x3400. For a quick test I've included in the repository Carlos Aguilar's LCD autodetect program. The binary is in 'detect/detect.bin' and can e sent to the player with:

"contrib/s1mp3/detect/detect.bin" s1mp3-run-file

This should find details about the LCD display on the device and display it. Hopefully this code will work on all supported Factor platforms as long as libusb is available. This covers Windows, Linux and Mac OS X that I know of.

If you get errors due to 'permissions' it may be because your user does not have permission to access the USB device directly. On Linux this can be fixed by running Factor as root. A better option though is to set up hotplug or udev to change the permission on the device when it is plugged in so you can access it.

Tags: factor 

2006-09-16

Writing your own MP3 player firmware

The S1 MP3 Player (or 's1mp3' as it is commonly known as) is a generic type of MP3 that is used and branded by a number of manufacturers. It uses a chip that is firmware updatable, where the firmware is written in Z80 machine code. A number of people have written third party tools allowing writing and running their own code on the MP3 player. The S1MP3 website and wiki has lots of details about this.

A program originally announced on the mailing list called loadram is used to upload Z80 code while the player sits on the 'upload new firmware' screen. This code is run, and can return back to the player. Not much seems to be known about the MP3 players yet but it is possible to write text to the LED screen on some models.

The manufacturers firmware updates can be 'unarchived', and they contain the Z80 binary programs that are run from the firmware menu. For example, the program to run the sound recorder is called RECORDER.AP. This can be replaced by your own Z80 program, repacked into the firmware and updated to the player. From then on, selecting 'Record' from the MP3 player menu will run your new program.

What interests me about this is it gives a small portable reprogrammable device. And hopefully something I can leverage Factor's interactive development system. I wrote an 8080 emulator to play the original Space Invaders arcade game written in Factor. 8080 and Z80 are quite similar. In fact I used Z80 assembler syntax in the 8080 emulator implementation.

It wouldn't be hard to base a Z80 emulator on top of this to allow testing programs to run on the MP3 player. And to write a Z80 assembler in Factor that generated the binaries to upload to the device. 'loadram' was Linux based and has been recently ported to Windows. It uses libusb to communicate with the player. By adding libusb wrappers to Factor it would be possible to assemble and upload directly to the player from a Factor environment - and it should work on Windows, Linux and Mac OS X.

I put together a quick wrapper around libusb and added it to my repository:

darcs get http://www.bluishcoder.co.nz/repos/factor

This has been tested to work on Linux and Mac OS X. I'll try adding a nice Factor wrapper around the alien definitions and get 'loadram' ported.

Note that to implement the alien wrappers I had to modify Factor's alien C structure wrappers to accept arrays of characters. The patch for this is in my repository but may not necessarily be pulled by Slava into the main Factor repository. I need to tidy it up and work on it before that might happen.

Once libusb is working and loadram is ported I'll try getting the Z80 emulator and disassembler going.

If you're looking for suitable players, Dick Smith Electronics in New Zealand, has at least two models that work. The first is their DSE branded 128MB MP3 player, model number A2439. This is a very basic model and is quite cheap. It has a black and white LCD screen which can be written too using the S1MP3 open source code.

Another model they have is the AStone Samba AV MP4 player. This has a colour OLED display and can play video and pictures - on its very tiny screen. Calling it an MP4 player is a bit of a stretch. It can't actually play MP4's. You use their client software to convert videos in other formats to an AMV format which has awful quality (not that it matters on the tiny screen) but otherwise works ok. The downside of this player is it seems that there is no known way of writing to the colour OLED displays yet.

What sort of things could be written to run on the MP3 players? I'm not sure yet but it'll be an interesting 'spare time' project to play with.

Tags: factor 


This site is accessable over tor as hidden service 6vp5u25g4izec5c37wv52skvecikld6kysvsivnl6sdg6q7wy25lixad.onion, or Freenet using key:
USK@1ORdIvjL2H1bZblJcP8hu2LjjKtVB-rVzp8mLty~5N4,8hL85otZBbq0geDsSKkBK4sKESL2SrNVecFZz9NxGVQ,AQACAAE/bluishcoder/-61/


Tags

Archives
Links