Bluish Coder

Little Smalltalk and Javascript for Minix

I've been working on porting some small programs to Minix to get familiar with what's required for ports. From there I hope to get some bigger stuff working that I really want to use like Factor and Erlang.

So far I have Little Smalltalk working (including the web based class browser and editor) and SpiderMonkey Javascript.

Porting is a little bit painful as there's quite a few things different under Minix. Eventually I want to try and develop some simple device drivers using these languages to see if the idea of user space device drivers in alternate languages is workable.

HTML Template Languages

There's a discussion going on in the Erlang mailing list about HTML template engines. This is JSP-style definition of HTML pages with portions replaced dynamically with data from an application.

I've leaned towards building HTML pages in code using Lisp style S-expressions for HTML, or Seaside style building of pages. This is also the approach I took in building the Factor continuation based web server and examples.

For the recent Factor program I built I wanted to use a template engine though. It was a bigger program and it was easier to modify templates and see the change automatically. For that I used an engine that another contributer had provided which allowed embedded Factor code.

The mailing list post pointed to a paper called 'Enforcing Strict Model-View Separation in Template Engines' which describes limiting the power of a template engine to prevent developers from embedding to much application logic in the template page. I was dubious but the paper is quite convincing.

It mentions 'StringTemplate', an engine built on these ideas. I'm still looking for a good way of doing templates in Factor, Erlang, and other languages, so this might be a good source of ideas.

If anyone has pointers to other articles or ideas on the topics please feel free to leave a comment or email me.

Lightweight threads with Narrative Javascript

I've put together a very simple lightweight threads example with Narrative Javascript. The demo creates two threads:

function t1() {
  var element = document.getElementById("one");
  var count = 0;
  while(true) {
    sleep->(100);
    element.innerHTML=count++;    
    concede->();
  }
}

function t2() {
  var element = document.getElementById("two");
  var count = 100;
  while(true) {
    sleep->(100);
    element.innerHTML=count--;    
    concede->();
  }
}

They set the text of an element on the page to the result of an incrementing or decrementing counter. It then sleeps for approximately 100 milliseconds. These are wrapped inside a process object and a scheduler started:

function start_demo() {
  var t = [];
  t[0] = new Process(t1);
  t[1] = new Process(t2);
  start(t);
  start_scheduler();
}

This start_demo is called when a button is pressed on the page and the threads start running. 'setTimeout' is used for the threads to give up timeslices back to the browser and to prevent the 'too much recursion' error that can otherwise occur.

The demo page is here. Source code to the scheduler is in schedule.njs and for the thread1 demo in thread1.njs.

Unenterprisey languages going strong

The Unenterprisey Languages Meeting held in Wellington yesterday went very well. The turnout was good with about 16 people there and the talks by Jonathan, Robert and Geoff were great. It's good to see and hear about languages like Io, Common Lisp and Erlang being used.

Jonathan talked about Io's prototype based OO system and how inheritance works in it. He went through an overview of Io itself, how the inheritance works, and how he used ideas gained from Io in a project he worked on recently. His project used a domain specific language he developed that used the inheritance ideas of Io to help manage the project. It looked pretty cool.

Geoff talked about Erlang, going through its history, syntax, reliability, and concurrency system. It was good to see him talk about the OTP system as I wasn't very familiar with that and his demo showed sending messages across processes with dynamic failover of nodes. When the primary node went down the secondary node took over. When it came back up again it again took over. Very neat!

Robert Strandh talked about GSharp, his musical score editor written in Common Lisp. It uses CLIM for the GUI and looks very nice. The focus of the talk was how he embedded a domain specific language into GSharp to handle fonts better. Specifically the language was modelled after MetaFont, but embedded as a DSL in Common Lisp as a macro. This would then get compiled as PostScript bezier paths. The macro itself was a nice example of intentional variable capture (ie. It was intentionally a non-hygienic macro) which was something I'd not seen in a 'real world' usage before.

Robert also showed another CLIM application he wrote to preview the fonts from the DSL. In 172 lines of Common Lisp code it had an impressive amount of functionality. CLIM looks to be a very powerful framework.

My talk was about Javascript. But Javascript used slightly differently than normal. I gave a brief overview of Javascript and its prototype based OO system. I then talked about using Javascript on the server as well as the client, and about the possibilities of having the same code run on both the server and client for things like field validation. This way one copy of the code can validate input fields in client side Javascript and when posted to the server the same code can also check that the server side POST was valid.

To demonstrate running the same code on the client and server I used Wiky, a nifty Javascript bidirectional markup converter. I showed the Wikybox running on the client, and demonstrated loading the exact same Javascript in Rhino and doing the same conversions on the server. The idea being you can have the markup converted to HTML for a preview on the client, send the markup to the server when saving, and use the same code on the server to store the HTML.

From there I demonstrated using Server Side Javascript to dynamically modify a running web server, adding servlets on the fly in Javascript.

I went on to talking about continuations and Javascript. For client side continuations I showed some Narrative Javascript examples and went through the lightweight threading and process communication examples I posted about previously.

Finally I finished with server side continuations in Javascript showing the thread migration example.

After the talks we all went out for a great meal at a local restaurant and ate, drank, and talked for most of the night.

Overall I'd say the alternative language interest and usage in New Zealand is definitely looking good. Thanks to everyone who attended and those that organised it - very worthwhile!

More on Ajax and server push

So far I've looked at two ways of doing server push. The first used a hidden IFRAME which had the SRC attribute set to a persistent connection on the server. The server would push Javascript code along this connection whenever it wanted the client to do something. This had the downside of the browser always displaying a 'loading' indicator as the page never completed loading. From a UI perspective this makes it unusable except as a fallback method of last resort.

The second was outlined in my previous post and used XMLHttpRequest. Like the IFRAME, the request was for a persistent connection. This time the browser wouldn't display a 'loading' indicator and was pretty much exactly what I wanted. I'd prefer it if I didn't have to close the connection after each event to prevent the need to re-establish a connection as this limits scalability a little.

The problem with not closing the connection is you don't know when an event is completely sent back to the client so there is no real guarantee that the client has received the entire event data at one time.

A way of getting around this is to use a mime type of multipart/x-mixed-replace. This allows you to define the 'chunks' in your page. So the browser knows when to process each chunk. Not all browsers support this for XMLHttpRequest initiated transactions though. Mozilla appears to have recently got support for it but IE does not. Alex Russell points to a Microsoft Support article that provides another means to do the same sort of thing but then you have to have two data formats depending on which method you are using.

Even if multipart/x-mixed-replace was the tool of choice there is one further problem with persistent connections. Most (if not all) web browsers only have a limited number of connections it uses for HTTP requests. Internet Explorer only uses two for example. This means if you use one for a persistent connection then all other requests only go through the single connection. If you load another page that uses a persistent connection then you're stuck. No more HTTP requests can be made. This pretty much kills the idea of keeping the connection persistent for general use, and is another reason why my previous post closed the connection between events.

In fact, I've since changed the example (not yet available for download) so that it closes the connection after a set timeout as well, to allow other HTTP requests to be processed. I was having problems with using Backbase whereby some of its server requests were blocking due to the persistent connection being around.

I've done some research over the last couple of days and found a third possible option. Flash allows persistent connections to the server via its XMLSocket class. It is also possible to communicate between the browser and an embedded flash application via Javascript. This would allow writing a small flash application that did nothing but make the connection to the server and then Javascript is used for sending and receiving events across that connection.

This has the advantage of using a different connection pool that the browsers, and you can use any protocol. The disadvantage is that XMLSocket can only connect to ports numbered greater than 1024 and it is not tunneled over HTTP so could be blocked by a firewall. But if the firewall issue is not a problem then it would be possible to write a small Erlang tcp server that communicated across that socket and to the browser.

Backbase actually does something very similar for its charting controls. These controls are embedded flash with the data sent to the control using Javascript from the browser. It does not use a socket connection to the server however.

Another library that does similar is the Canvas object in Nevow. That library embeds a flash object in the page which is used for displaying vector graphics. It makes a connection to the server and allows the server to update the canvas as desired via communication over that socket.

The best general library for doing this sort of thing would seem to be the AFLAX library. This amazing library allows calling any Flash library call from Javascript. Unfortunately it's Flash 8 only and that version of Flash is not yet available for Linux as far as I'm aware.

What I plan to do is write a small flash application to embed to do the server connection and explore using that for the server push events. As a fallback I can have the library use XMLHttpRequest if there is a problem due to firewall restrictions. The final fallback could be an IFRAME if the browser doesn't support XMLHttpRequest.