Bluish Coder: self

Exporting Self objects to source files using the Transporter

2015-11-19T15:00:00+13:00

Following on from my previous video on using the Self GUI to create objects, I've done another that demonstrates how to export the objects to Self source files where they can be managed with a version control system and shared with other developers.

The video is available as self_transporter.webm, self_transporter.mp4 or on YouTube.

More information on using the transporter tool can be obtained from:

Demo of programming in Self

2015-11-18T15:00:00+13:00

The Self programming language has a GUI environment which can be a bit intimidating to people new to the system. The Self Handbook covers it but it's difficult to convey usage with text and static images.

I've created a video that demonstrates how to develop in Self using the GUI to create a 'bank account' object and manipulate it. The idea of this was based on a discussion on the Self issue tracker about tutorials for Self.

The video is available as self_bankaccount.webm, self_bankaccount.mp4 or on YouTube.

Improving Linux Font Support in Self

2015-01-08T19:00:00+13:00

In the Linux version of the Self programming language implementation the fonts used are standard X11 fonts. On modern Linux systems these don't look great and a common question asked in the mailing list is how to improve it. Fonts on the Mac OS X build of Self use a different system and they look much better. It would be good to convert the Linux version to use freetype to gain more control over fonts.

I worked on adding Freetype support a couple of years ago and wrote about it on the mailing list. I haven't done much on it since then but the code is in my github repository under the linux_fonts branch. That work adds Self primitives to access Freetype but does not integrate it into the Self font system. I hope to be able to continue this work sometime but I'm unlikely to get to it in the near future. This post is to point to the code, show how to use the primitives, in case someone else would like to take it forward as a project.

To try the code out you'll need to build Self in the usual manner but using that branch. Once the desktop is launched I test it by creating an object with the following slots:

(| parent* = traits oddball.
   window <- desktop w anyWindowCanvas.
   draw <-- nil.
   font <- nil.
   xftcolor <- nil.
   xrcolor <- nil.
 |)

With an outliner for this object I create an evaulator for it and, one at a time, evaluate the following code snippets:

draw: window display xftDrawCreate: window platformWindow
  Visual: window display screen defaultVisualOfScreen
  Colormap: window display screen defaultColormapOfScreen

xftcolor: xlib xftColor new.
xrcolor: xlib xRenderColor new.
xrcolor alpha: 16rffff

window display xftColorAllocValue: window display screen defaultVisualOfScreen
  Colormap: window display screen defaultColormapOfScreen
  RenderColor: xrcolor
  XftColor: xftcolor

font: window display xftFontOpenNameOnScreen: window display screen number
 Name: 'verdana-18'

draw xftDrawString8: xftcolor
  Font: font X: 100 Y: 100
  String: 'Hello World!'

The results in the text 'Hello World!' appearing at position 100@100 on the desktop in the Verdana font. The video below demonstrates this to show how it's done in the Self user interface:

The video is available as self_font.webm or self_font.mp4.

This is a common workflow I use to prototype things in Self. I create an empty object and populate it with slots to hold data. With an evaluator created for that object these slots are accessible without needing to have an object to call them on. This is known as 'implicit self calls'. The message for the slot implicitly is sent to the current object. I create and remove slots as needed. I can use the Self outliner to drill down on the slots to look at and manipulate those objects specifically if needed.

Hopefully this Freetype support can be used as a base for better looking fonts on Linux. If you are keen to take it further, or have ideas on how to integrate it, I can be contacted using the details at the bottom of my blog, or you can raise it in the Self mailing list or on github.

Self Benchmarking

2014-12-22T23:00:00+13:00

I was asked on twitter about the current speed of the Self implementation. The request was for a method send benchmark so I wrote a simple one and compared against Pharo, a Smalltalk implementation.

The implementation and technology behind Self is quite old in comparison to modern compiler implementations but at the time it was state of the art. I hoped it would hold up reasonably well. The test I wrote in Self was:

(|
  doSomething = ( ^self ).
  test = ( |n <- 0|
           [ n < 100000000 ] whileTrue: [ doSomething. n: n + 1 ]
         )
|)

Running this in the Self shell shows:

"Self 1" _AddSlots: ...code snippet from above...
shell
"Self 2" [ test ] time.
2587

2.5 seconds seems a bit slow to me but I tested in Pharo to confirm and to see how it compares. The Pharo code looks almost exactly like the Self code:

doSomething = ^self
test = |count|
       count := 0.
       [ count < 100000000 ] whileTrue: [
         count := count + 1.
         doSomething.
       ].

[ MyObject new test ] timeToRun
  => 0:00:00:00.239

That's 239ms vs 2,587ms, a factor of over 10x. Further investigation revealed that calling 'time' in Self seems to cause the code to run slower. If I call the 'test' method first, and then call 'time' then it's much faster:

"Self 2" [ test ] time.
2587
"Self 3" [ test ] time.
2579
"Self 4" test.
nil
"Self 5" [ test ] time.
650
"Self 6" [ test ] time.
628

At 650ms it is about 2.7x slower than Pharo, an improvement over 10x. More investigation is needed to see if there is room for other improvements.

The Self implementation has some primitives that can be changed to show debugging information from the JIT. All primitives can be listed with:

primitives primitiveList do: [ | :e | e printLine ].

Looking through this shows some interesting ones prefixed with _Print that can be set to output debug data. One is _PrintCompiledCode. Setting this to true allows viewing the generated assembler code on the Self console.

"Self 16" _PrintCompiledCode: true.
false
"Self 17" 40 + 2.
...
  // loadOop2
movl $0xa0 (40), #-16(%ebp)
  // loadOop2
movl $0x8 (2), #-20(%ebp)
  // loadArg
movl #-20(%ebp), %ebx
movl %ebx, #4(%esp)
  // selfCall
movl #-16(%ebp), %ebx
movl %ebx, (%esp)
nop
nop
nop
call 0x8186597 <SendMessage_stub> (bp)
  // begin SendDesc
jmp L7f
  .data 3
jmp L9f
  .data 0
  .data 0
  .data 0x4578341 ('+')
  .data 4
L7: 
L8: 
  // end SendDesc
movl %eax, #-16(%ebp)
  // epilogue
movl #-16(%ebp), %eax
  // restore_frame_and_return
leave
ret

Others, like _PrintInlining display debug information related to inlining code.

"Self 18" _PrintInlining: true
fales
"Self 19" test.
*inlining size, cost 0/size 0 (0x8b7e864)
*PIC-type-predicting - (1 maps)
*type-casing -
*inlining - (smallInt.self:153), cost 1/size 0 (0x8b7ee38)*
 *inlining asSmallInteger (number.self:108), cost 1/size 0 (0x8b7fa94)*
  *inlining raiseError, cost 0/size 0 (0x8b80530)*
  *inlining asSmallIntegerIfFail: (smallInt.self:302), cost 0/size 0 (0x8b808fc)*
 *inlining TSubCC:
 *cannot inline value:With:, cost = 10 (rejected)
 *marking value:With: send ReceiverStatic
 *sending value:With:
*sending -
*inlining size:, cost 0/size 0 (0x8b8434c)
*inlining rep, cost 0/size 0 (0x8b846a8)
*PIC-type-predicting removeFirstLink (1 maps)
*type-casing removeFirstLink
*inlining removeFirstLink (list.self:300), cost 2/size 0 (0x8b84b48)*
 *inlining next, cost 0/size 0 (0x8b85628)
 *PIC-type-predicting remove (1 maps)
 *type-casing remove
 *cannot inline remove, cost = 9 (rejected)
 *sending remove
*sending removeFirstLink
*PIC-type-predicting value (1 maps)
*type-casing value
*inlining value, cost 0/size 0 (0x8b86570)*
*sending value
*inlining asSmallInteger (number.self:108), cost 1/size 0 (0x8b7e5b0)*
 *inlining raiseError, cost 0/size 0 (0x8b7f074)*
 *inlining asSmallIntegerIfFail: (smallInt.self:302), cost 0/size 0 (0x8b7f440)*
*inlining TSubCC:
*cannot inline value:With:, cost = 10 (rejected)
*marking value:With: send ReceiverStatic
*sending value:With:
nil

For more involved benchmarks there is some code shipped with the Self source. It can be loaded with:

"Self 28" bootstrap read: 'allTests' From: 'tests'.
reading ./tests/allTests.self...
reading ./tests/tests.self...
reading ./tests/programmingTests.self...
reading ./tests/debugTests.self...
reading ./tests/lowLevelTests.self...
reading ./tests/numberTests.self...
reading ./tests/deltablue.self...
reading ./tests/sicTests.self...
reading ./tests/branchTests.self...
reading ./tests/nicTests.self...
reading ./tests/testSuite.self...
reading ./tests/languageTests.self...
reading ./tests/cons.self...
reading ./tests/benchmarks.self...
reading ./tests/richards.self...
reading ./tests/parser.self...
reading ./tests/parseNodes.self...
modules allTests

There are methods on the bootstrap object for running the tests and printing results. For example:

"Self 32" benchmarks measurePerformance

                 compile    mean       C    mean/C       %
recur:                 5       0
sumTo:                 2       7
sumFromTo:             2       7
fastSumTo:             2       6
nestedLoop:            2      10
...

There is also measurePerformance2 and measurePerformance3 methods. The code comments for the measure2 and measure3 methods explain the differences.

Self 2 was well known for generating very fast code that compared favourably with C. This implementation of this was described in Craig Chamber's thesis. Compilation was slow however so in Self 3 and 4 two new compilers were created. These were 'nic' and 'sic'. I believe this is covered in Urs Holzle's thesis The 'nic' compiler is the 'Non Inlining Compiler' and is simpler to implement. It's the compiler you write to get Self bootstrapped and running on new platforms fairly quickly. There is no inlining and no type feedback so performance is slower as shown by the benchmarking when changing the compiler used, as described below. The 'sic', or 'Single Inlining Compiler', generates better code through more optimisations. While neither is as fast as the Self 2 compiler it is faster to compile code and makes for a better interactive system. You can read more about this in the Merlintec Self FAQ.

There is a defaultCompiler slot in the benchmark object that can be set to nic or sic to compare the different JIT compilers that Self implements. Comparing the 'nic' compiler vs the 'sic' compiler shows a speedup of about 6x in the 'richards' benchmark when using 'sic'.

There's probably a fair bit of low hanging fruit to improve run times. I don't think the x86 backend has had as much work on it as the Sparc or PPC backends. The downside is much of the compiler code is written in C++ so for people interested in 'Self the language' it's not as fun to hack on. Klein was an attempt to write a Self VM in Self and includes a compiler and assembler which might make a more interesting project for those that want to use Self itself to implement compiler code.

Changing Attributes in Self objects

2014-12-17T17:00:00+13:00

A question came up in the Self irc channel today about how to change the 'comment' string for an object. Comments are stored as attributes on an object. These are accessed using mirrors.

A mirror for an object can be obtained using the asMirror method. The trait traits mirrors abstractMirror provides a number of methods that can be used to change the object via the mirror. This involves everything that is normally done through the programming environment. It includes adding, removing and changing slots, categories, attributes amongst other things. By drilling down in this trait you can see a programming category, which expands to include a changing annotation category and that holds a comments category. Within that you can find the methods for setting comments.

The video below shows the steps to find this from within the programming environment and demonstrates changing the comment on the initial shell object on the desktop. The code to change that comment is just:

self asMirror comment: 'hello world'

The video is available as self_comment.webm or self_comment.mp4.

Multiple Users in a Self World

2013-09-16T23:00:00+12:00

One of the things I like about the Self programming language is it has lots of interesting old projects written in Self included in the source distribution. These include a Dylan interpreter, Cecil interpreter, web browser, and partial Java implementation amongst other things.

Once of the features Self has is the ability to allow remote users to connect to a Self instance running the Morphic user interface. This is usually done using X11's facility to share remote windows. But included in the Self source is a client written in Java and a server written in Self that allows remote users to connect using a Java Applet in a web browser.

The code for this has bitrot slightly but I've updated it just enough to work and demo. My fixes have been merged into the official Self repository on github.

Server side

With these changes and a build of Self you should start with a fresh image containing the Morphic code (UI2). From the self console run:

$ cd objects
$ ../Self 
"Self 1" 'applications/javaServer/loadJavaServer.self' _RunScript

This will load the server code. It's best to do this in a fresh instance as it modifies some UI objects that result in Self having to rebuild a number of dependencies for the objects in the system. This can make an existing system sluggish for a bit.

Once loaded, start the desktop:

"Self 2" desktop open

From a shell in the desktop, create a copy of the javaDaemon object using 'Get it' and put it somewhere:

javaDaemon copy

The outliner for this allows changing the port slot if needed, which defaults to 1420. Middle click on the outliner's title and choose 'Show Morph'. This creates a default tan square that can be placed anywhere. The world that this morph lives in is the world that remote users will connect to.

From the outliner run the start method. There will be a prompt warning you that this is a security issue as you're allowing remote access to your machine. Answering Yes will run the server.

Client side

The client code is written in Java. The source is in objects/applications/javaClient. The shell script BuildJavaClient will build the Java code. The Test.html page in that directory should be served from a web server and accessed by a web browser. Note that due to Java security settings this must be served from an actual web server. I used a local thttpd instance. There may also need to be some changing of Java Applet settings in the Java control panel to allow the applet to make socket connections.

Once 'Test.html' is loaded you can enter a username, keep the password empty, and connect. You should now see the Self desktop.

Usage

Unfortunately when the client connects sometimes the Self desktop stops and it won't respond. To restart it enter in the Self console:

desktop go

When this is done you'll see the remote users 'hand' (Self term for mouse pointer) in the Self desktop as an arrow with their name. A border will be in the world with the color of the user showing the extent of the desktop that the user can see in their remote session.

Dragging items around in both sessions is reflected in the world. Other users can connect too. The Java client uses 'Shift+Left Click' as the right mouse button and 'Control+Left Click' as the middle mouse button.

Demo

I did a short screencast of using the remote sharing to show how it works. If your browser supports HTML video it should show below.

If your browser doesn't support HTML video, you can view directly kansas.ogv or kansas2.mp4.

Future

I think a dedicated protocol like this is probably better than using remote X11. Self currently is quite tied to X but if it could be moved off it it would allow easier porting to Windows, Android and other non-X platforms. Perhaps using something like SDL2 to get cross platform and hardware acceleration.

More information about the protocol and implementation of the Java client is available in JavaClientServerOverview.txt

An interesting project would to be port the javaClient code to JavaScript using an HTML canvas or WebGL to display the remote work. The server could be updated to communicate via HTTP, WebSockets, or even WebRTC's peer to peer facilities.

Running the Self Programming Language on Android

2013-09-11T23:00:00+12:00

I've written about the Self programming language before and still occasionally fire up the GUI environment to explore prototype object development. The Self implementation is JIT based and doesn't have an interpreter. I've wanted to have the Self environment running on a mobile device but most tablets and phones are ARM based and Self doesn't have an ARM backend.

Intel based mobile devices are starting to appear however so I decided to try and get Self running on one. The GUI backend for Self on Linux uses X11 and it's a bit too large a project to do an Android GUI backend so I just tried getting the console REPL of Self working.

Building on Linux

My first step was to revive my port of the Self build system that uses Tup instead of the original custom shell scripts. By doing this I'd gain a better understanding of how Self is built and then move to using Android tools. My rebased version of Self that uses tup to build is in the tup branch of my Self github fork. With tup installed this can be built with (probably works on Linux only, sorry):

$ git clone https://github.com/doublec/self
$ cd self
$ git checkout -b tup origin/tup
$ tup init
$ tup upd

This creates a Self executable in the root of the Self checkout. A bootstrapped snapshot can be created with:

$ cd objects
$ ../Self 
Self Virtual Machine Version 4.1.13
Copyright 1989-2003: The Self Group (type _Credits for credits)

for I386:  LogVMMessages = true
for I386:  PrintScriptName  = true
for I386:  Inline = true
for I386:  SICDeferUncommonBranches = false (not implemented)
for I386:  SICReplaceOnStack = false (not implemented)
for I386:  SaveOutgoingArgumentsOfPatchedFrames = true
VM# 'worldBuilder.self' _RunScript

The 'worldBuilder.self' _RunScript command loads a script that loads the code for the standard Self runtime. It prompts if you want UI2, which you do if you want the GUI, and UI1, which you don't unless you want to explore the old gui.

Once loaded you can run some basic commands:

"Self 0" 'hello world' print
hello world'hello world'
"Self 1" 100 factorial.
a bigInt(9332621544394415...00000000000000000000)
"Self 2" [ 1000 factorial ] time.
1552
"Self 3" desktop open.

The last comand, desktop open, runs the Self GUI. You can see a screencast of me using the GUI in my displaying images with Self post.

Android Standalone Toolchain

To build Self for Android we need to generate a standalone toolchain from the Android NDK. To generate an x86 toolchain:

$ cd $NDK_HOME
$ ./build/tools/make-standalone-toolchain.sh  \
     --arch=x86 \
     --install-dir=/somewhere/stk

This generates the toolchain and copies it to the /somewhere/stk directory. That should be changed to where you want it installed and it needs to be added to the path:

$ export PATH=/somewhere/stk/bin:$PATH

Android ncurses

Self uses termlib from the ncurses library. This can be built from source and installed using the standalone toolchain:

$ wget http://ftp.gnu.org/pub/gnu/ncurses/ncurses-5.9.tar.gz
$ tar xvf ncurses-5.9.tar.gz
$ cd ../ncurses-5.9
$ ./configure --host=i686-android-linux \
              --prefix=/somewhere/stk \
              --with-termlib
$ make
$ make install

The prefix in the configure line above will install it in the location of the standalone toolchain so it can be found easily when building Self.

Android Self

I've generated a patch file that allows building Self in Android. It's pretty rough but gets the console REPL working and can then be fixed up later. To apply and build:

$ git clone https://github.com/doublec/self
$ cd self
$ git checkout -b tup origin/tup
$ curl http://bluishcoder.co.nz/self/android.patch | patch -p1
$ tup init
$ tup upd

The resulting executable should be pushed to a directory on the Android device. Pushing the objects directory will provide the files to allow bootstrapping the Self runtime system:

$ adb push Self /data/local/tmp
$ adb push objects /data/local/tmp
$ adb shell
device$ cd /data/local/tmp
device$ chmod 0755 Self
device$ ./Self
VM# 'worldBuilder.self' _RunScript

You can also run Self from within a terminal emulator. If you do this I recommend installing the Hacker's Keyboard as well.

Running the benchmark suite on my laptop gives about 2 seconds elapsed time. On the Asus Fonepad it's 10 seconds, so approximately 5x slower. The benchmark can be run with:

"Self 0" [ benchmarks run ] time

Binaries

A precompiled binary is available at i686-android-linux-Self.bz2. Uncompress it before copying it to the device. You'll need the objects directory from the git source to boostrap the runtime environment:

$ wget http://bluishcoder.co.nz/self/i686-android-linux-Self.bz2
$ bunzip2 i686-android-linux-Self.bz2
$ mv i686-android-linux-Self.bz2 Self
$ adb push Self /data/local/tmp
$ git clone https://github.com/doublec/self
$ cd self
$ git checkout -b tup origin/tup
$ adb push objects /data/local/tmp
...as previously to run...

Summary

This provides basic console Self usage. It requires knowing how to program in Self without using the GUI, which is not the most common way of writing Self programs. An interesting next exercise would be trying to get a GUI working.

The images below show Self running on two x86 Android devices. On the left is a screenshot of running on a Samsung Galaxy Tab 3 10.1 and on the right running on an Asus Fonepad with bluetooth keyboard. Click on the images for the larger version.

Unfortunately this build doesn't work on ARM as mentioned previously. I suspect an ARM backend would be a significant amount of work.

For more reading on Self:

Self Handbook
Self Papers
Prototype Based Programming Languages
Attack of the clones
self-interest mailing list
Self IRC channel is #self-lang on Freenode

Displaying images with Self

2009-07-27T23:32:00+12:00

The Self source code has some example programs, one of which is a web browser. This was developed quite a long time ago and is unable to nicely display most pages on the modern web but it does a reasonable job of basic pages. On the Linux port there were some problems with displaying images however. I tracked this down to two issues:

It only natively decoded image formats that aren't really used on the web (Sun Raster format for example). The GIF and JPG handling code spawned external programs, which I didn't have on my Linux install, to convert to a format it could handle before displaying.
There was a bug in the color management code on X11 for displays greater than 8 bit.

The first was a quick fix by changing the programs that it used to do the conversion to one readily available on Linux. I changed it to use the Image Magick 'convert' program. This enabled displaying code in an imageMorph by setting the image for the morph:

setImage: ui2Image copyFromGIFFile: 'test.gif'

I tracked the second issue down to code that was sending a message to an object that didn't understand that message (Static typing fans can say 'I told you so' here). Self has a 'paintManager' which handles color mapping for displays that can't display a large number of colors. On 8 bit displays this is a 'paintManager'. On displays with more colors it's a 'unmappedPaintManager'. The latter doesn't have some of the slots of the former and there was code trying to do this. I changed this to check if the paintManager instance can do color mapping before sending the messages.

With that change the web browser could display images. An example page that works with the browser (one of the few...), and code to display it:

'http://www.bluishcoder.co.nz/self/imagetest.html' asURL getPageForUser

The patches with these fixes are available from the webbrowser branch of my fork of the Self git repostory on github. The fixes were only a few lines of code.

I did a short screencast showing loading image morph's and displaying the web page in the browser. And before you say, "wow, that GUI looks so 90's", don't forget - it was.

Screencast

Building and using Self on Linux

2009-07-27T00:02:00+12:00

The original implementation of the Self programming language is easy to build from source. Russell Allen maintains a git repository on github containing the source.

I use Arch Linux with 'flex' and 'tcsh' packages installed for various parts of the build process. To build a Self binary:

$ git clone git://github.com/russellallen/self.git
$ cd self/release
 $ sh buildLinuxVM
$ cd ..
$ ./Self

The build takes about 30 minutes on my laptop. There will be many compilation warnings but these can be safely ignored. The result of the build is a 'Self' executable in the root directory of the cloned source. The executable can be run but most of the Self library is not loaded. You need to create an image to run or use an existing one. About the only thing you can do with the Self VM without an image is use primitives.

The 'objects' directory contains the source for the core Self library, the User Inteface, as well as other applications and interesting stuff. These can be used to build an image to run Self. The file 'all2.self' can be used to create an image including the graphical user interface. 'small.self' can be used to build an image without the user interface. The primitive '_RunScript' is used to load these files. Once loaded you can save a 'snapshot' (this is the term Self uses for images) which can be resumed in later sessions.

$ cd objects
$ ../Self
Self Virtual Machine Version 4.1.13, Sun 26 Jul 09 16:59:28 Linux
Copyright 1989-2003: The Self Group (type _Credits for credits)

for I386:  LogVMMessages = true
for I386:  PrintScriptName  = true
for I386:  Inline = true
for I386:  SICDeferUncommonBranches = false (not implemented)
for I386:  SICReplaceOnStack = false (not implemented)
for I386:  SaveOutgoingArgumentsOfPatchedFrames = true
VM # 'all2.self' _RunScript
reading all2.self...
reading ./core/init.self...
reading ./core/allCore.self...
reading ./core/systemStructure.self...
reading ./core/defaultBehavior.self...
...
reading ./ui2/outliner/profileSliceGrpMod.self...
reading ./ui2/outliner/profileSelfSlotMdl.self...
reading ./ui2/outliner/powerOperations.self...
verifying newgen: eden from to oldgen: old0 old1 z p r S v O m N M i  done
Starting: Collecting Garbage...
Finished: Collecting Garbage
Starting: Refilling module cache...
Finished: Refilling module cache
"Self 0" saveAs: 'ui.snap'
Starting: Writing snapshot to ui.snap...
Finished: Writing snapshot to ui.snap
shell
"Self 1" quit
Save to ui.snap before quitting? 
  y => save, then quit
  n => quit without saving
  RET => cancel
Response: n

Notice the call to 'saveAs:' to save the image snapshot. 'quit' is used to exit the running Self session. This image can now be resumed using the '-s' argument to 'Self' whenever you want to run Self:

$ cd objects
$ ../Self -s ui.snap
for I386:  LogVMMessages = true
for I386:  PrintScriptName  = true
for I386:  Inline = true
for I386:  SICDeferUncommonBranches = false (not implemented)
for I386:  SICReplaceOnStack = false (not implemented)
for I386:  SaveOutgoingArgumentsOfPatchedFrames = true

Welcome to the Self system!  (Version 4.4)


Copyright 1992-2009 AUTHORS, Sun Microsystems, Inc. and Stanford University.
See the LICENSE file for license information.

Type _Credits for full credits.

VM version: 4.1.13

"Self 1"

At this point you can now use any Self expressions. All code is run in the contents of an object called 'shell'. You can start the graphical user interface by passing the 'open' message to the 'desktop' object:

desktop open

The desktop will appear with a 'trash can' object and an outliner for the 'shell' object. Left clicking on the middle button labelled 'E' in the group of three buttons on the right of the shell will open an expression evaluator allowing you to enter Self expressions. Middle click on objects to get an object specific menu. On the desktop this provides a menu item to quit when you want to leave the session.

There are some interesting applications in the 'objects' directory. For example, there is a Smalltalk implementation written in Self. This can be loaded at the shell with the command:

bootstrap read: 'smalltalk' From: 'applications/smalltalk'

When it completes loading you can find a 'smalltalk' item on the desktop's right click menu. Choosing that provides a menu to load a standard Smalltalk browser, workspace and inspector, allowing you to write Smalltalk code. There's a document that describes the Smalltalk emulator for more information about it. There are other interesting applications in the objects directory including a web browser, Java emulator, a C preprocessor, Cecil implementation and a parser/lexer generator. Most of this code is old and in various states of usability but it provides some interesting examples of Self code to look at and play with.

The Self tutorial provides a good introduction to using the Self user interface.

Prototype Based Programming Languages

2009-07-16T13:11:00+12:00

I've been reading up on protoype based programming languages recently. Mainly using the Io Programming Language and Self but also looking at LambdaMOO and similar languages. A good overview of using the prototype based approach to building programs is Organizing Programs without Classes. This post is based on examples from that paper and from Attack of the Clones which covers design patterns using Self.

Self

In the Self programming language objects are created using a literal object syntax. This syntax defines code and slots within the (| and |) delimiters. An example object looks like:

point = (|
  parent* = traits point.
  x.
  y.
|)

This creates an object with 3 slots. The slots are 'parent', 'x' and 'y'. The 'parent' slot is assigned an inital value of 'traits point' which is the traits object for points (more on what this is later). The '*' that is suffixed to the 'parent' slot means it is a prototype slot and used in the prototype slot lookup chain.

This means when a message is sent to the 'point' object the lookup starts with the 'point' object. If a slot with the messages name is not found in that object then each prototype slot (those suffixed with '*') are searched looking for a slot with that name.

So in the 'point' case, looking for 'x' will find it immediately in the point object. Looking for 'print' will not so it will look for it in the 'traits point' object. If it's not there it will look in that objects prototype slots and so on until it is found or the search is exhausted.

The idiom in Self (and other prototype based languages) is to create global objects like these and use 'clone' to create copies of it. So creating two different points would look like:

a = point clone.
b = point clone.

Notice that 'point' has no method slots. Only the 'x' and 'y' which contain data. The methods are defined in the 'traits point' object. The definition of that could look something like:

traits point = (|
  parent* = traits clonable.
  print = x println y println.
|)

This provides a method to print the 'x' and 'y' values of the object and another parent that provides other basic functionality like the ability to clone. The 'traits point' object doesn't define any data slots. It defines only methods. However it uses 'x' and 'y' messages that aren't defined. It expects to be used in a prototype slot of another object that defines the 'x' and 'y' slots (like our 'point' example earlier.

Separating the code out into data objects and trait objects allows the trait object to be reused in other objects. For example, an object that computes the 'x' and 'y' values rather than storing them can re-use the traits object:

computed_point = (|
  parent* = traits point.
  x = ..compute x..
  y = ..compute y..
|)

This 'computed_point' can be used anywhere a point is expected. The traits object sends the 'x' and 'y' messages when it needs their value and it doesn't matter if there stored as data as in the 'point' object, or as methods that calculate the value as in the 'computed_point' object. Each 'point' and 'computed_point' object shares a single trait object instance. This avoids the need to have multiple copies of the methods in each point object instance.

The prototype slots of an object effectively form an inheritance relationship. The way to subclass objects in Self is to create a new object and set a prototype slot to an instance of the parent object. Usually it's the trait objects that map the subclass relationship since it is those objects that contain the object behaviour (ie. the methods and no data). An example follows in Self of how a 'polygon' and 'filled_polygon' object can be modelled (this is from the 'Organizing Programs without Classes' paper):

polygon_traits = (|
  draw = ...draw using 'vertices' slot to get points...
|)

polygon = (|
  parent* = polygon_traits.
  vertices.
|)

filled_polygon_traits = (|
  parent* = polygon_traits.
  draw = ...draw using 'vertices' and 'fillPattern'...
|)

filled_polygon = (|
  parent* = filled_polygon_traits.
  vertices.
  fillPattern;
|)

Cloning the 'polygon' object and sending the 'draw' message will draw the polygon using the list of points held in the 'vertices' slot. Cloning the 'filled_polygon' object and sending the 'draw' message will draw using the specialized 'draw' method that also uses the 'fillPattern' slot to fill the polygon when drawing. This can re-use the 'draw' method in the 'polygon_traits' object if needed.

The new 'filled_polygon' object did require defining a new 'vertices' slot. Self allows multiple prototype slots, each of which is involved in the lookup for slot names. We can share the 'vertices' from the 'polygon' object by making that an additional prototype slot in 'filled_polygon'. This is often termed a 'data parent':

filled_polygon = (|
  parent* = filled_polygon_traits.
  dataParent* = polygon clone.
  fillPattern;
|)

Notice the 'dataParent' slot is a prototype slot (suffixed with a '*'). This means it participates in the slot lookup process. The data parent approach has an advantage over the previous example in that if we change the representation of the 'polygon' object then all new 'filled_polygon' instances will get this new representation. We don't need to edit the 'filled_polygon' definition for the modified or additional slots.

In the 'filled_polygon' example we re-used the 'vertices' slot from 'polygon'. We can also define subclasses that implement 'vertices' differently than 'polygon'. For example, a rectangle that stores the four corners and computes the vertices from this. Due to the seperation of state and behaviour this can be modelled easily:

rectangle_traits = (|
  parent* = polygon_traits.
  draw = ...draw rectangle using left, right, top, bottom...
  vertices = ...compute vertices list using left, right, etc...
|)

rectangle = (|
  parent* = rectangle_traits.
  left.
  right.
  top.
  bottom.
|)

Inheritance in Self can be dynamic if the prototype slots are made assignable. This means, at runtime, we can change the value of the slot used during message lookup, resulting in different behaviour. This can be used for objects that can be in different states.

An example is a 'file' object. It can be opened or closed. Some methods in 'file' can only be used when the file is open, and some only when it is closed. This could be managed by conditional checks in each method. Or the parent of the object could be changed to a different traits object depending on the state - this avoids the need for each method to check if the file is in the open or closed state:

open_file_traits = (|
  read = ...
  close = setParent: close_file_traits.
|)

closed_file_traits = (|
  open = setParent: open_file_traits.
|)

file = (|
  parent* = closed_file_traits.
|)

Methods like 'open' are only available on closed files. 'read' can only be called on opened files. This is basically the Strategy pattern made easy using Self's dynamic inheritance.

Io

Whereas Self defines the prototype lookup chain to be that of the prototype slots in an object, Io instead has a slot called 'protos' which is a list of all objects in the prototype chain. Instead of creating slots with a name suffixed with '*' you append to the existing 'protos' list.

The 'protos' list is initially populated when you clone an object with the object that you cloned. This is unlike Self where copying an object does a shallow copy of all the slots of that object. In Io you get a 'differential inheritance' model where your newly created object has no slots, just a 'protos' field that contains the original object that was cloned. The Self 'point' example I used earlier looks like:

Point := Object clone do(
  x := 0
  y := 0
)

Calling 'clone' on this new 'Point' object results in a new object that does not contain it's own 'x' and 'y' values. Instead its 'protos' field points to the 'Point' object which contains the values. When you set the 'x' value on the clone it will then create its own 'x' slot rather than changing the prototypes. In this way clones of big objects where relatively few slots are changed will save some memory:

a := Point clone
b := Point clone
a x = 5
a x println
 => 5
b x println
 => 0
a protos first == Point
 => true
b protos first == Point
 => true

This does have an interesting side effect in that if you clone a clone then you can end up with a longish prototype chain for the method lookup:

a := Point clone
b := a clone
c := b clone
c protos first == b
 => true
c protos first protos first == a
 => true
c protos first protos first protos first == Point
 => true

Inheritance is handled in much the same manner as Self but you need to manipulate the 'protos' slot instead of having multiple prototype slots. The filled polygon example looks like:

PolygonTraits = Object clone do(
  draw := method(vertices foreach(v, v println))
)

Polygon := PolygonTraits clone do(
  vertices := list(1,2,3,4)
)

FilledPolygonTraits := PolygonTraits clone do(
  draw := method(resend; fillPattern println)
)

FilledPolygon := FilledPolygonTraits clone do(
  appendProto(Polygon clone)
  fillPattern := "solid"
)

Polygon clone draw
  => 1
     2
     3
FilledPolygon clone draw
  => 1
     2
     3
     "solid"

'appendProto' appends an object to the prototype chain which is initially 'FilledPolygonTraits' in this examples as that was the initial object we cloned. The dynamic inheritance example can also be done in Io:

OpenFileTraits := Object clone do(
  read := method(n, "Reading #{n} bytes" interpolate println)
  close := method(
    self removeProto(OpenFileTraits)
    self appendProto(ClosedFileTraits)
  )
)

ClosedFileTraits := Object clone do(
  open := method(
    self removeProto(ClosedFileTraits)
    self appendProto(OpenFileTraits)
  )
)

File := Object clone do(
  init := method(self appendProto(ClosedFileTraits))
)

f := File clone
f read(255)
  => Exception: File does not respond to 'read'
f open
f read(255)
  => reading 255 bytes
f open
  => Exception: File does not respond to 'open'
f close

It's a bit more work than in Self to manually manage the prototype chain but does work.

JavaScript

JavaScript is also a prototype based programming language. Unlike Self or Io it only allows one object to be used as the prototype in any given object. This is stored in a hidden 'proto' member and cannot be updated once set on construction (some implementations allow changing it however). Objects are created by using the 'new' keyword on a constructor function that initializes the object. For now I'll leave it as an exercise for the reader to implement the examples above in JavaScript. I'd be interested in the approaches people take.