Bluish Coder

Editing remote files with Acme in Inferno OS

2013-06-11T01:00:00+12:00

Inferno OS ships with a version of the acme text editor. I’ve tried to use acme on and off for a bit and it never stuck. I’ve always been a vim and emacs user. Recently I watched a video by Russ Cox called A Tour of Acme that motivated me to try it again.

The video is short and covers a lot of what makes Acme interesting. Most of it is related to the extensibility of the editor via shell commands dealing with piped input and output alongside the power of sam editing commands.

Some useful acme papers and tutorials are:

A Tour of Acme
Acme: A User Interface for Programmers
Acme man page
A Tutorial for the Sam Command Language
The Text Editor Sam (for more on the command language)

For a recent task I had restored a backup of my Creatures Developer Resource website, recovering from a hard drive failure. I wanted to remove the Google ad code from each page of the website.

To do this from within Inferno and using Acme I first ensured I had the styx service running on the remote web server and had the host file system bound so remote users could connect to it:

; bind '#U*' /mnt/host
; svc/styx

On my client Inferno machine I mounted this remote directory and ran acme to edit the files on it.

% mount tcp!example.com!styx /n/host
% acme

The acme editor starts and from here I can edit files on the remote server by accessing /n/host/mnt/host. The first step was to load all HTML files in the webserver directory into acme. This was done by executing the following command in an acme window that had its location set to /n/host/mnt/host/var/www/example.com (using mouse button 2 to execute):

Edit B <du -an|grep 'html?$'

From the Sam Reference it can be seen that B is used to load files into the editor. In this case I load the output of the command du -an|grep 'html?$'. The command du -an provides a recursive list of all files in the current directory and subdirectories and the grep limits it to those ending in htm or html. Once executed acme has loaded all the files we’re interested in from the remote server.

The Google Ad snippet to be removed looks like:

<!-- Gooogle -->
...
<!-- Gooogle -->

The following sam command will find this snippet and delete it:

,x<!-- Gooogle -->(.|\n)*<!-- Gooogle -->/ d

The , means the starting range is the entire file. The x command executes another comand for each part of the range that matches the regular expression - in this case the entire Google Ad snippet. The command to execute for each match is d which is to delete the region. To have acme run this on all open HTML files I used:

Edit X/html?$/ ,x<!-- Gooogle -->(.|\n)*<!-- Gooogle -->/ d

The X command iterates over all open windows with the name matching the regular expression and runs the following command, which I described above. This results in all the ad snippets from being removed from all the HTML files. To save them back to disk:

Edit X/html?$/ w

And to close the acme windows:

Edit X/html?$/ D

The task is done, closing acme and unmounting the remote file system:

% unmount /n/host

This task was pretty simple and could be done with a number of existing Unix tools but I wanted to give Acme a try with a real world task. It worked well editing remote files, and iterating towards the correct sam commands to use was easy by interactively testing on single files and using the p command to print ranges instead of deleting. Make sure authentication is set up before sharing remote files though!

Remote Shells with Inferno OS

2013-06-05T18:00:00+12:00

Inferno OS provides a mechanism to connect to remote computers and execute commands similar to the way SSH works on Unix systems. The command to do this is cpu. This can be used to execute commands, start a remote shell, and share GUI applications.

To use cpu you’ll need to set up authentication and encryption. This provides the mechanism to authentiate to the remote machine and encrypt the session data.

The rstyx command needs to be run on the remote host to start the services that cpu uses to connect:

; ndb/cs
; ndb/dns
; svc/rstyx

The ndb commands shown there start the services to use the network and DNS. Usually this is started beforehand but I list them for completeness.

With rstyx running and authentication configured the client can connect to the remote machine using cpu:

% cpu example.com
; ...now on server...

This creates a session that is encrypted during authentication but then unencrypted. To use an encrypted session the -C flag is required:

% cpu -C rc4_256 example.com
; ls
; ...server directory listing...

cpu takes an optional argument, that defaults to the shell, to execute:

% cpu -C rc4_256 example.com ls
; ...server directory listing...
%

The namespaces available to the cpu command are mapped to the directory /n/client/ in a cpu session. This means the remote shell or command can use resources on the client. For example a phone running Inferno can cpu to a remote host and that remote host can make calls, send messages, etc using the phones resources. By controlling what namespaces exist before cpu is called the client can restrict what resources are available in the remote session.

Another example of resource sharing is to run GUI programs on the server but have the display appear on the client, similar to how X11 forwarding operates. Doing this requires having the wm window mananger on the client export its interface to a namespace (it’s one of the few applications that doesn’t do this by default on Inferno) and then in the cpu session bind that to a location that GUI programs can see. On the client the export would look like:

; mount {wmexport} /mnt/wm
; cpu -C rc4_256 example.com
; wmimport -w /n/client/mnt/wm acme &

This runs the editor acme on the server but the display is shown on the client. The -w flag passed to wmimport gives the path to an exported directory provided by wmexport. In this case we use the directory exported on the client which cpu has exposed under /n/client/mnt/wm.

Any graphical program can be run this way including GUI shells, games, etc. Unfortunately, like X11 forwarding, it’s fairly slow over low bandwidth connections. For local connections it’s quite usable however. This could be a useful way to share a phone application on a desktop or a desktop application on a phone when it’s connected to the same network.

A combination of resources on the remote server and the client can be used to work around the low bandwidth issue. The acme editor exposes directories and files to control the editor. By running acme on the client, cpu-ing to the remote server so it can see these directories, the programs on the remote end can send data to the client acme window. An example follows:

; acme
; ...create a terminal in acme by 2nd-button clicking `win`...
; ...in the acme terminal window...
; cpu -C rc4_256 cd.pn

Now the remote session running inside acme will expose the needed interfaces to control the client acme program via /n/client/mnt/acme:

; echo Hello >/n/client/mnt/acme/new/body

This command, executed on the server in the remote session, creates a new window in the client acme program with the text “Hello”.

Building Wasp Lisp and MOSREF for Android

2013-05-09T19:00:00+12:00

Wasp Lisp is a small cross platform lisp by Scott Dunlop that I’ve written about before and MOSREF is the secure remote injection framework that is one of the applications written with it.

I’ve been wanting to get Wasp running on Android and Gonk (the low level Android layer of Firefox OS) for debugging and small applications. One of the nice features of Wasp is being able to have a minimal interpreter running on a platform and send byte code from another system to that interpreter which is loaded and run. You can even send the bytecode compiler itself to be loaded and run in the interpreter to get a full system on the target. This is the basis of how MOSREF works where the drone gets sent code to run from the console system.

Standalone Toolkit

Building Wasp for Android requires generating a standalone toolchain from the Android NDK. This results in a vesion of gcc and libraries that can be run from conventional makefiles and configure scripts.

To generate a standalone toolchain to build Wasp I used:

$NDK_ROOT/build/tools/make-standalone-toolchain.sh \
      --platform=android-8 \
      --install-dir=$STK_ROOT

Replace NDK_ROOT with the path to the Android SDK and STK_ROOT with the destination where you want the standalone toolkit installed. The android-8 in the platform makes the standalone toolkit generate applications for FroYo and above (See STABLE-APIS.html.

Adding $STK_ROOT/bin to the PATH will make the compiler available:

export PATH=$PATH:$STK_ROOT/bin

Building libevent for Android

The Wasp virtual machine uses libevent to enable asynchronous operations. Building this for Android using the standalone toolkit is straightforward:

wget https://github.com/downloads/libevent/libevent/libevent-2.0.21-stable.tar.gz
tar xvf libevent-2.0.21-stable.tar.gz
cd libevent-2.0.21-stable
./configure --host=arm-linux-androideabi --prefix=$STK_ROOT
make
make install

This configures libevent to be built using the ARM compilers in the standalone toolkit and to install the resulting libraries in the same install location as the toolkit. This makes it easy for other applications to find and link with the library.

Building the Wasp stub for Android

A Wasp VM stub file is used for generating Wasp applications. The bytecode for an application is appended to the stub file and the result made to be executable. By having stubs for various operating systems and architectures you can create executables specific to them easily. To build the Android stub:

git clone git://github.com/swdunlop/WaspVM.git
cd WaspVM
STRIP=arm-linux-androideabi-strip \
  CC=arm-linux-androideabi-gcc \
  CFLAGS="-I $STK_ROOT/include -L $STK_ROOT/lib" \
  ARCH=arm-linux \
  OS=android \
  make

To build we define the correct version of the build programs to compile for ARM and set the architecture and OS to stop the build system from picking up the host system version of these. The result is a waspvm-android-arm-linux stub in the stubs directory.

Creating the host Wasp programs

To easily create Wasp programs for different platforms we’ll need versions of them for the host we are running on. By doing a ‘clean’ followed by a ‘make’ on the host we’ll clean out the Android build files and rebuild what’s needed for the host. The ‘clean’ process doesn’t remove the stub file we created in the previous step which still allows us to do android Wasp programs.

make clean
make

This produces a waspvm-linux-x86_64 stub in my stubs directory. To generate the Wasp interpreter and other programs:

make repl

Exit out of the repl with (exit) and there will be a wasp, waspc and waspld program for the host system. Now we can make a version of wasp for Android:

cd mod    
../waspc -exe ../awasp -platform android-arm-linux bin/wasp.ms

The waspc command takes a stub file (defined by the platform or stub argument) and appends the bytecode resulting from the compilation of the Wasp lisp code given as an argument. wasp.ms is the source for the intepreter. The output is set as awasp which is our executable for running on Android. We can use a similar command for producing Android versions of the other Wasp programs. MOSREF for example:

../waspc -exe ../amosref -platform android-arm-linux bin/mosref.ms

Running on Android

Running these executables on Android involves pushing them to somewhere writeable on the phone. If you have a rooted Android device (or a Firefox OS device) you can push them pretty much anywhere. On my non-rooted Jellybean Galaxy Note 2 I’m limited to just /data/local/tmp:

adb push awasp /data/local/tmp
adb shell
cd /data/local/tmp
chmod 0755 awasp
./awasp
>> (+ 1 2 3)
:: 6

This gives you a running Wasp interpreter. To develop and do interesting things on the phone you really need the Wasp mod files and the stub files copied over:

adb push mod /data/local/tmp
adb push stubs /data/local/tmp
adb shell
cd /data/local/tmp
./awasp
>> (import "lib/http-client")
:: #t
>> (http-response-body (http-get "http://icanhazip.com/"))
:: "xxx.xxx.xxx.xxx\n"
>> (exit)

MOSREF Android drone

It’s simple now to create a MOSREF drone that runs on Android. Make sure the MOSREF console has a stubs directory containing the Android stub then create the drone as:

console> drone drone1 myphone android-arm-linux
Drone executable created.
Listening for drone on 10000...

Copy the created drone1 onto the phone and run it:

adb push mod/drone1 /data/local/tmp
adb shell
cd /data/local/tmp
chmod 0755 drone1
./drone1

Now on the console you can see the drone on the phone:

console> nodes
NODES: console online address: 192.168.1.101 port: 10000
       myphone online
console> on myphone sh ls
...directory listing...
console on myphone do (print "hello\n")
...prints hello on the phone 'adb shell' session...
console> on myphone load lib/http-file-server.ms
:: spawn-http-file-server
console> on myphone do (offer-http-file 8080 "/test" "text/plain" "Hello world!")
:: [queue 1F5DD00]

The load command shown above loads and compiles the Wasp lisp file on the host and sends the compiled bytecode to the drone on the phone. We then run the lisp code on the phone to start an HTTP server to serve data.

Ranged integer types and bounds checking

2013-05-07T17:00:00+12:00

There’s a discussion going on in the Rust mailing list about ranged integer types and whether they’re useful for safety. Some of the points raised have been about the performance costs of dynamic bounds checks and the overhead of code for static bounds checks. I thought I’d write a post with my experiences with overfow checking in Gecko and with programming languages that can do overflow checking at compile time.

Gecko

There have been places in Gecko (the core Firefox engine) where overflow detection at compile time would be useful. An example is in the WAV audio format reading code. In this C++ code we have the following code:

static const int64_t BLOCK_SIZE = 4096;
int64_t readSize = std::min(BLOCK_SIZE, remaining);
int64_t frames = readSize / mFrameSize;

const size_t bufferSize = static_cast<size_t>(frames * mChannels);
nsAutoArrayPtr<AudioDataValue> sampleBuffer(new AudioDataValue[bufferSize]);

Here frames * mChannels can overflow with the result of the memory buffer allocated following it being of the wrong size and a memory access error occurring later. We can do a run time check that it won’t overflow but this is error prone if someone changes BLOCK_SIZE to be of a size that causes overflow, or the size of AudioDataValue changes to do the same. It’ll take an edge case in a test to find the problem. Better is to ensure that it will never overflow at compile time. The PR_STATIC_ASSERT construct can do this for us:

PR_STATIC_ASSERT(uint64_t(BLOCK_SIZE) < UINT_MAX / sizeof(AudioDataValue) / MAX_CHANNELS);
const size_t bufferSize = static_cast<size_t>(frames * mChannels);
nsAutoArrayPtr<AudioDataValue> sampleBuffer(new AudioDataValue[bufferSize]);

Now if BLOCK_SIZE, AudioDataValue or MAX_CHANNELS changes it will complain if the result overflows at compile time. Because C++ doesn’t have dependent types and constraint solving, we don’t have the ability to statically confirm that frames * mChannels avoids overflow automatically. We have to encode how those values are calculated in the PR_STATIC_ASSERT but it’s better than nothing.

Similar code follows to check readSize:

PR_STATIC_ASSERT(uint64_t(BLOCK_SIZE) < UINT_MAX / sizeof(char));
nsAutoArrayPtr<char> dataBuffer(new char[static_cast<size_t>(readSize)]);

We encode the rule that BLOCK_SIZE must not be greater than UINT_MAX knowing that readSize is capped at BLOCK_SIZE.

Other uses of PR_STATIC_ASSERT are in that same file and various other places in the codebase. For example, Bug 623998. This was caused by an overflow in the implementation of the C++ operator new so we used PR_STATIC_ASSERT to ensure that the argument passed to operator new couldn’t overflow based on analysis of the constraints of the arguments we pass to it.

ATS

The ATS programming language has a dependent type system that allows expressing ranges for integers at the static level. Unfortunately the current implementation uses machine integers in the constraint solver and this prevents easily expressing constraints involving overflow.

The Applied Operating System, an operating system kernel written in ATS, uses a patched ATS that uses libgmp for constraint solving and it declares types such overflow causes a type error. For example, the following code will not typecheck:

fun test () = {
  val a = ubyte1_of 250
  val c = ubyte1_of 6

  val d = a + b
}

The type checker recognizes that a is the exact unsigned byte value of 250 and that c is the exact type value of unsigned byte 6. Adding these together causing a byte greater than 255 which is a type error.

Constraints on types are propogated as expected. For example:

fun add_me {a,b: Ubyte} (a: ubyte a, b: ubyte b): ubyte (a+b) = a + b

fun test () = {
  val x = add_me (ubyte1_of 200, ubyte1_of 10)  
}

Here the arguments to add_me cannot be determined statically (assume test and add_me are compiled seperately so the compiler does not know the exact values at compile time) and the type checker will complain about the a+b since it can’t prove that any unsigned byte added to another unsigned byte is less than the maximum value of an unsigned byte.

A solution is to add to the static system to say that the arguments to add_me add up to less than 255 and will not overflow and make callers check the range before calling:

fun add_me {a,b: Ubyte | a+b <= 255 } (a: ubyte a, b: ubyte b): ubyte (a+b) 
  = a + b

fun test () = {
  val x = add_me (ubyte1_of 200, ubyte1_of 10)  
}

With judicious use of constraints you can then push out the range check from the inner loops to the callers at the outmost region to reduce overhead. Or you can add dynamic checks close to the inner loops to reduce the burden on callers. The good thing is the compiler tells you when you get it wrong.

ATS2

ATS2 is a work in progress with improvements and additions to ATS. You’ll need a recent version of ATS to build and then follow the instructions to get a working ATS2.

One of the improvements in ATS2 is allowing using libgmp to deal with constraints so range checks can be done to detect overflow. Hongwei Xi posted to the mailing list an example of using this to detect integer overflow in bsearch at compile time. This file can be typechecked with:

patsopt -tc -d doc/EXAMPLE/MISC/arith_overflow.dats

The arith_overflow example starts off by declaring an abstract type, intb to represent bounded integers:

abst@ype intb (i: int) // bounded integers

The type is indexed over a sort int. This is a dependent type that represents the value of the bounded integer. Cast functions are provided to allow converting between a normal integer and a bounded integer:

castfn intb2int {i:int} (i: intb i): int (i)
castfn int2intb {i:int | isintb(i)} (i: int i): intb (i)

The int2intb cast function has a contraint on its type index saying that the given integer is a bounded int via the isintb call. This is defined as:

stacst INTMIN : int and INTMAX : int
stadef isintb (i:int): bool = (INTMIN <= i && i <= INTMAX)
praxi lemma_INTMINMAX (): [INTMIN < ~0X7FFF ; INTMAX >= 0x7FFF] void

The way this works is two sorts constants are declared, INTMIN and INTMAX. The definition isintb constrains the index i to be between these two values. The values are not yet defined so all the type system knows for now is that isintb returns true if the given number is between two other numbers.

The axiom (introduced with praxi and is equivalent to a prfun or proof function) lemma_INTMINMAX states that INTMIN is < -0x&FFFF and INTMAX >= 0x7FFF. Once this proof function is called it tells the constraint system what the intb is bounded by.

Basic mathematic operations are provided for a bounded integer and the standard mathematical operators are overloaded with these. For example, adding two bounded integers:

fun add_intb_intb {i,j:int | isintb(i+j)} (i: intb (i), j: intb (j)):<> intb (i+j)
overload + with add_intb_intb

This is similar to the add_me function shown previously using the modified ATS compiler. It adds two bounded ints with the contraint that the result must also be a bounded int. Because the type system uses libgmp for the contraints it is able to ensure that the type system itself doesn’t overflow when doing this check.

With these definitions for bounded ints in place it’s possible to write code that will not typecheck if overflow occurs. The example in arith_overflow implements bsearch. Within the bsearch implementation it has the follow code:

val m = (l + r) / 2

Here l and r are bounded ints. Adding l and r can overflow and this will fail to compile. Changing the code to do effectively the same thing but avoiding adding the two numbers to prevent overflow results in compiling because the type checker can prove no overflow occurs:

val m = l + half (r - l)

Where half is defined to return the number divided by two:

fun half_intb {i:nat} (i: intb (i)):<> intb (ndiv(i,2))
overload half with half_intb

Summary

I think overflow checking at the type level is a very useful feature. It helps find important bugs early as people hardly ever check for overflow. When they do it’s at runtime and the code is often not exercised until an issue actually occurs so may not handle things correctly.

For Mozilla C++ code we can use PR_STATIC_ASSERT (or preferably the recently introduced MOZ_STATIC_ASSERT) to help get some of the benefits of compile time overflow checking.

Social Engineering Attempt Warning

2013-05-01T16:00:00+12:00

This morning when I checked my email I got a nice email from my server hosting provider titled “Root password reset”. That’s not usually a good sign when you haven’t done any password resets recently.

The email was a service request response from the provider asking for confirmation to reset the root account on my server. I responded that this appeared to be a hack attempt, got online to chat with them and got the details of what happened.

An attacker had registered a domain, chrisdouble.co.nz, this morning. They rang my hosting provider to say they were me, and to reset the email address in my account to that domain. They managed to convince them to do this.

Once that was done they were able to get into the customer service portal for the server. No server maintenance could be done there but they got access to my phone number, billing details and the ability to raise service requests.

They raised a service request to reset the root password on the server. The service response was that they couldn’t do this as the server was set up to use SSH keys and passwords were disabled. The attackers response to that was to ask them to physically access the server and re-enable password logins.

The support person emailed back to confirm this request but the email to the attackers domain bounced. They then emailed the previous address, which is my real email, and that was the one I read this morning.

While I was talking to the support person to resolve the issue the attacker was on the phone with this hosting company as well, trying to prove their identity as me. They had a number of personal details and it seemed to be a pretty well thought out social engineering attempt. In the end they realised we were fishing for information from them and they hung up.

It is amazing how close they got to being able to gain access to the server. If I hadn’t been awake to read that email and respond quickly support would most likely have given them access. I’ve now arranged a protocol with them to prevent this from happening again.

If anyone I know receives an email or contact from chrisdouble.co.nz it is not me. If you are unsure you can ask for me to sign a message with GPG. My GPG fingerprint can be obtained by contacting me directly. Because the attacker gained access to the service portal they have a reasonable amount of identifying information (address, phone number, date of birth, etc). Don’t take any of these as information to confirm an unusal contact as being me.

Backing up Gmail messages with offlineimap

2013-04-30T18:00:00+12:00

A while back I realised I had a ton of email archived on Gmail which I would be sad to lose if I lost access to my Google account or couldn’t access the internet for some reason. I also wanted a backup in case I decided to migrate away from Gmail to use another service.

The approach I took was to use offlineimap to download the contents of my mail using Gmail’s IMAP support. I set it up to download a few days of email at a time so I wouldn’t encounter any bandwidth limiting from Google or risk getting my account temporarily suspended for aggressive use.

I chose to use ‘Maildir’ format for the downloaded mail so I could use notmuch locally to read and search.

The matter of dealing with Gmail folders is a bit tricky. These are exposed as IMAP folders and if you’re not careful you can end up downloading emails multiple times for each folder. I didn’t really want the folder structure. I just wanted all emails and I’d use the tagging mechanism of notmuch to add tags after the fact.

The secret to ignoring folders is to create a folderfilter entry in the .offlineimaprc file. This is a lambda function that given a folder name should return true if it’s a folder you want to be downloaded by offlineimap. I use:

folderfilter = lambda foldername: foldername in ['[Gmail]/All Mail', '[Gmail]/Sent Mail']

This downloads “All Mail” and “Sent Mail”. This way I get everything in my Gmail without the folder structure.

I chose to add a nametrans entry so that the downloaded folders in the Maildir have more relevant names. nametrans is a lambda function that, given a folder name, returns the name that should be used locally for that folder. Here I translate “All Mail” to “all” and “Sent Mail” to “sent”:

nametrans = lambda foldername: 
              re.sub('^\[Gmail\]/All Mail$', 'all', 
              re.sub('^\[Gmail\]/Sent Mail$', 'sent',foldername))

To connect to Gmail the following entries are used in the remote repository section:

type = Gmail
remotehost = imap.gmail.com
realdelete=no
maxconnections=1
ssl = yes
cert_fingerprint = 6d1b5b5ee0180ab493b71d3b94534b5ab937d042
remoteport = 993
remoteuser = ...
remotepass = ...

My local repository section is:

type = Maildir
localfolders = ~/.Mail

To prevent having to run offlineimap for a long time on the initial sync I did it over a series of days. I used the maxage setting in the Account section. When set mail older than this number of days is not synced. So I’d set it for 100 days, do a sync. Then I’d increase it by a 100 the next day and do another sync. Over a series of days/weeks I have all my email. Once completely synced I removed the entry from the .offlineimaprc file. I’m not sure what the best value is and maybe it doesn’t matter but this worked for me.

My .offlineimaprc then looks like:

[general]
accounts = gmail
ui = TTY.TTYUI

[Account gmail]
localrepository = gmailLocal
remoterepository = gmailRemote
maxage = 1000

[Repository gmailLocal]
type = Maildir
localfolders = ~/.Mail

[Repository gmailRemote]
type = Gmail
remotehost = imap.gmail.com
realdelete=no
maxconnections=1
ssl = yes
cert_fingerprint = 6d1b5b5ee0180ab493b71d3b94534b5ab937d042
remoteport = 993
remoteuser = ...
remotepass = ...
nametrans = ...show above...
folderfilter = ...show above...

I used notmuch to process and search the Maildir locally. By setting synchronize_flags=true in my .notmuch-config file I could read the offline email in notmuch, incrementally sync with offlineimap, and the ‘read’, ‘replied’, etc flags are synchronized between them.

To tag with notmuch I run a script after syncing with .offlineimap that tags based on certain criteria. Something like:

offlineimap
notmuch new
notmuch tag +sent -- folder:sent and not tag:sent 
notmuch tag +bugzilla -inbox -- tag:inbox and from:bugzilla-daemon@mozilla.org
notmuch tag +ats -inbox -- tag:inbox and to:ats-lang-users
...etc...

Running Firefox OS on Nexus S

2013-04-22T20:00:00+12:00

Last year I posted about how to build and run B2G on the Nexus S. Much has changed since then and this post attempts to provide up to date instructions on building Firefox OS and installing the latest Gaia version.

I wrote ‘attempts’ because the support for the Nexus S bitrots frequently due to (I assume) most development effort going towards developer devices and devices that intend to be shipped. These instructions worked for me on my Nexus S and gives me a phone that can use Wifi, make calls, use 3G data, take photo’s, record video and install applications.

Getting the source

The main source repository is on github, https://github.com/mozilla-b2g/b2g. The first step is to clone this repository. I clone this into a directory called nexus-s:

$ git clone git://github.com/mozilla-b2g/B2G nexus-s
$ cd nexus-s

Create a file in this directory called .userconfig containing the following:

export CC=gcc-4.6
export CXX=g++-4.6
export HIDPI=1

The first two entries are required if you are building on Ubuntu 12.10 to use gcc version 4.6, which you must install. B2G does not build using gcc version 4.7.

The last entry, HIDPI, results in applications using the correct resources for the screen size on the Nexus S. Without this many things will be scaled incorrectly.

Configuring

Once the source is obtained you need to pull down the sub-repositories required for the Nexus S. This step downloads multiple gigabytes of data and can take a long time:

nexus-s$ BRANCH=v1-train ./config.sh nexus-s

Note the use of the BRANCH environment variable. This configures our build to use the relatively stable v1-train branch. This branch gets regular updates but tends to be better tested than master which is often broken for not-quite-supported devices like the Nexus S.

Although we are using the v1-train we are going to use the master branch of Gaia so we get the latest and greatest user interface. We’ll change to this later.

Gecko changes

The branch of Gecko that v1-train uses has some bugs on the Nexus S. To fix this we need to apply some patches. The first is bug 832653, video playback broken on Nexus S. The patches relevant to Gecko from this bug that need to be applied are:

Fix graphic buffer crash

The following set of commands downloads these patches and applies them in a branch of the Gecko tree that the config.sh step previously obtained:

nexus-s       $ cd gecko
nexus-s/gecko $ git checkout -b nexus-s m/v1-train
nexus-s/gecko $ curl -k -L https://bug832653.bugzilla.mozilla.org/attachment.cgi?id=734988 >p1.patch
nexus-s/gecko $ patch -p1 <p1.patch
nexus-s/gecko $ git commit -a -m "Nexus S patches"
nexus-s/gecko $ cd ..

Gonk changes

Part of bug 832653 makes a change to the low level Gonk layer of B2G. This is in the libstagefright video decoding libraries. The change is to prevent output buffer starvation which results in video decoding stopping after a few frames. The required patch is:

Fix output buffer starvation

To apply this patch:

nexus-s                 $ cd frameworks/base
nexus-s/frameworks/base $ git checkout -b nexus-s 
nexus-s/frameworks/base $ curl -k -L https://bugzilla.mozilla.org/attachment.cgi?id=735002 >p1.patch
nexus-s/frameworks/base $ patch -p1 <p1.patch
nexus-s/frameworks/base $ git commit -a -m "Nexus S patches"
nexus-s/frameworks/base $ cd ../..

Gaia changes

There are Gaia changes required as part of bug 869289 as well. This change prevents the error where videos are not listed if the thumbnail cannot be generated:

List videos even if thumbnail generation fails

We also want to be on the Gaia master branch to get the latest and greatest code. The following comands switch to this branch and apply our patch:

nexus-s      $ cd gaia
nexus-s/gaia $ git checkout -b nexus-s mozillaorg/master
nexus-s/gaia $ curl -k -L https://bug869289.bugzilla.mozilla.org/attachment.cgi?id=746196 >p1.patch
nexus-s/gaia $ patch -p1 <p1.patch
nexus-s/gaia $ git commit -a -m "Nexus S video app thumbnail fix"

There is a bug related to video recording on the Nexus S. To fix this we need a patch from bug 832638:

Set video recording profile in camera app

To apply this:

nexus-s/gaia $ curl -k -L https://bugzilla.mozilla.org/attachment.cgi?id=704241 >p2.patch
nexus-s/gaia $ patch -p1 <p2.patch
nexus-s/gaia $ git commit -a -m "Nexus S video recording fix"

Building

Now that all the required changes have been made we can build:

nexus-s $ ./build.sh

Installing

Once the build is complete you should be able to flash the Nexus S with the results. Note that this completely wipes everything on your phone:

nexus-s $ ./flash.sh

The phone should reboot after this step with Firefox OS installed and prompting for information from the “First Run” app.

New Zealand changes

Some New Zealand sites don’t detect Firefox OS and serve their mobile versions. This can be changed by adding overrides to gaia/build/ua-override-prefs.js. I add the entries in b2gnzuaoverride.txt to this file to get stuff.co.nz, mega.co.nz and trademe.co.nz serving the correct mobile site. Once added to this file, reinstall Gaia on the phone.

nexus-s $ ./flash.sh gaia

Updating

To update the source to latest versions we need to ‘rebase’ our changes on top of the most recent repository changes. The following steps will do this:

nexus-s $ git pull
nexus-s $ ./config.sh nexus-s
nexus-s $ cd gecko
nexus-s/gecko $ git rebase m/v1-train
nexus-s/gecko $ cd ../gaia
nexus-s/gaia  $ git rebase mozillaorg/master
nexus-s/gaia  $ cd ..
nexus-s $ ./build.sh

You may need to do some fixing up of merge conflicts if changes have occurred to the areas we’ve patched.

To flash the phone, overwriting everything on it, including all user data:

nexus-s $ ./flash.sh

To flash the phone, keeping (most of…) your existing user data:

nexus-s $ ./flash.sh system
nexus-s $ ./flash.sh gecko
nexus-s $ ./flash.sh gaia

Note that flashing gaia might remove some of your installed application icons. You’ll need to reinstall those apps to fix that.

Parts of this site temporarily down

2013-04-03T14:00:00+13:00

Unfortunately the server running this site had a hard drive issue which resulted in a number of corrupt files. You’ll probably notice that some links to here result in a ‘404’ error due to them being not found. I’m slowly recovering the lost data and hope to have the files restored as soon as I can.

An Introduction to using pointers in ATS

2013-01-25T18:00:00+13:00

The ATS programming language was designed to enable safe systems programming. This requires dealing with pointers safely. In this post I intend to go through the basics of pointer handling in ATS and how this safety is achieved.

Raw Pointers

ATS has a basic pointer type called ptr. This is a non-dependent type and is the equivalent to a void* in the C programming language. It also has a dependently typed pointer type that is indexed over an addr sort (sorts are the types of type indexes) that represents the address of the pointer. This allows the proof and type system to reason about pointer addresses. Pointer arithmetic can be done on values of these types but they can’t be safely dereferenced. The following example shows some basic usage:

extern fun malloc (s: size_t): ptr = "mac#malloc"
extern fun free (p: ptr):void = "mac#free"

implement main() = let
  val a = malloc(42);
in
  print (a); print_newline();
  print (a + 1); print_newline ();
  free(a)
end

This program offers no compile time safety for dealing with the pointers. You can leave the free call out and it will compile, run and leak memory. You can use unsafe functions to dereference the pointer to retrieve and store data at that location:

staload "prelude/SATS/unsafe.sats"
staload _ = "prelude/DATS/unsafe.dats"

extern fun malloc (s: size_t): [l:addr | l > null] ptr l = "mac#malloc"
extern fun free {l:addr | l > null} (p: ptr l):void = "mac#free"

implement main() = let
  val a = malloc(sizeof<int> * 2)
in
  ptrset<int> (a, 10);
  ptrset<int> (a + sizeof<int>, 20);
  print (ptrget<int> (a)); print_newline();
  print (ptrget<int> (a + sizeof<int>)); print_newline ();
  free(a)
end

ptrset and ptrget are template functions in the unsafe module that allow storing and retrieving values using raw pointers. No safety guarantees are available and it is assumed that the pointers are valid and the pointer arithmetic is done correctly. One slight change to this example is it uses the dependently typed variant of ptr with the constraint that the index is greater than null. ptrset and ptrget are defined to take a pointer that is non null so malloc and free in this example had to be modified to use these dependent types to prove the pointer is non-null.

At Views

ATS has a concept called ‘views’ which is part of the proof system. Views are linear resources that exist at type checking time and are erased during compilation. They have no runtime overhead and can be used to prove aspects of a program. For pointer handling there exists a type of view commonly called an ‘At View’ or ‘@ view’ because of their syntax. An at-view that represents that a type T is stored at a memory location L is written as T @ L.

An at-view is often returned when memory containing a value of a particular type is allocated. The functions to get and set this memory then take the at-view as a proof argument along with the raw pointer. This tells the type system that you have a proof that the raw pointer points to that particular type. An at-view is linear so must be destroyed. By requiring the deallocating function to consume the at-view as a proof argument it can prevent further access to that memory address. It no longer exists to be able to be passed to the get and set functions.

Using this approach will result in compile time errors if the pointer is used incorrectly. The following demonstrates the approach:

staload _ = "prelude/DATS/pointer.dats"

extern fun malloc (s: sizeof_t int): [l:agz] (int @ l | ptr l) = "mac#malloc"
extern fun free {l:agz} (pf: int @ l | p: ptr l): void = "mac#free"

implement main() = let
  val (pf | a) = malloc (sizeof<int>)          // 1
in
  ptr_set_t<int> (pf | a, 10);                 // 2
  print (ptr_get_t (pf | a)); print_newline(); // 3
  free (pf | a)                                // 4
end

In this example, malloc is changed to return a proof as well as the raw pointer. In ATS syntax proof arguments are to the left of the pipe, |, symbol while non-proof arguments are to the right. The proof here is an at-view indicating that the type int is stored at memory address l - the same memory address as the raw pointer returned. The free function is changed to take the at-view as a proof argument. It is consumed by the free call (the absence of a ! symbol as part of the argument type means it is consumed). The addresses in these definitions have been changed to be agz which is short for ‘address greater than zero’ and is equivalent to the [l:addr | l > null] form shown previously.

In line 1 malloc is called passing it the size of an integer. The at-view is stored in the pf variable and the raw pointer in a. In line 2 the standard prelude function ptr_set_t is used to set the value at the pointer address to 10. This requires the passing of the at-view to prove that we have an integer stored at that memory address. pf is of type int @ l where l is the same address as the index type of a. Line 3 obtains the value at that address using the prelude function ptr_get_t. Again we need the proof to get it. Line 4 calls free and consumes the proof.

This example is safer than the previous examples:

We can’t forget to call free as we would be left with the linear at-view alive and it would fail to compile.
We can’t pass any random pointer to ptr_get_t or ptr_set_t. These functions require an at-view that has the same address in it’s type index as the raw pointers type index.
We can’t malloc memory less than the size of the data we are storing there. The definition of malloc declares that its initial argument is of type sizeof_t int which is an integer that is the actual value returned by sizeof int. Any other value passed would be a compile error.

Implicit at-view usage

The use of at-views, ptr_get_t and ptr_set_t tends to clutter code somewhat. ATS provides some syntax to dereference pointers that automatically look for valid at-view proofs in scope. The changes to the previous example to use this are:

implement main() = let
  val (pf | a) = malloc {int} (sizeof<int>)
in
  !a := 10;
  print (!a); print_newline();
  free (pf | a)
end

The ! operator is like the unary * operator in the C programming language. It dereferences a pointer. It requires an at-view to be in scope that has the same address as its indexed type as the pointer’s indexed type. In this case the variable a uses address l which matches the int @ l type of the pf variable. := is assignment. As pf is in scope for both the assignment and the print call the code typechecks. If pf wasn’t in scope the code would fail to compile.

Getting pointers to variables

The unary & operator can be used like in the C programming language to get a pointer to a variable. Given a variable containing an integer a, then &a is of type ptr. Note that this is a raw pointer which you can’t derefence without an at-view in scope of type int @ l. To obtain an at-view from an existing variable on the stack you use the view@ function. The result of view@ on a variable of type T is an at-view of type T @ l where l is the memory address of the variable.

The following code has an add_one function that increments an integer by one given a pointer to it. The body of main has an integer, a, on the stack and view@ is used to obtain the at-view allowing dereferencing a pointer to it:

fn add_one {l:agz} (pf: !int @ l | p: ptr l): void = 
  ! p := !p + 1

implement main() = let
  var a: int = 0
in
  print (a); print_newline();
  add_one (view@ a | &a);
  print (a); print_newline();
end

Pairs

One of the earlier raw pointer examples in this post allocated a memory area containing two integers and used pointer arithmetic to access the second integer offset from the original pointer. So far the at-view usage examples here have only used single values stored at a memory address. To access consecutive memory locations we can use arrays. ATS doesn’t have arrays built into the language, rather they are implemented using views, at-views and pointers.

To demonstrate how this is implemented we’ll construct a simple example of rolling our own types to safely access memory containing two integers. Then I’ll show how to use the standard prelude array handling for the same problem and hopefully that will give an idea of how things work under the covers.

If we have a memory area containing two integers that we need to access individually we will need at-views for both memory locations containing the integers. We could write a function that takes both at-views individually:

extern fun malloc (n: size_t (sizeof(int)*2)):
            [l:agz] (int @ l, int @ (l+sizeof(int)) | ptr l) = "mac#malloc"
extern fun free {l:agz} (pf1: int @ l, pf2: int @ (l+sizeof(int)) | p: ptr l):void = "mac#free"

fn set {l:agz} (pf1: !int @ l, pf2: !int @ (l+sizeof(int)) | p: ptr l): void = begin
  !p := 10;
  !(p+sizeof<int>) := 20
end

implement main() = let
  val (pf1, pf2 | a) = malloc(sizeof<int> * 2)
in
  set (pf1, pf2 | a);
  print (!a); print_newline();
  print (!(a+sizeof<int>)); print_newline ();
  free(pf1, pf2 | a)
end

In this example, malloc is defined to allocate a memory area containing two integers. It restricts the number of bytes it receives as an argument to match exactly that of the size of two integers. It returns a pointer and two at-views as proofs. The first being an int @ l and the second being an int @ (l+sizeof(int)). This provides a caller of malloc with at-views allowing access to the integers that can be stored in the returned memory.

The function set requires these two proofs so it can set the value of both integers in the memory area. The first is done via a simple dereference of the pointer, p. The second adds sizeof<int> to that pointer and dereferences it to set the value. This succeeds because there is a proof in scope (the pf2 at-view) that is for the correct type, int and the memory address of the value being set. A mistake in the pointer arithmetic would result in a compile error.

Passing multiple proofs around like this is a chore. To make it easier we can wrap them in a dataview. A dataview is like a datatype but creates a linear datastructure used in proofs. This is the same example with the at-views wrapped in a dataview:

dataview pair_v (l:addr) = PAIR (l) of (int @ l, int @ (l + sizeof(int)))

extern fun malloc (n: size_t (sizeof(int)*2)): [l:agz] (pair_v l | ptr l) = "mac#malloc"
extern fun free {l:agz} (pf: pair_v l | p: ptr l):void = "mac#free"

fn set {l:agz} (pf: !pair_v l | p: ptr l): void = let
  prval PAIR (pf1, pf2) = pf
in
  !p := 10;
  !(p+sizeof<int>) := 20;
  pf := PAIR (pf1, pf2)
end

fn show {l:agz} (pf: !pair_v l | p: ptr l): void = let
  prval PAIR (pf1, pf2) = pf
in
  print (!p); print_newline();
  print (!(p+sizeof<int>)); print_newline ();
  pf := PAIR (pf1, pf2)
end

implement main() = let
  val (pf | a) = malloc(sizeof<int> * 2)
in
  set (pf | a);
  show (pf | a);
  free(pf | a)
end

The pair_v dataview has a single constructor, PAIR, that takes two at-views as arguments, one for each integer held in the pair. malloc now returns a pair_v and free consumes it. For set to be able to dereference the pointer it needs to have the at-views in scope, not bundled in the pair_v instance. To get them in scope it uses pattern matching on the PAIR constructor to obtain the two proofs and have them in scope. As a dataview is a linear type this will consume the pair_v. This is why at the end of the scope the PAIR constructor is used to create the pair_v again and reassign it to the pf proof variable.

This proof unpacking and repacking is a common idiom for dealing with at-views and other proofs that need to be in scope. These are type level operations and are erased at compile time so contain no run time overhead.

The show function does the same to access the proofs to enable it to print the values at the memory address. It’s also possible to use proof functions to further encapsulate access to the pointers.

A nice feature of the use of at-views is you can have an array of memory containing lots of values but when you pass pointers to functions requiring only portions of this array you can give it a specific at-view or proof for only the area it needs to manage. The function cannot access any other part of the array since it does not have a proof for it. For example:

dataview pair_v (l:addr) = PAIR (l) of (int @ l, int @ (l + sizeof(int)))

extern fun malloc (n: size_t (sizeof(int)*2)): [l:agz] (pair_v l | ptr l) = "mac#malloc"
extern fun free {l:agz} (pf: pair_v l | p: ptr l):void = "mac#free"

fn set1 {l:agz} (pf: !int @ l | p: ptr l): void = 
  !p := 10;

fn show1 {l:agz} (pf: !int @ l | p: ptr l): void = 
  (print (!p); print_newline())

implement main() = let
  val (pf | a) = malloc(sizeof<int> * 2)
  prval PAIR (pf1, pf2) = pf
in
  set1 (pf1 | a);
  show1 (pf1 | a);
  pf := PAIR (pf1, pf2);
  free(pf | a)
end

Although set1 and show1 here get a pointer to the memory address containing two integers they can only access the single integer at the pointer it is given. This is because the at-view it takes only is for int @ l. To access the integer following this it would need to somehow obtain the at-view for it and it cannot.

Arrays

The ATS prelude has an array_v view for dealing with arrays that are a memory area containing consecutive elements of the same type. The dataview definition looks like (it is actually not defined like this, but operates the same as if it was):

dataview array_v ( a:viewt@ype+, int, addr) =
  | {n:int | n >= 0} {l:addr}
    array_v_cons (a, n+1, l) of (a @ l, array_v (a, n, l+sizeof a))
  | {l:addr} array_v_nil (a, 0, l)

An array_v has three type indexes. They are the type of the item in the array, the number of items in the array, and the address of the first element. It has two constructors. array_v_nil is an empty array. array_v_cons has two arguments, the first being the at-view for the first item and the second being an array_v of the rest of the array. This is the previous pair example using array_v instead:

staload "prelude/SATS/array.sats"

extern fun malloc (n: size_t (sizeof(int)*2)): [l:agz] (array_v (int, 2, l) | ptr l) = "mac#malloc"
extern fun free {l:agz} (pf: array_v (int, 2, l) | p: ptr l):void = "mac#free"

fn set1 {l:agz} (pf: !int @ l | p: ptr l): void = 
  !p := 10;

fn show1 {l:agz} (pf: !int @ l | p: ptr l): void = 
  (print (!p); print_newline())

implement main() = let
  val (pf | a) = malloc(sizeof<int> * 2)
  prval (pf1, pf2) = array_v_uncons (pf)
in
  set1 (pf1 | a);
  show1 (pf1 | a);
  pf := array_v_cons (pf1, pf2);
  free(pf | a)
end

There are various other operations for dealing with packing and unpacking the at-views from the array_v view in prelude/SATS/array.sats. Due to the list like nature of the array_v definition it’s easy to write recursive routines that walk through the array providing only access to the single item currently being operated on.

Initialized vs uninitialized memory

When using pointers you often have to deal with a pointer to memory that has not yet been initialized. This is common when first allocating the memory. ATS differentiates unitialized memory from initialized by appending a ? to the type. For example, an int? is a integer that has not been initialized. A int? @ l is an at-view for an uninitialized integer at memory address l. When the variable is initialized the type changes to drop the ?.

This is useful for defining functions that must have valid data. The following will fail to compile because the show1 function requires the memory to first have been set:

extern fun malloc (n: sizeof_t int): [l:agz] (int? @ l | ptr l) = "mac#malloc"
extern fun free {l:agz} (pf: int? @ l | p: ptr l):void = "mac#free"

fn set1 {l:agz} (pf: !int? @ l >> int @ l | p: ptr l): void = 
  !p := 10;

fn show1 {l:agz} (pf: !int @ l | p: ptr l): void = 
  (print (!p); print_newline())

implement main() = let
  val (pf | a) = malloc(sizeof<int>)
in
//  set1 (pf | a);
  show1 (pf | a);
  free(pf | a)
end

Uncommenting the set1 call enables it to compile. The set1 definition declares that it requires an at-view for an uninitialized int and after calling the at-view becomes one for an initialized int. The syntax T1 >> T2 means that on calling a T1 is required but after calling it becomes a T2.

Conclusion

ATS provides access to pointers at the level of the C programming language but provides a type system, combined with proofs, that enable using this memory safely. This comes with no runtime overhead. There are many other ways of structuring memory access using ATS, beyond the simple examples here. ATS allows memory allocation on the stack, which is reclaimed when the stack frame exits, and prevents access to the memory after it is reclaimed, optional garbage collection, and other features which I haven’t gone through.

Some pointers to futher reading on this subject, and which helped with the writing of this post, are:

Inferno OS IRC client with persistent connections

2012-12-19T20:00:00+13:00

I’ve written before about how Inferno makes it easy to share resources across devices and networks. In this post I show how to use IRC from within Inferno and then how to run the IRC connection handling on a server and the GUI on a client machine. The server can stay connected giving a persistent IRC connection similar to how IRC proxies and bouncers are used.

The IRC client I’m using for Inferno is written by mjl. A large number of useful Inferno and Plan 9 utilities with source is available at mjl’s bitbucket repository. The one that provides IRC capability is ircfs. This is written in a manner that exposes IRC connections as a file system using the 9p2000 protocol. The GUI communicates with IRC via the file system.

Building

Building from within Inferno requires a couple of environment variables to be set. ROOT is the path to the root of the Inferno system and SYSHOST should be set to ‘Inferno’:

% cd ircfs
% ROOT=/
% SYSHOST=Inferno
% mk install
cd module ; mk install
mk: 'install' is up to date
cd appl ; mk install
cd cmd ; mk install
rm -f /dis/testirc.dis && cp testirc.dis /dis/testirc.dis
rm -f /dis/ircfs.dis && cp ircfs.dis /dis/ircfs.dis
cd wm ; mk install
mk: 'install' is up to date
cd lib ; mk install
mk: 'install' is up to date
cd man ; mk install
mk: 'install' is up to date

This installs ircfs into the standard Inferno application and command locations. It may not be desirable to clutter the existing system locations with additional programs like this. If you prefer you can create local directories to hold the installed programs and bind this over the system directories using a union file system. ircfs installs a useful man page which is worth reading to find out all the options.

Running ircfs

When ircfs is executed it creates a directory structure that contains the files that allow clients to send and receive data to the service. To run it we first create a directory underneath /mnt/irc for the server we want to connect to, then run ircfs passing this directory name:

% mkdir -p /mnt/irc/freenode
% mount {ircfs freenode} /mnt/irc/freenode
% ls /mnt/irc/freenode
/mnt/irc/freenode/0
/mnt/irc/freenode/ctl
/mnt/irc/freenode/event
/mnt/irc/freenode/nick
/mnt/irc/freenode/pong
/mnt/irc/freenode/raw

The example above shows the files that exist after running and mounting the ircfs service. The mount command uses the variant that runs and mounts a 9p service.

The running ircfs hasn’t connected to any IRC server yet. Connections and controlling of the IRC server is managed via writing or reading from the files under /mnt/irc/freenode. The protocol is explained in the ircfs man page. For example, writing to ctl allows us to connect to a server. Reading from nick gives the current nickname used by the connected user.

Ensure that ndb/cs is running to allow network connections from Inferno before continuing.

Running the IRC GUI

The GUI for ircfs is run with the command wm/irc, optionally passing the path to the ircfs directory structure as an argument:

% wm/irc /mnt/irc/freenode

On the left of the wm/irc screen is a list of the servers and channels. Currently there is status and (freenode). The (freenode) window is for sending commands to the irc server once we’re connected. To connect to irc.freenode.net enter the following in the (freenode) area:

/connect irc.freenode.net mynick

The status messages from the server should now appear. The commands that ircfs understands are documented in the /ctl section of the ircfs man page. Standard IRC client commands like /join, etc work and the channels will appear on the list in the left of the client window. The ircfs server remains running and connected if the GUI is closed. Running the GUI again with the path to the ircfs directory will display the existing channels connected and you can continue where you left off before it was closed.

GUI commands

The GUI knows some special commands that can be passed to it using /win. These are explained in the main page for wm/irc.

add can be used to add new ircfs directories to a running GUI. If you start wm/irc without passing it a directory for example you can type the following in the status window to connect to one:

/win add /mnt/irc/freenode

del removes the current ircfs directory so the GUI no longer displays informatation about it. You can use add to get the GUI to redisplay it. Note that these two commands don’t affect the actual IRC connection. That remains managed by ircfs. It only affects what is displayed in the GUI:

/win del

The windows command gives a list of all the windows for the current ircfs connection and their control number. This number can be used to close and open channel windows in the GUI:

/win windows
open:
  (freenode)   (0)
  #inferno     (1)
not open:

/win close
/win windows
open:
  (freenode)   (0)
not open:
  #inferno     (1)

/win open 1
/win windows
open:
  (freenode)   (0)
  #inferno     (1)
not open:

away can be used to mark the user as away or back on all servers. exit will exit the GUI but leave ircfs connections intact.

Persistent connections

I prefer to have a persistent IRC connection on a server and connect to it from a client so I can catch up with channel logs and activity while I’ve been away. This allows other users to leave me messages when I’m disconnected. It also allows connecting using multiple devices and keeping my conversation active. This approach usually requires running IRSSI in a screen session on a server, or running an IRC proxy or bouncer.

A similar approach can be done using ircfs by running the ircfs server on a remote machine and on the client mounting the ircfs filesystem. The connection can be encrypted and authenticated.

The following is a simple setup. Install Inferno on the remote server and build and install ircfs as described previously. Run ircfs to run the server. In this example I also add a command line switch to log the IRC connection so I can have channel logs:

; mkdir -p /mnt/irc/freenode
; mount {ircfs -t -l /usr/myuser/irclogs freenode} /mnt/irc/freenode
; styxlisten net!*!16667 {export /mnt/irc}

On the client we mount this exported filesystem using encryption (replace example.com with the hostname of the server):

% mount -C sha1/rc4_256 net!example.com!16667 /mnt/irc
% ls /mnt/irc/freenode
/mnt/irc/freenode/0
/mnt/irc/freenode/ctl
/mnt/irc/freenode/event
/mnt/irc/freenode/nick
/mnt/irc/freenode/pong
/mnt/irc/freenode/raw

The directory and files shown are those from the remote server. Running wm/irc will now use the servers ircfs files:

% wm/irc /mnt/irc/freenode

You can now connect to the irc.freenode.net server, join channels then close the IRC window and shut down the client. Next time you restart the client, mount the remote filesystem again and continue where you left off.

Conclusion

The man pages for ircfs and wm/irc have more detail on the commands and keys that can be used. There are other things that can be done with ircfs other than GUI IRC programs. It’s possible to write IRC bots and programs by utilising the control files and directories of an ircfs session.

The ircfs source is written in Limbo and is quite easy to follow. It’s a good example of mapping an existing protocol onto Inferno’s “everything is a file” metaphor and making use of the tools to share resources. One use of this is having Inferno on other devices, like a phone, so you can connect to IRC connections running on a local machine if you temporarily need to be mobile.