Programming Languages, Martials Arts and Computers. The Weblog of Chris Double.
2013-04-03
Unfortunately the server running this site had a hard drive issue which resulted in a number of corrupt files. You'll probably notice that some links to here result in a '404' error due to them being not found. I'm slowly recovering the lost data and hope to have the files restored as soon as I can.
2013-01-25
The ATS programming language was designed to enable safe systems programming. This requires dealing with pointers safely. In this post I intend to go through the basics of pointer handling in ATS and how this safety is achieved.
ATS has a basic pointer type called ptr. This is a non-dependent type and is the equivalent to a void* in the C programming language. It also has a dependently typed pointer type that is indexed over an addr sort (sorts are the types of type indexes) that represents the address of the pointer. This allows the proof and type system to reason about pointer addresses. Pointer arithmetic can be done on values of these types but they can't be safely dereferenced. The following example shows some basic usage:
extern fun malloc (s: size_t): ptr = "mac#malloc"
extern fun free (p: ptr):void = "mac#free"
implement main() = let
val a = malloc(42);
in
print (a); print_newline();
print (a + 1); print_newline ();
free(a)
end
This program offers no compile time safety for dealing with the pointers. You can leave the free call out and it will compile, run and leak memory. You can use unsafe functions to dereference the pointer to retrieve and store data at that location:
staload "prelude/SATS/unsafe.sats"
staload _ = "prelude/DATS/unsafe.dats"
extern fun malloc (s: size_t): [l:addr | l > null] ptr l = "mac#malloc"
extern fun free {l:addr | l > null} (p: ptr l):void = "mac#free"
implement main() = let
val a = malloc(sizeof<int> * 2)
in
ptrset<int> (a, 10);
ptrset<int> (a + sizeof<int>, 20);
print (ptrget<int> (a)); print_newline();
print (ptrget<int> (a + sizeof<int>)); print_newline ();
free(a)
end
ptrset and ptrget are template functions in the unsafe module that allow storing and retrieving values using raw pointers. No safety guarantees are available and it is assumed that the pointers are valid and the pointer arithmetic is done correctly. One slight change to this example is it uses the dependently typed variant of ptr with the constraint that the index is greater than null. ptrset and ptrget are defined to take a pointer that is non null so malloc and free in this example had to be modified to use these dependent types to prove the pointer is non-null.
ATS has a concept called 'views' which is part of the proof system. Views are linear resources that exist at type checking time and are erased during compilation. They have no runtime overhead and can be used to prove aspects of a program. For pointer handling there exists a type of view commonly called an 'At View' or '@ view' because of their syntax. An at-view that represents that a type T is stored at a memory location L is written as T @ L.
An at-view is often returned when memory containing a value of a particular type is allocated. The functions to get and set this memory then take the at-view as a proof argument along with the raw pointer. This tells the type system that you have a proof that the raw pointer points to that particular type. An at-view is linear so must be destroyed. By requiring the deallocating function to consume the at-view as a proof argument it can prevent further access to that memory address. It no longer exists to be able to be passed to the get and set functions.
Using this approach will result in compile time errors if the pointer is used incorrectly. The following demonstrates the approach:
staload _ = "prelude/DATS/pointer.dats"
extern fun malloc (s: sizeof_t int): [l:agz] (int @ l | ptr l) = "mac#malloc"
extern fun free {l:agz} (pf: int @ l | p: ptr l): void = "mac#free"
implement main() = let
val (pf | a) = malloc (sizeof<int>) // 1
in
ptr_set_t<int> (pf | a, 10); // 2
print (ptr_get_t (pf | a)); print_newline(); // 3
free (pf | a) // 4
end
In this example, malloc is changed to return a proof as well as the raw pointer. In ATS syntax proof arguments are to the left of the pipe, |, symbol while non-proof arguments are to the right. The proof here is an at-view indicating that the type int is stored at memory address l - the same memory address as the raw pointer returned. The free function is changed to take the at-view as a proof argument. It is consumed by the free call (the absence of a ! symbol as part of the argument type means it is consumed). The addresses in these definitions have been changed to be agz which is short for 'address greater than zero' and is equivalent to the [l:addr | l > null] form shown previously.
In line 1 malloc is called passing it the size of an integer. The at-view is stored in the pf variable and the raw pointer in a. In line 2 the standard prelude function ptr_set_t is used to set the value at the pointer address to 10. This requires the passing of the at-view to prove that we have an integer stored at that memory address. pf is of type int @ l where l is the same address as the index type of a. Line 3 obtains the value at that address using the prelude function ptr_get_t. Again we need the proof to get it. Line 4 calls free and consumes the proof.
This example is safer than the previous examples:
free as we would be left with the linear at-view alive and it would fail to compile.ptr_get_t or ptr_set_t. These functions require an at-view that has the same address in it's type index as the raw pointers type index.malloc memory less than the size of the data we are storing there. The definition of malloc declares that its initial argument is of type sizeof_t int which is an integer that is the actual value returned by sizeof int. Any other value passed would be a compile error.The use of at-views, ptr_get_t and ptr_set_t tends to clutter code somewhat. ATS provides some syntax to dereference pointers that automatically look for valid at-view proofs in scope. The changes to the previous example to use this are:
implement main() = let
val (pf | a) = malloc {int} (sizeof<int>)
in
!a := 10;
print (!a); print_newline();
free (pf | a)
end
The ! operator is like the unary * operator in the C programming language. It dereferences a pointer. It requires an at-view to be in scope that has the same address as its indexed type as the pointer's indexed type. In this case the variable a uses address l which matches the int @ l type of the pf variable. := is assignment. As pf is in scope for both the assignment and the print call the code typechecks. If pf wasn't in scope the code would fail to compile.
The unary & operator can be used like in the C programming language to get a pointer to a variable. Given a variable containing an integer a, then &a is of type ptr. Note that this is a raw pointer which you can't derefence without an at-view in scope of type int @ l. To obtain an at-view from an existing variable on the stack you use the view@ function. The result of view@ on a variable of type T is an at-view of type T @ l where l is the memory address of the variable.
The following code has an add_one function that increments an integer by one given a pointer to it. The body of main has an integer, a, on the stack and view@ is used to obtain the at-view allowing dereferencing a pointer to it:
fn add_one {l:agz} (pf: !int @ l | p: ptr l): void =
! p := !p + 1
implement main() = let
var a: int = 0
in
print (a); print_newline();
add_one (view@ a | &a);
print (a); print_newline();
end
One of the earlier raw pointer examples in this post allocated a memory area containing two integers and used pointer arithmetic to access the second integer offset from the original pointer. So far the at-view usage examples here have only used single values stored at a memory address. To access consecutive memory locations we can use arrays. ATS doesn't have arrays built into the language, rather they are implemented using views, at-views and pointers.
To demonstrate how this is implemented we'll construct a simple example of rolling our own types to safely access memory containing two integers. Then I'll show how to use the standard prelude array handling for the same problem and hopefully that will give an idea of how things work under the covers.
If we have a memory area containing two integers that we need to access individually we will need at-views for both memory locations containing the integers. We could write a function that takes both at-views individually:
extern fun malloc (n: size_t (sizeof(int)*2)):
[l:agz] (int @ l, int @ (l+sizeof(int)) | ptr l) = "mac#malloc"
extern fun free {l:agz} (pf1: int @ l, pf2: int @ (l+sizeof(int)) | p: ptr l):void = "mac#free"
fn set {l:agz} (pf1: !int @ l, pf2: !int @ (l+sizeof(int)) | p: ptr l): void = begin
!p := 10;
!(p+sizeof<int>) := 20
end
implement main() = let
val (pf1, pf2 | a) = malloc(sizeof<int> * 2)
in
set (pf1, pf2 | a);
print (!a); print_newline();
print (!(a+sizeof<int>)); print_newline ();
free(pf1, pf2 | a)
end
In this example, malloc is defined to allocate a memory area containing two integers. It restricts the number of bytes it receives as an argument to match exactly that of the size of two integers. It returns a pointer and two at-views as proofs. The first being an int @ l and the second being an int @ (l+sizeof(int)). This provides a caller of malloc with at-views allowing access to the integers that can be stored in the returned memory.
The function set requires these two proofs so it can set the value of both integers in the memory area. The first is done via a simple dereference of the pointer, p. The second adds sizeof<int> to that pointer and dereferences it to set the value. This succeeds because there is a proof in scope (the pf2 at-view) that is for the correct type, int and the memory address of the value being set. A mistake in the pointer arithmetic would result in a compile error.
Passing multiple proofs around like this is a chore. To make it easier we can wrap them in a dataview. A dataview is like a datatype but creates a linear datastructure used in proofs. This is the same example with the at-views wrapped in a dataview:
dataview pair_v (l:addr) = PAIR (l) of (int @ l, int @ (l + sizeof(int)))
extern fun malloc (n: size_t (sizeof(int)*2)): [l:agz] (pair_v l | ptr l) = "mac#malloc"
extern fun free {l:agz} (pf: pair_v l | p: ptr l):void = "mac#free"
fn set {l:agz} (pf: !pair_v l | p: ptr l): void = let
prval PAIR (pf1, pf2) = pf
in
!p := 10;
!(p+sizeof<int>) := 20;
pf := PAIR (pf1, pf2)
end
fn show {l:agz} (pf: !pair_v l | p: ptr l): void = let
prval PAIR (pf1, pf2) = pf
in
print (!p); print_newline();
print (!(p+sizeof<int>)); print_newline ();
pf := PAIR (pf1, pf2)
end
implement main() = let
val (pf | a) = malloc(sizeof<int> * 2)
in
set (pf | a);
show (pf | a);
free(pf | a)
end
The pair_v dataview has a single constructor, PAIR, that takes two at-views as arguments, one for each integer held in the pair. malloc now returns a pair_v and free consumes it. For set to be able to dereference the pointer it needs to have the at-views in scope, not bundled in the pair_v instance. To get them in scope it uses pattern matching on the PAIR constructor to obtain the two proofs and have them in scope. As a dataview is a linear type this will consume the pair_v. This is why at the end of the scope the PAIR constructor is used to create the pair_v again and reassign it to the pf proof variable.
This proof unpacking and repacking is a common idiom for dealing with at-views and other proofs that need to be in scope. These are type level operations and are erased at compile time so contain no run time overhead.
The show function does the same to access the proofs to enable it to print the values at the memory address. It's also possible to use proof functions to further encapsulate access to the pointers.
A nice feature of the use of at-views is you can have an array of memory containing lots of values but when you pass pointers to functions requiring only portions of this array you can give it a specific at-view or proof for only the area it needs to manage. The function cannot access any other part of the array since it does not have a proof for it. For example:
dataview pair_v (l:addr) = PAIR (l) of (int @ l, int @ (l + sizeof(int)))
extern fun malloc (n: size_t (sizeof(int)*2)): [l:agz] (pair_v l | ptr l) = "mac#malloc"
extern fun free {l:agz} (pf: pair_v l | p: ptr l):void = "mac#free"
fn set1 {l:agz} (pf: !int @ l | p: ptr l): void =
!p := 10;
fn show1 {l:agz} (pf: !int @ l | p: ptr l): void =
(print (!p); print_newline())
implement main() = let
val (pf | a) = malloc(sizeof<int> * 2)
prval PAIR (pf1, pf2) = pf
in
set1 (pf1 | a);
show1 (pf1 | a);
pf := PAIR (pf1, pf2);
free(pf | a)
end
Although set1 and show1 here get a pointer to the memory address containing two integers they can only access the single integer at the pointer it is given. This is because the at-view it takes only is for int @ l. To access the integer following this it would need to somehow obtain the at-view for it and it cannot.
The ATS prelude has an array_v view for dealing with arrays that are a memory area containing consecutive elements of the same type. The dataview definition looks like (it is actually not defined like this, but operates the same as if it was):
dataview array_v ( a:viewt@ype+, int, addr) =
| {n:int | n >= 0} {l:addr}
array_v_cons (a, n+1, l) of (a @ l, array_v (a, n, l+sizeof a))
| {l:addr} array_v_nil (a, 0, l)
An array_v has three type indexes. They are the type of the item in the array, the number of items in the array, and the address of the first element. It has two constructors. array_v_nil is an empty array. array_v_cons has two arguments, the first being the at-view for the first item and the second being an array_v of the rest of the array. This is the previous pair example using array_v instead:
staload "prelude/SATS/array.sats"
extern fun malloc (n: size_t (sizeof(int)*2)): [l:agz] (array_v (int, 2, l) | ptr l) = "mac#malloc"
extern fun free {l:agz} (pf: array_v (int, 2, l) | p: ptr l):void = "mac#free"
fn set1 {l:agz} (pf: !int @ l | p: ptr l): void =
!p := 10;
fn show1 {l:agz} (pf: !int @ l | p: ptr l): void =
(print (!p); print_newline())
implement main() = let
val (pf | a) = malloc(sizeof<int> * 2)
prval (pf1, pf2) = array_v_uncons (pf)
in
set1 (pf1 | a);
show1 (pf1 | a);
pf := array_v_cons (pf1, pf2);
free(pf | a)
end
There are various other operations for dealing with packing and unpacking the at-views from the array_v view in prelude/SATS/array.sats. Due to the list like nature of the array_v definition it's easy to write recursive routines that walk through the array providing only access to the single item currently being operated on.
When using pointers you often have to deal with a pointer to memory that has not yet been initialized. This is common when first allocating the memory. ATS differentiates unitialized memory from initialized by appending a ? to the type. For example, an int? is a integer that has not been initialized. A int? @ l is an at-view for an uninitialized integer at memory address l. When the variable is initialized the type changes to drop the ?.
This is useful for defining functions that must have valid data. The following will fail to compile because the show1 function requires the memory to first have been set:
extern fun malloc (n: sizeof_t int): [l:agz] (int? @ l | ptr l) = "mac#malloc"
extern fun free {l:agz} (pf: int? @ l | p: ptr l):void = "mac#free"
fn set1 {l:agz} (pf: !int? @ l >> int @ l | p: ptr l): void =
!p := 10;
fn show1 {l:agz} (pf: !int @ l | p: ptr l): void =
(print (!p); print_newline())
implement main() = let
val (pf | a) = malloc(sizeof<int>)
in
// set1 (pf | a);
show1 (pf | a);
free(pf | a)
end
Uncommenting the set1 call enables it to compile. The set1 definition declares that it requires an at-view for an uninitialized int and after calling the at-view becomes one for an initialized int. The syntax T1 >> T2 means that on calling a T1 is required but after calling it becomes a T2.
ATS provides access to pointers at the level of the C programming language but provides a type system, combined with proofs, that enable using this memory safely. This comes with no runtime overhead. There are many other ways of structuring memory access using ATS, beyond the simple examples here. ATS allows memory allocation on the stack, which is reclaimed when the stack frame exits, and prevents access to the memory after it is reclaimed, optional garbage collection, and other features which I haven't gone through.
Some pointers to futher reading on this subject, and which helped with the writing of this post, are:
2012-12-19
I've written before about how Inferno makes it easy to share resources across devices and networks. In this post I show how to use IRC from within Inferno and then how to run the IRC connection handling on a server and the GUI on a client machine. The server can stay connected giving a persistent IRC connection similar to how IRC proxies and bouncers are used.
The IRC client I'm using for Inferno is written by mjl. A large number of useful Inferno and Plan 9 utilities with source is available at mjl's bitbucket repository. The one that provides IRC capability is ircfs. This is written in a manner that exposes IRC connections as a file system using the 9p2000 protocol. The GUI communicates with IRC via the file system.
Building from within Inferno requires a couple of environment variables to be set. ROOT is the path to the root of the Inferno system and SYSHOST should be set to 'Inferno':
% cd ircfs
% ROOT=/
% SYSHOST=Inferno
% mk install
cd module ; mk install
mk: 'install' is up to date
cd appl ; mk install
cd cmd ; mk install
rm -f /dis/testirc.dis && cp testirc.dis /dis/testirc.dis
rm -f /dis/ircfs.dis && cp ircfs.dis /dis/ircfs.dis
cd wm ; mk install
mk: 'install' is up to date
cd lib ; mk install
mk: 'install' is up to date
cd man ; mk install
mk: 'install' is up to date
This installs ircfs into the standard Inferno application and command locations. It may not be desirable to clutter the existing system locations with additional programs like this. If you prefer you can create local directories to hold the installed programs and bind this over the system directories using a union file system. ircfs installs a useful man page which is worth reading to find out all the options.
When ircfs is executed it creates a directory structure that contains the files that allow clients to send and receive data to the service. To run it we first create a directory underneath /mnt/irc for the server we want to connect to, then run ircfs passing this directory name:
% mkdir -p /mnt/irc/freenode
% mount {ircfs freenode} /mnt/irc/freenode
% ls /mnt/irc/freenode
/mnt/irc/freenode/0
/mnt/irc/freenode/ctl
/mnt/irc/freenode/event
/mnt/irc/freenode/nick
/mnt/irc/freenode/pong
/mnt/irc/freenode/raw
The example above shows the files that exist after running and mounting the ircfs service. The mount command uses the variant that runs and mounts a 9p service.
The running ircfs hasn't connected to any IRC server yet. Connections and controlling of the IRC server is managed via writing or reading from the files under /mnt/irc/freenode. The protocol is explained in the ircfs man page. For example, writing to ctl allows us to connect to a server. Reading from nick gives the current nickname used by the connected user.
Ensure that ndb/cs is running to allow network connections from Inferno before continuing.
The GUI for ircfs is run with the command wm/irc, optionally passing the path to the ircfs directory structure as an argument:
% wm/irc /mnt/irc/freenode
On the left of the wm/irc screen is a list of the servers and channels. Currently there is status and (freenode). The (freenode) window is for sending commands to the irc server once we're connected. To connect to irc.freenode.net enter the following in the (freenode) area:
/connect irc.freenode.net mynick
The status messages from the server should now appear. The commands that ircfs understands are documented in the /ctl section of the ircfs man page. Standard IRC client commands like /join, etc work and the channels will appear on the list in the left of the client window. The ircfs server remains running and connected if the GUI is closed. Running the GUI again with the path to the ircfs directory will display the existing channels connected and you can continue where you left off before it was closed.
The GUI knows some special commands that can be passed to it using /win. These are explained in the main page for wm/irc.
add can be used to add new ircfs directories to a running GUI. If you start wm/irc without passing it a directory for example you can type the following in the status window to connect to one:
/win add /mnt/irc/freenode
del removes the current ircfs directory so the GUI no longer displays informatation about it. You can use add to get the GUI to redisplay it. Note that these two commands don't affect the actual IRC connection. That remains managed by ircfs. It only affects what is displayed in the GUI:
/win del
The windows command gives a list of all the windows for the current ircfs connection and their control number. This number can be used to close and open channel windows in the GUI:
/win windows
open:
(freenode) (0)
#inferno (1)
not open:
/win close
/win windows
open:
(freenode) (0)
not open:
#inferno (1)
/win open 1
/win windows
open:
(freenode) (0)
#inferno (1)
not open:
away can be used to mark the user as away or back on all servers. exit will exit the GUI but leave ircfs connections intact.
I prefer to have a persistent IRC connection on a server and connect to it from a client so I can catch up with channel logs and activity while I've been away. This allows other users to leave me messages when I'm disconnected. It also allows connecting using multiple devices and keeping my conversation active. This approach usually requires running IRSSI in a screen session on a server, or running an IRC proxy or bouncer.
A similar approach can be done using ircfs by running the ircfs server on a remote machine and on the client mounting the ircfs filesystem. The connection can be encrypted and authenticated.
The following is a simple setup. Install Inferno on the remote server and build and install ircfs as described previously. Run ircfs to run the server. In this example I also add a command line switch to log the IRC connection so I can have channel logs:
; mkdir -p /mnt/irc/freenode
; mount {ircfs -t -l /usr/myuser/irclogs freenode} /mnt/irc/freenode
; styxlisten net!*!16667 {export /mnt/irc}
On the client we mount this exported filesystem using encryption (replace example.com with the hostname of the server):
% mount -C sha1/rc4_256 net!example.com!16667 /mnt/irc
% ls /mnt/irc/freenode
/mnt/irc/freenode/0
/mnt/irc/freenode/ctl
/mnt/irc/freenode/event
/mnt/irc/freenode/nick
/mnt/irc/freenode/pong
/mnt/irc/freenode/raw
The directory and files shown are those from the remote server. Running wm/irc will now use the servers ircfs files:
% wm/irc /mnt/irc/freenode
You can now connect to the irc.freenode.net server, join channels then close the IRC window and shut down the client. Next time you restart the client, mount the remote filesystem again and continue where you left off.
The man pages for ircfs and wm/irc have more detail on the commands and keys that can be used. There are other things that can be done with ircfs other than GUI IRC programs. It's possible to write IRC bots and programs by utilising the control files and directories of an ircfs session.
The ircfs source is written in Limbo and is quite easy to follow. It's a good example of mapping an existing protocol onto Inferno's "everything is a file" metaphor and making use of the tools to share resources. One use of this is having Inferno on other devices, like a phone, so you can connect to IRC connections running on a local machine if you temporarily need to be mobile.
2012-12-18
In my post about sharing resources in Inferno I used the -A command line switch to commands to disable authentication. This allowed anyone to make connections and the data was passed across them unencrypted. This post explains how to set up an Inferno system to use authentication and to encrypt connections.
For authentication to work each machine that wishes to communicate securely must have a certificate signed by a common signing authority. That signing authority should be a separate machine with the sole task of running the signing processes.
The signing server needs to have a public and private key used for signing certificates. To generate these keys run the command auth/createsignerkey from within the Inferno shell. The command takes a name argument which is the name that will appear as the signer name in the certificates generated by the server:
; auth/createsignerkey example.com
This command creates a /keydb/signerkey file containing the keys. It's important to note that if this file is regenerated then any existing certificates issued by the server will be invalidated as the private key will have been lost. Keep this file safe!
The act of authentication involves checking that the signing server and the user know a shared secret. The secrets are kept in the /keydb/keys file. The file, which starts off empty, is protected by a password. Run the svc/auth program to start the services which manage this file. You will need to provide the password, or create one if none exists:
; svc/auth
Key: ...enter password/passphrase...
Confirm key: ...confirm password/passphrase...
Each user to be authenticated must be set up on the signer by running the auth/changelogin command. This prompts for a password and an expiry date for the user. The expiry date is used as the maximum valid date for the certificates created for that user. At least one user must exist. Create a user on the signer for the current user on the signing machine (This might be inferno or your host username, depending on your setup). You need to run auth/changelogin for each remote or local user to be authenticated. That user can then run passwd command at a later time to change the password.
; auth/changelogin myuser
new account
secret: ...enter password/passphrase...
confirm: ...confirm password/passphrase...
expires [DDMMYYYY/permanent, return = 17122013]: ...enter expiry date or confirm default...
change written
A client of authenticated Inferno services needs to have a certificate obtained from the signing server. The request for this certificate is done using getauthinfo. There is a also a wm/getauthinfo variant that uses the Inferno GUI to prompt for required data. Other that that it is functionally the same as getauthinfo.
The keyname argument to getauthinfo should either be default or of the form net!example.com where 'example.com' is the hostname of the server with the resources being accessed. This must exactly match the name given to bind or mount when accessing the service. This is how those commands know to find the certificate file. The default keyname is used by file servers as the default certificate file for incoming connections.
When running getauthinfo you are prompted for the signing server to be used, the username on the signing server that you wish to get the certificate for, a password for that username (which was created with auth/changelogin previously, and whether you want to save the certificates in a file. If you answer yes to 'save in file?' then an actual physical file is created in the users /usr/username/keyring directory. If 'no' is answered then a fileserver is started on the client which serves a secure temporary file bound over the name in the given directory. This is removed when the file is unbound or Inferno is stopped.
For the following example I create a default file on a new server so it can act as a file server to remote clients. I'm using file.example.com as the hostname for this file server and example.com as the hostname for the signing server - replace this with the actual servers name. In these examples I assume that ndb/cs has been run to provide network services:
; getauthinfo default
use signer [$SIGNER]: example.com
remote user name [myuser]: ...enter username...
password: ...enter a password....
save in file [yes]: yes
On a client machine that wants to access resources on the file server run getauthinfo, but using the net!hostname form of the keyname, so bind and mount can find it.
; getauthinfo net!file.example.com
use signer [$SIGNER]: example.com
remote user name [myuser]: ...enter username...
password: ...enter password for user...
save in file [yes]: yes
The request for certificates only needs to be done once, assuming 'save in file?' was set to 'yes'. The certificate information is valid until the expiry date set for the user.
Both server and client can now connect securely now that they have certificates provided by the common signing server. An example from my previous post on sharing resources was sharing a directory. On the file server:
; listen 'tcp!*!8000' { export '/usr/myuser' & }
This no longer has the -A switch to disable authentication. To connect from the client:
; mount 'tcp!file.example.com!8000' /mnt/remote
This command will actually fail with a message saying it couldn't find 'keyring/default' if we haven't generated a default certificate. This is because our connection string includes a port and our getauthinfo previously did not. Our getauthinfo used 'net!file.example.com' but we're connecting to 'net!file.example.com!8000'. The default for mount is to look for a file with the name exactly as specified with the host portion of the connection string and falling back to 'default'.
We can solve this by generating a default file with getauthinfo, or one for net!file.example.com!8000 which includes the port, or specifying the exact file on the mount command line by using the -k switch:
; mount -k 'net!file.example.com' 'tcp!file.example.com!8000' /mnt/remote
These mount examples will authenticate using the certificates but will not encrypt the session. To encrypt as well as authenticate use the -C switch to mount to set the hashing and encryption algorithm to use. See the ssl documentation for the supported algorithms. For example:
; mount -C sha1/rc4_256 'tcp!file.example.com!8000' /mnt/remote
All the examples in my previous post can now be done with authenticated and encrypted sessions.
The examples in this post show explicitly passing the hostname of the signing server in places. The file /lib/ndb/local contains information for default servers. Changing the SIGNER key to the signing servers hostname will avoid the need of always specifying this information.
The signing server should have the sole task of running svc/auth. It shouldn't be used for other tasks. Doing so results in issues where parts of svc/auth exit when a client fails to authenticate resulting in the signing server no longer handling authentication. This is briefly warned about in svc/keyfs:
Keyfs should be started only on the machine acting as authentication server (signer), before a listener is started for signer(8). Note that signer and keyfs must share the name space. Furthermore, no other application except the console should see that name space.
This mailing list post writes more about this. This is why I use a separate file server to export the path in the listen statement in the examples here.
The following sources have additional information and examples on using authentication and encryption in Inferno. In particular the man pages for the commands have a lot of detail:
2012-11-07
I posted previously about running the Inferno operating system on an Android phone. Inferno on a phone device interests me because of the approaches it takes on distributing resources across machines. By having such a system on multiple machines and devices it should be possible to use features of the phone on a desktop system (like SMS messaging, copying photos back and forth, etc) or use features of servers on the phone (cloud file storage for example).
In this post I'll go over some simple examples of how to share resources using Inferno on a desktop machine and a remote server. I'll then show how an Inferno based phone can shares its resources. I use the hosted version of Inferno for these examples where Inferno runs as a user process under an existing operating system.
To build Inferno under Linux you need to obtain the inferno-20100120.tgz file from the Inferno download page. This file contains a snapshot of the mercurial based source code repository and the binary font files used by the system. Once unpacked it needs to be updated to the latest source code version using mercurial commands:
$ wget http://www.vitanuova.com/dist/4e/inferno-20100120.tgz
$ tar zxvf inferno-20100120.tgz
$ cd inferno
$ hg pull -u
Once the update is complete you will need to edit the mkconfig file so that the following settings are changed:
ROOT=/root/of/the/inferno/directory
SYSHOST=Linux
OBJTYPE=386
Ensure that ROOT matches the directory location where the Inferno source code was unpacked. To build, set your PATH to the Linux/386/bin subdirectory of the unpacked Inferno source and run the following commands:
$ export PATH=$PATH:~/src/inferno/Linux/386/bin
$ ./makemk.sh
$ mk nuke
$ mk install
The result is an emu executable living in Linux/386/bin.
You can run Inferno and interact directly with a shell using emu:
$ emu
; ls
...file listing...
This is how I run Inferno on a headless server. On a machine with a display you can run a GUI:
$ emu -g1024x768
; wm/wm
...window system appears...
The -g command line option sets the size of the Inferno OS host window.
Inferno has a concept called a namespace which is a hierarchical collection of files or resources. Every process running in Inferno has its own local namespace. By modifying this namespace for each process you can grant or deny access to particular resources.
Some commands that manipulate the namespace are:
bindmountunmountexportThese commands will be used in the examples following to show how to share and access resources from local and remote machines.
Bind attaches, moves or hides local resources. The simplest case is binding an existing directory to a new location. For example:
; mkdir /tmp/myappl
; bind /appl /tmp/myappl
This makes the directory /appl, which contains the source for some of the Inferno OS commands, to the location /tmp/myappl. the current shell and all its child processes can see the /tmp/myappl directory. Processes that aren't children of the current shell cannot.
Binding over an existing directory hides that directory until it is unbound (using unmount). It's possible to bind over an existing directory such that the source directory is overlaid with the existing directory. You can choose whether files in the source directory or the destination directory have precedence if there are duplicate names. This is known as a union filesystem.
An example of union filesystem usage is replacing the PATH environment variables in other operating systems. The /dis directory on Inferno holds the executable Inferno OS commands. Instead of maintaining PATH environment variables for users to manage order of lookup for locally built commands and global OS commands you instead manage it via union directories using bind.
If there is a directory in the user's home directory called dis, you can allow commands located there to be looked up first by doing:
; bind -b /usr/myname/dis /dis
if you want files in the global dis directory to have precedence:
; bind -a /usr/myname/dis /dis
You can stack as many directories as desired. The -a command line argument means add 'after' the union directory. the -b command line argument means add 'before' the union directory.
When running Inferno as a user process on an existing host operating system you can bind to directories that exist on the host. The special kernel device path, '#U', is for the host file system:
; bind '#U*' /tmp/z
Now the directory /tmp/z is mapped to the root of the host. You can map any host path, not just the root:
; bind '#U*/home' /tmp/z
This can be used to mount a users host filesystem home directory to be used as their Inferno home directory:
; bind '#U*/home' /usr
I start Inferno with the home directory shared and logged into the Inferno system as my host username using a shell script which runs the equivalent of:
$ export PATH=$PATH:/src/inferno/Linux/386/bin
$ export EMU="-r/home/$USER/src/inferno -c1 -g1920x1008"
$ exec emu $* /dis/sh.dis -a -c "bind '#U*/home' /usr; wm/wm wm/logon -u $USER"
For this to work you'll need to ensure you have the directories and files Inferno expects to find in the home directory. The files from the Inferno /usr/inferno directory can be copied for this purpose.
Note: One important point to note in the following examples is that the -A switch used in both the listen and mount commands disables authentication. All the data is being sent unencrypted and no login information was used to connect. Don't do this for real systems, I'm only doing this to show the basic commands. It's possible and recommended to set up authentication and encryption for real world usage. Once done this can replace the use of sftp on Inferno systems. Just export and bind the filesystems needed and all access is authenticated and all data encrypted.
To access a resource on another machine (or another instance of hosted Inferno OS on the same machine) you can run the listen command on the remote machine, exporting the namespace you want the client machine to be able to bind:
$ emu
; ndb/cs
; ndb/dns
; listen -A 'tcp!*!8000' { export '#U*/home/myuser' & }
;
The ndb/cs and ndb/dns commands start the network services used for DNS lookup and other features. the listen command starts a listener on port 8000 exporting the myuser home directory from the host filesystem. I picked 8000 as an arbitary port number - any one is fine. A client machine can now connect to this using:
$ emu
; ndb/cs
; ndb/dns
; mkdir /tmp/myuser
; mount -A 'tcp!remote.example.com!8000' /tmp/myuser
; cd /tmp/myuser
; ls
...remote file listing...
; unmount /tmp/myuser
Replace 'remote.example.com' with the IP address or domain name of the remote machine. The mounted directory works like a local directory from the point of view of the client. You can get directory listings, copy files, edit files, etc.
This accessing of remote resources isn't just limited to files. You can export other kernel devices. You can remotely list and debug running processes on a remote machine by exporting and mounting /prog. On the remote machine:
; listen -A 'tcp!*!8000' { export /prog & }
On the client machine:
; mkdir /tmp/debug
; mount -A 'tcp!remote.example.com!8000' /tmp/debug
; ls /tmp/debug
...list of processes on remote machine...
; unmount /tmp/debug
Another interesting example is mounting a remote machines /net directory over the top of the existing clients /net directory. The result of this is all network access made via the shell and its children will use the remote machine's network. This effectively creates a network tunnel that is encrypted and authenticated (if you don't use the -A switch and setup the authentication system).
To demonstrate this lets assume the client machine can't accept incoming connections due to NAT. The remote machine however is unrestricted. On the remote machine run:
; listen -A 'tcp!*!8000' { export /net & }
On the client machine we connect outwards as before - outbound connections aren't restricted:
; webgrab http://automation.whatismyip.com/n09230945.asp
; cat n09230945.asp
...public facing ip address of client machine...
; mount -A 'tcp!remote.example.com!8000' /net
This mount usage binds the remote /net onto our own /net. Any future network requests from this shell on the client machine will come from the remote:
; webgrab http://automation.whatismyip.com/n09230945.asp
; cat n09230945.asp
...ip address of remote machine...
We can even expose resources on the client machine. The following is run on the client:
; listen -A 'tcp!*!8001' { export / & }
On a third machine, that has no access to the client at all, but can access the remote machine:
; mkdir /tmp/client
; mount -A 'tcp!remote.example.com!8001' /tmp/client
; ls /tmp/client
...client file listing...
You'll note that we're connecting to port 8001 on the remote machine. This is where the listen is attached to due to us mapping the remotes /net directory on the client. But it's the client's root directory that was exported so that's what the third machine gets access to. No incoming connections are made to the client itself.
For this example you'll need to have Inferno running on an Android phone. This can be done by installing Hellaphone or following my instructions on building and installing Inferno on a Nexus S using Mozilla's Boot to Gecko as a base.
SMS messages can be sent on the phone by writing to the '/phone/sms' file. The following would send a simple text message to number +6412345678 (an invalid New Zealand number):
; echo send 6412345678 'hello' >/phone/sms
By exporting /phone and mounting it on a desktop system we can control the sending of SMS messages from there. On the phone:
; ndb/cs
; ndb/dns
; listen -A 'tcp!*!8000' { export /phone & }
On the desktop machine:
; ndb/cs
; ndb/dns
; mkdir /tmp/phone
; mount -A 'tcp!ip.address.of.phone!8000' /tmp/phone
; echo send 6412345678 'hello' >/tmp/phone/sms
It's also possible to detect ringing, answer calls, read received SMS messages, etc.
Unfortunately things are a bit more complex in the real world. My phone carrier doesn't allow inbound connections to the phone. They also seem to prevent outbound connections on some ports. Highly annoying. To fix this I installed OpenSSH on the phone using opkg. Using OpenSSH I created a tunnel to my remote server which I could then map /net in a similar manner to what I've described previously.
On my remote server:
$ emu
; ndb/cs
; ndb/dns
; listen -A 'tcp!*!8000' { export /net & }
On the phone:
# ssh -N myuser@remote.example.com -L 8000:127.0.0.1:8000 &
# emu-g
; ndb/cs
; ndb/dns
; mount -A 'tcp!127.0.0.1!8000' /net
; listen -A 'tcp!*!8001' { export /phone & }
On my desktop:
; ndb/cs
; ndb/dns
; mount -A 'tcp!remote.example.com!8001' /tmp/phone
; echo send 6412345678 'hello' >/tmp/phone/sms
The routing through the remote server is the same as my previous example. There's probably a way of avoiding the OpenSSH tunnelling by using another port but I didn't spend time investigating exactly what the carrier was doing.
Inferno makes it quite easy to share resources amongst devices. The idea of having a phone which can attach to other machines to access files and export phone functionality to other machines to more conveniently write SMS messages and emails appeals to me. It'll be interesting to see how far this can be pushed and what ideas people come up with in this area. Hopefully more development will go into Inferno on the phone and some of these ideas can be explored.
Remember that the examples above that used the -A switch to mount and listen used unencrypted and unauthenticated sessions. This is useful for quick testing but for actual usage -A should be avoided. Setting up authentication is described at Pete Elmore's post on clusters. The 4th edition release notes also has information on this.
Other useful Inferno resources:
This site is accessable over tor as hidden service 6vp5u25g4izec5c37wv52skvecikld6kysvsivnl6sdg6q7wy25lixad.onion, or
Freenet using key:
USK@1ORdIvjL2H1bZblJcP8hu2LjjKtVB-rVzp8mLty~5N4,8hL85otZBbq0geDsSKkBK4sKESL2SrNVecFZz9NxGVQ,AQACAAE/bluishcoder/-61/
