Bluish Coder

Using Tor with Firefox OS

Update - Please read my followup post for some additional information and updated steps on building and installing tor on Firefox OS.

Please read the disclaimer at the end of this article. This is a proof of concept. It's a manual process and you shouldn't depend on it. Make sure you understand what you are doing.

I'm a fan of Tor. The Tor site explains what it does:

Tor is free software and an open network that helps you defend against traffic analysis, a form of network surveillance that threatens personal freedom and privacy, confidential business activities and relationships, and state security.

I make my personal website available as a Tor hidden service accessible from mh7mkfvezts5j6yu.onion. I try to make other sites I'm involved with also have a presence over Tor. I do a fair amount of my browsing over the Tor network for no reason other than I can and it limits the opportunity for people snooping on my data.

I want to be able to use Tor from Firefox OS. In particular I want it embedded as low level as possible so I have the option of all traffic going over Tor. I don't want to have to configure socks proxies.

Firefox OS doesn't allow native applications. The low level underlying system however is based on Linux and Android and can run native binaries. Starting with a rooted Firefox OS install I built Tor and used iptables to reroute all network traffic to work over it. This is a first step and is what this article demonstrates how to get going so power users can try it out. My next step would be to investigate integrating it into the build system of Firefox OS and providing ways to start/stop it from the OS interface.

The first stage of building is to have an Android standalone toolchain installed. I describe how to do this in my Wasp Lisp on Android post or you can use a Nix package I created for use with the Nix package manager.

Building libevent

Tor requires libevent to build. I'm using static libraries to make creating a standalone tor binary easier. The following will build libevent given the standalone toolchain on your path:

$ cd $HOME
$ mkdir build
$ cd build
$ wget https://github.com/downloads/libevent/libevent/libevent-2.0.21-stable.tar.gz
$ tar xvf libevent-2.0.21-stable.tar.gz
$ cd libevent-2.0.21-stable
$ ./configure --host=arm-linux-androideabi \
              --prefix=$HOME/build/install \
              --enable-static --disable-shared
$ make
$ make install

Building zlib

Tor requires openssl which in turn requires zlib:

$ cd $HOME/build
$ wget http://zlib.net/zlib-1.2.8.tar.gz
$ tar xvf zlib-1.2.8.tar.gz
$ cd zlib-1.2.8
$ CC=arm-linux-androideabi-gcc ./configure --prefix=$HOME/build/install --static
$ make
$ make install

Building openssl

$ cd $HOME/build
$ wget http://www.openssl.org/source/openssl-1.0.1h.tar.gz
$ tar xvf openssl-1.0.1h.tar.gz
$ cd openssl-1.0.1h
$ CC=arm-linux-androideabi-gcc ./Configure android no-shared --prefix=$HOME/build/install
$ make
$ make install

Building tor

$ cd $HOME/build
$ wget https://www.torproject.org/dist/tor-0.2.4.22.tar.gz
$ cd tor-0.2.4.22
$ ./configure --host=arm-linux-androideabi \
              --prefix=$HOME/build/install \
              --enable-static-libevent
$ make
$ make install

Packaging Tor for the device

To run on the Firefox OS device I just installed the tor binary and a configuration file that enables transaparent proxing as per the Tor documentation on the subject. I put these in a directory that I push to an easily accessible place on the device:

$ mkdir $HOME/build/device
$ cd $HOME/build/device
$ cp $HOME/build/install/bin/tor .
$ cat >torrc
  ...contents of configuration file...
$ adb push $HOME/build/device /data/local/tor

The configuration file is:

DataDirectory /data/local/tor
Log notice file /data/local/tor/tor.log
RunAsDaemon 1
SOCKSPort 127.0.0.1:9050 IsolateDestAddr
SOCKSPort 127.0.0.1:9063
VirtualAddrNetwork 10.192.0.0/10
AutomapHostsOnResolve 1
TransPort 9040
DNSPort 9053

Running tor

I haven't integrated tor into the device at all so for this proof of concept I adb shell into it to run it and configure the iptables to redirect traffic:

$ adb shell
# cd /data/local/tor
# ./tor -f torrc &
# iptables -t nat -A OUTPUT ! -o lo -p udp --dport 53 -j REDIRECT --to-ports 9053 
# iptables -t nat -A OUTPUT ! -o lo -p tcp -j REDIRECT --to-ports 9040

Testing

The device should now be sending traffic over Tor. You can test by visiting sites like whatismyip.com or icanhazip.com to see if it reports a different IP address and location to what you normally have. You can also try out hidden services like mh7mkfvezts5j6yu.onion which should show this site.

Removing

Killing the Tor process and removing the iptables entries will set the network back to normal:

$ adb shell ps|grep tor
$ adb shell
# kill ...process id of tor...
# iptables -t nat -F

You can optionally delete the /data/local/tor directory to remove all tor files:

$ adb shell rm -r /data/local/tor

Future

This is just a proof of concept. Don't depend on this. You need to restart Tor and the iptables commands on reboot. I'm not sure how well interaction with switching to/from WiFi and GSM works. Ideally Tor would be integrated with Firefox OS so that you can start and stop it as a service and maybe whitelist or blacklist sites that should and shouldn't use Tor. I hope to do some of this over time or hope someone else gets excited enough to work on it too.

Another privacy aspect I'd like to investigate is whether TextSecure (or a similar service) could be integrated in the way it's done in CyanogenMod:

"The result is a system where a CyanogenMod user can choose to use any SMS app they'd like, and their communication with other CyanogenMod or TextSecure users will be transparently encrypted end-to-end over the data channel without requiring them to modify their work flow at all."

Ideally my end goal would be to have something close to that described in the hardening Android post on the Tor Project blog.

I'm not sure how possible that is though. But Firefox OS is open source, easy to build and hack on, and runs on a lot of devices, including multi booting on some. Adding things like this to build your own custom phone OS that runs web applications is one of the great things the project enables. Users should feel like they can dive in and try things rather than wait for an OS release to support it (in my opinion of course).

Test Builds

A tar file containing a precompiled tor and the torrc is available at b2g_tor.tar.gz.

Disclaimer

All files and modifications described and provided here are at your own risk. Don't tinker on devices you depend on and don't want to risk losing data. These changes are not an official Mozilla project and do not represent any future plans for Mozilla projects.

Dual Booting Android and Firefox OS on the Nexus 5

I've gone through periods of using a Firefox OS phone as my main device but I've usually fallen back to Android due to needing to use some Android only programs and I don't like carrying two phones around. Today I decided to investigate how to get dual boot Android with custom Firefox OS builds. Thankfully it was actually pretty easy.

The boot manager I used to get this to work is MultiROM Manager, available from the Play store for rooted phones. The source is for MultiROM Manager is available on github. The phone I used was the Nexus 5. The instructions here assume you are familiar with adb and fastboot already.

Be aware that all these changes may lose the data you have on the device if you haven't already unlocked the boot loader and rooted the device.

Make a backup of your Android settings and applications

With the device plugged in and visible from adb:

$ adb backup -apk -shared -all

This can be restored later if needed with:

$ adb restore backup.ab

Unlock the bootloader

The Nexus 5, and other Google devices, make it easy to unlock the bootloader. With the device plugged in and visible from adb:

$ adb reboot bootloader
$ fastboot oem unlock

Follow the screen instructions. This will erase everything on the device!

Rooting the Nexus 5

I used CF-Auto-Root. I downloaded the version for the Nexus 5 and used fastboot to boot the image inside of it:

$ unzip CF-Auto-Root-hammerhead-hammerhead-nexus5.zip
$ fastboot boot image/CF-Auto-Root-hammerhead-hammerhead-nexus5.img

The device will reboot and perform the steps necessary to root it.

Install MultiROM Manager

Install MultiROM Manager from the Play store. Run the app and choose Install after ticking the MultiROM, Recovery and Kernel check boxes. Follow the onscreen instructions.

Build Firefox OS

The Mozilla Developer Network has instructions for building Firefox OS. Assuming all the pre-requisites are installed the steps are:

$ git clone git://github.com/mozilla-b2g/B2G b2g
$ cd b2g
$ ./config.sh nexus-5
$ PRODUCTION=1 MOZILLA_OFFICIAL=1 ./build.sh

Don't flash the device from here. We'll create a MultiROM compatible ROM file to boot from.

Create Firefox OS ROM file

Create a directory to hold the ROM contents and copy the results of the build into it:

$ mkdir rom
$ cd rom
$ rsync -rL ../out/target/product/hammerhead/system .
$ rsync -rL ../out/target/product/hammerhead/data .
$ cp ../out/target/product/hammerhead/boot.img .

For the rsync copy I deliberately choose not to copy symbolic links and to instead re-copy the original file. I had difficulty getting symbolic links working and need to investigate.

An Android ROM requires a META-INF directory containing a script that performs the update process. The following commands create this directory, copy the binary to run the script and the script itself:

$ mkdir -p META-INF/com/google/android/
$ cp ../tools/update-tools/bin/gonk/update-binary META-INF/com/google/android/
$ curl http://bluishcoder.co.nz/b2g/updater-script >META-INF/com/google/android/updater-script

The updater script is one I wrote based on existing ones. It's pretty easy to follow if you want to read and change it.

The final step is to ZIP the directories, sign them and push to a directory on the device:

$ zip -r9 b2g.zip *
$ java -jar ../prebuilts/sdk/tools/lib/signapk.jar \
            ../build/target/product/security/testkey.x509.pem \
            ../build/target/product/security/testkey.pk8 \
             b2g.zip signed_b2g.zip
$ adb push signed_b2g.zip /sdcard/

Install Firefox OS ROM

Boot into recovery mode by pressing volume down and the power on button at the same time (or run adb reboot recovery). From the recovery menu choose 'Advanced' followed by 'MultiROM', then Add ROM.

Make sure Android is selected and Don't Share is chosen for "Share Kernel with Internal ROM". Click Next, choose Zip file and select the file we created in the signing step previously. Swipe to confirm as requested.

If this succeeds, Reboot and touch the screen during the 'Auto boot' display to get the list of ROMS to run. Choosing the one we just installed should boot Firefox OS.

Other ROMs

With MultiROM you can install other ROMS and even Ubuntu Touch. I'd like to get Inferno OS running under MultiROM as well so I can boot between all the operating systems I like to tinker with on one device.

Try it

I've placed a complete Firefox OS ROM for use with MultiROM on the Nexus 5 in b2g_nexus5.zip. This was built from B2G master branch so may be broken in various aspects (The camera doesn't work for example) but will allow you to try the multi boot process out if you can't do builds. This is not an official Mozilla build and was generated by me personally. Use at your own risk.

PicoLisp on Android

PicoLisp is a small Lisp interpreter that tries to stay true to the path of 'code is data' as much as possible. It has some nice features, including a persistent database and built in prolog engine. The system is quite reflective and this is shown in the way you can use vim to browse the live system.

There is a 32-bit and 64-bit version. The 64-bit version has some nice additional features but for running on Android the 32 bit version is needed. Building for Android is fairly easy. The approach is similar to the way I built Wasp Lisp on Android.

The first step is to install a 'standalone toolchain' version of the Android NDK. The steps to do this is described in my Wasp Lisp post so I won't repeat them here. The main complication for my setup was getting this to work on NixOS. I created a Nix package definition to install the NDK and generate the standalone toolchain.

The 32-bit version of picoLisp is built from the src subdirectory in the source download. Some minor changes are needed to use the Android cross compiler from the standalone toolchain. I've put these in android.patch. To apply the patch and build:

$ tar xvf picoLisp-3.1.6.tgz
$ cd picoLisp
$ curl http://bluishcoder.co.nz/picolisp/android.patch | patch -p1
$ cd src
$ make TARGET=Android

To make testing easier I push the entire PicoLisp install to my Android device. For JellyBean and above it might be hard to find a place to install it with the correct permissions. On my KitKat device I can use /data/local/tmp even though that directory isn't visible by listing it. You can push to it and cd to it from the shell:

$ cd picoLisp
$ adb push picoLisp /data/local/tmp/picoLisp

PicoLisp creates some files in the .pil directory off the HOME directory. This should be explicitly set when running it in the Android shell since the HOME directory is not writeable:

$ adb shell
$ cd /data/local/tmp/picoLisp
$ HOME=/data/local/tmp ./pil +
: (+ 1 2)
-> 3

Unfortunately the 32-bit version of PicoLisp doesn't have the Native C Calls that the 64-bit version has. It does allow inline C code but this requires a gcc compiler to be available. It may be possible to use a natively compiled gcc.

In the meantime it's possible to write shared libraries in C which can be loaded on Android. An example of doing this is ext.c in the distribution. This is already compiled and copied to Android in the steps above. Any Lisp symbol prefixed by a name plus ':' is checked if it exists in a shared library with the given name. So the symbol ext:Snx is the Snx function in the ext shared library. An example of calling this in Android:

: (ext:Snx "hello")
-> "HL"

Running the PicoLisp webserver on Android looks like:

$ cd /data/local/tmp/picoLisp
$ HOME=/data/local/tmp ./pil @lib/http.l  --server 8080 @doc/hello.l  -wait

This can be accessed from the web browser on the device at http://127.0.0.1:8080.

Unfortunately in recent versions of Android (at least KitKat) it's not possible to run PicoLisp from /data/local/tmp from within the terminal emulator apps available on the App store as far as I can see. I get permission denied errors. Earlier versions of Android work in my testing. If your phone is rooted this won't be a problem, it should work. I've also tested PicoLisp on a rooted Firefox OS phone and it works fine. Writing server side programs in PicoLisp that can be accessed as Web Applications on Firefox OS could be an interesting project for testing and prototyping ideas.

Other users have gotten PicoLisp running on Android as well:

• Running PicoLisp on Android with Web Server
• Mailing list post on the subject
• Android PicoLisp project

Firefox Development on NixOS

Now that I've got NixOS installed I needed a way to build and make changes to Firefox and Firefox OS. This post goes through the approach I've taken to work on the Firefox codebase. In a later post I'll build on this to do Firefox OS development.

Building Firefox isn't difficult as NixOS has definitions for standard Firefox builds to follow as examples. To build from a local source repository it requires all the pre-requisite packages to be installed. I don't want to pollute my local user environment with all these packages though as I develop on other things which may have version clashes. As an example, Firefox requires autoconf-2.13 whereas other systems I develop with require different verisons.

NixOS (through the Nix package manager) allows setting up build environments that contain specific packages and versions. Switching between these is easy. The file ~/.nixpkgs/config.nix can contain definitions specific for a user. I add the definitions as a packageOverride in this file. The structure of the file looks like:

{
  packageOverrides = pkgs : with pkgs; rec {
    ..new definitions here..
  };
}

My definition for a build environment for Firefox is:

firefoxEnv = pkgs.myEnvFun {
  name = "firefoxEnv";
  buildInputs = [ stdenv pkgconfig gtk glib gobjectIntrospection
                  dbus_libs dbus_glib alsaLib gcc xlibs.libXrender
                  xlibs.libX11 xlibs.libXext xlibs.libXft xlibs.libXt
                  ats pango freetype fontconfig gdk_pixbuf cairo python
                  git autoconf213 unzip zip yasm alsaLib dbus_libs which atk
                  gstreamer gst_plugins_base pulseaudio
                ];

  extraCmds = ''
   export C_INCLUDE_PATH=${dbus_libs}/include/dbus-1.0:${dbus_libs}/lib/dbus-1.0/include
   export CPLUS_INCLUDE_PATH=${dbus_libs}/include/dbus-1.0:${dbus_libs}/lib/dbus-1.0/include
   LD_LIBRARY_PATH=\$LD_LIBRARY_PATH:${gcc.gcc}/lib64
   for i in $nativeBuildInputs; do
     LD_LIBRARY_PATH=\$LD_LIBRARY_PATH:\$i/lib
   done
   export LD_LIBRARY_PATH
   export AUTOCONF=autoconf
  '';
};

The Nix function pkgs.myEnvFun creates a program that can be run by the user to set up the environment such that the listed packages are available. This is done using symlinks and environment variables. The resulting shell can then be used for normal development. By creating special environments for development tasks it becomes possible to build with different versions of packages. For example, replace gcc with gcc46 and the environment will use that C compiler version. Environments for different versions of pango, gstreamer and other libraries can easily be created for testing Firefox builds with those specific versions.

The buildInputs field contains an array of the packages to be avaliable. These are all the pre-requisites as listed in the Mozilla build documentation. This could be modified by adding developer tools to be used (Vim, Emacs, Mercurial, etc) if desired.

When creating definitions that have a build product Nix will arrange the dynamic loader and paths to link to the correct versions of the libraries so that they can be found at runtime. When building an environment we need to change LD_LIBRARY_PATH to include the paths to the libraries for all the packages we are using. This is what the extraCmds section does. It is a shell script that is run to setup additional things for the environment.

The extraCmds in this definition adds to LD_LIBRARY_PATH the lib directory of all the packages in buildInputs. It exports an AUTOCONF environment variable to be the autoconf executable we are using. This variable is used in the Mozilla build system to find autoconf-2.13. It also adds to the C and C++ include path to find the DBus libraries which are in a nested dbus-1.0 directory.

To build and install this new package use nix-env:

$ nix-env -i env-firefoxEnv

Running the resulting load-env-firefoxEnv command will create a shell environment that can be used to build Firefox:

$ load-env-firefoxEnv
...
env-firefoxEnv loaded
$ git clone git://github.com/mozilla/gecko-dev
...
$ cd gecko-dev
$ ./mach build

Exiting the shell will remove access to the pre-requisite libraries and tools needed to build Firefox. This keeps your global user environment free and minimizes the chance of clashes.

Installing NixOS on a ThinkPad W540 with encrypted root

I recently got a ThinkPad W540 laptop and I'm trying out the NixOS Linux distribution:

NixOS is a GNU/Linux distribution that aims to improve the state of the art in system configuration management. In existing distributions, actions such as upgrades are dangerous: upgrading a package can cause other packages to break, upgrading an entire system is much less reliable than reinstalling from scratch, you can't safely test what the results of a configuration change will be, you cannot easily undo changes to the system, and so on.

I use the Nix package manager alongside other distributions and decided to try out the full operating system. This post outlines the steps I took to install NixOS with full disk encryption using LVM on LUKS.

Windows

The W540 comes with Windows 8.1 pre-installed and recovery partitions to enable rebuilding the system. I followed the install procedure to get Windows working and proceeding to make a recovery USB drive so I could get back to the starting state if things went wrong. Once this completed I went on with installing NixOS.

NixOS Live CD

I used the NixOS Graphical Live CD to install. I could have used the minimal CD but I went fo the graphical option to make sure the basic OS worked fine on the hardware. I installed the Live CD to a USB stick from another Linux machine using unetbootin.

To boot from this I had to change the W540 BIOS settings:

• Change the USB drive in the boot sequence so it was the first boot option.
• Disable Secure Boot.
• Change UEFI to be UEFI/Legacy Bios from the previous UEFI only setting.

Booting from the USB drive on the W540 worked fine and got me to a login prompt. Logging in with root and no password gives a root shell. Installation can proceed from there or the GUI can be started with start display-manager.

Networking

The installation process requires a connected network. I used a wireless network. This is configured in the Live CD using wpa_supplicant. This required editing /etc/wpa_supplicant.conf to contain the settings for the network I was connecting to. For a public nework it was something like:

network={
  ssid="My Network"
  key_mgmt=NONE
}

The wpa_supplicant service needs to be restarted after this:

# systemctl restart wpa_supplicant.service

It's important to get the syntax of the wpa_supplicant.conf file correct otherwise it will fail to restart with no visible error.

Partition the disk

Partitioning is done manually using gdisk. Three partitions are needed:

• A small partition to hold GPT information and provide a place for GRUB to store data. I made this 1MB in size and it must have a partition type of ef02. This was /dev/sda1.
• An unencrypted boot partition used to start the initial boot, and load the encrypted partition. I made this 1GB in size (which is on the large side for what it needs to be) and left it at the partition type 8300. This was /dev/sda2.
• The full disk encrypted partition. This was set to the size of the rest of the drive and partition type set to 8e00 for "Linux LVM". This was /dev/sda3.

Create encrypted partitions

Once the disk is partitioned above we need to encrypt the main root partition and use LVM to create logical partitions within it for swap and root:

# cryptsetup luksFormat /dev/sda3
# cryptsetup luksOpen /dev/sda3 enc-pv
# pvcreate /dev/mapper/enc-pv
# vgcreate vg /dev/mapper/enc-pv
# lvcreate -L 40G -n swap vg
# lvcreate -l 111591 -n root vg

The lvcreate commands create the logical partitions. The first is a 40GB swap drive. The laptop has 32GB of memory so I set this to be enough to store all of memory when hibernating plus extra. It could be made quite a bit smaller. The second creates the root partition. I use the -l switch there to set the exact number of extents for the size. I got this number by trying a -L with a larger size than the drive and used the number in the resulting error message.

Format partitions

The unencrypted boot partition is formatted with ext2 and the root partition with ext4:

# mkfs.ext2 -L boot /dev/sda2
# mkfs.ext4 -O dir_index -j -L root /dev/vg/root
# mkswap -L swap /dev/vg/swap

These should be mounted for the install process as follows:

# mount /dev/bg/root /mnt
# mkdir /mnt/boot
# mount /dev/sda2 /mnt/boot
# swapon /dev/vg/swap

Configure NixOS

NixOS uses a declarative language for the configuration file that is used to install and configure the operating system. An initial file ready to be edited should be created with:

$ nixos-generate-config --root /mnt

This creates the following files:

• /mnt/etc/nixos/configuration.nix
• /mnt/etc/nixos/hardware-configuration.nix

The latter file is rewritten everytime this command is run. The first file can be edited and is never rewritten. For the initial boot I had to make one change to hardware-configuration.nix. I commented out this line:

# services.xserver.videoDrivers = [ "nvidia" ];

I can re-add it later when configuring the X server if I want to use the nvidia driver.

The changes that need to be made to configuration.nix involve setting the GRUB partition, the Luks data and any additional packages to be installed. The Luks settings I added were:

boot.initrd.luks.devices = [
  { 
    name = "root"; device = "/dev/sda3"; preLVM = true;
  }
];

I changed the GRUB boot loader device to be:

boot.loader.grub.device = "/dev/sda";

To enable wireless I made sure I had:

networking.wireless.enable = true;

I added my preferred editor, vim, to the system packages:

environment.systemPackages = with pkgs; [
  vim
];

Enable OpenSSH:

services.openssh.enable = true;

I've left configuring X and other things for later.

Install NixOS

To install based on the configuration made above:

# nixos-install

If that completes successfully the system can be rebooted into the newly installed NixOS:

# reboot

You'll need to enter the encryption password that was created during cryptsetup when rebooting.

Completing installation

Once rebooted re-enable the network by performing the /etc/wpa_supplicant.conf steps done during the install.

Installation of additional packages can continue following the NixOS manual. This mostly involves adding or changing settings in /etc/nixos/configuration.nix and then running:

# nixos-rebuild switch

This is outlined in Changing the Configuration in the manual.

Troubleshooting

The most common errors I made were syntax errors in wpa_supplicant.conf and configuration.nix. The other issue I had was not creating the initial GPT partition. GRUB will give an error in this case explaining the issue. You can reboot the Live USB drive at any time and mount the encrypted drives to edit files if needed. The commands to mount the drives are:

# cryptsetup luksOpen /dev/sda3 enc-pv
# vgscan --mknodes
# vgchange -ay
# mount /dev/bg/root /mnt
# mkdir /mnt/boot
# mount /dev/sda2 /mnt/boot
# swapon /dev/vg/swap

Tips

environment.systemPackages in /etc/configuration.nix is where you add packages that are seen by all users. When this is changed you need to run the following for it to take effect:

# nixos-rebuild switch

To find the package name to use, run something like (for vim):

$ nix-env -qaP '*'|grep vim

A user can add their own packages using:

$ nix-env -i vim

And remove with:

$ nix-env -e vim

A useful GUI for connecting to wireless networks is wpa_gui. To enable this add wpa_supplicant_gui to environment.systemPackages in /etc/nixos/configuration.nix followed by a nixos-rebuild switch. Add the following line to /etc/wpa_supplicant.conf:

ctrl_interface=/var/run/wpa_supplicant

Restart wpa_supplicant and run the gui:

$ systemctl restart wpa_supplicant.service
$ sudo wpa_gui

It's possible to make custom changes to Nix packages for each user. This is controlled by adding definitions to ~/.nixpkgs/config.nix. The following config.nix will provide Firefox with the official branding:

{
  packageOverrides = pkgs : with pkgs; rec {
    firefoxPkgs = pkgs.firefoxPkgs.override { enableOfficialBranding = true; };
  };
}

Installing or re-installing for the user will use this version of Firefox:

$ nix-env -i firefox