Bluish Coder: zeronet

ZeroMe - Decentralized Microblogging on ZeroNet

2017-10-12T21:00:00+13:00

I wrote about ZeroNet a few years ago when it was released and it mostly was about decentralized websites. In the time between then and now it has gained a lot of features and regular use. It still does distributed websites well but it adds features suchs as support for sharing large files, merging sites and distributed pseudo-anonymous identity. This post is about an application hosted on ZeroNet called ZeroMe. It's a microblogging platform (i.e. A twitter like system) that takes advantage of ZeroNet's identity system and merging of sites.

Starting ZeroNet

To start with ZeroMe you first need to install ZeroNet. Given all the right dependancies are installed you can clone the github repository and run the Python script to get online:

$ git clone https://github.com/HelloZeroNet/ZeroNet
$ cd ZeroNet
$ ./zeronet.py

This will start the ZeroNet node and run a local webserver on port 43110 to access it. This port is available on the local machine only. It also opens a port on 15441 to communicate with other nodes. This port is open to the internet at large and will attempt to punch a hole through firewalls using UPNP.

If you have the Tor daemon running then ZeroNet will connect to it and bridge between clearnet nodes and nodes running on Tor. You can force the node to run only on Tor by providing a command line argument:

$ ./zeronet.py --tor always

In this configuration ZeroNet will not expose the port 15441 over the internet. It will spawn Tor onion services and expose the port to those for communication with other ZeroNet nodes running on onion services. Access to the local node is still done via port 43110 but you should use a browser configured to use Tor if anonymity is a concern since sites can run arbitary JavaScript. The main ZeroNet 'Hello' page warns you if you are not using Tor Browser in this configuration.

Gaining an identity

ZeroMe requires an identity to be created. Identities are created through identity ZeroNet sites. The main one, operated by the creator of ZeroNet, is ZeroId. When you first use a site that requires an identity you will be prompted to create one from any identity providers you have access to. For this example I'll use ZeroId, accessible at zeroid.bit - http://localhost:43110/zeroid.bit.

Click Get auth cert and choose a username. There are two ways to make the request for the identity. The first is a request over the internet via HTTP. If you're not running the browser over Tor this will expose your IP address to the identity service. The other is using BitMessage, which is an anonymous messaging system. ZeroId will request that you send a confirmation message to a BitMessage address.

This may seem very centralized - it requires the identity service to be running, if it goes down you can't create a new identity. There are other identity services on ZeroNet that have different requirements for creating an identity. And there's an API to create your own. Identities created using these work on any service using the ZeroNet identity API in the same was as ZeroId identities.

The ZeroId site will show your identity name once you've signed up:

From ZeroId you can search for existing users and choose to Mute or Unmute. By muting a particular identity you won't see any posts from that identity. This is useful for dealing with spam or users posting on topics you don't want to see. This will prevent you seeing their content on any ZeroNet site.

You can manage muted users from the Hello page left sidebar menu:

Joining a ZeroMe Hub

Access ZeroMe by activating the ZeroMe icon on the main user interface or going directly to Me.ZeroNetwork.bit - http://localhost:43110/Me.ZeroNetwork.bit/

A prompt on the top right asks you to accept a merger site. Merge sites are a way to split large sites into a number of smaller ones. You should approve this request.

Click on "Select user to post new content" to create a ZeroMe user. It will prompt for an identity to choose and you can select the one already created in ZeroId above.

Once an identity is chosen you'll be asked to select a 'hub'. ZeroMe works by having the user interface operate from the main ZeroMe site with data on users and posts stored in 'Hub' sites. You only see posts from users that belong to hubs that you have requested. There is a hub operated by the ZeroMe creator and it will be shown as Sun hub on this page along with any other Hub sites you may already have requested. Click Download on a hub to download existing posts and content for that hub and Join to join one.

Downloading a hub can take some time. On the Hello ZeroNet page you can see the progress of any sites currently being downloaded.

There's a useful site called 0hub.bit that lists known hubs and lets you download them all. It's at http://localhost:43110/0hub.bit/ and clicking the Click me to unlock all hub post in ZeroNet will start downloading them. This takes some time but is worth it since you can then see all posts by everyone on ZeroMe. You can delete hubs if you decide you don't want to see content from a particular hub.

Microblogging with ZeroMe

Now that you've got an identity, signed into ZeroNet and joined a hub you can starting reading content and posting. The Everyone tab in the user interface will show posts from all users, even those you haven't followed. This is useful at the beginning to find users and content.

Posts are in Markdown format and can include images. There is no limit to the size of a post and posts can have comments. You can visit profile pages of users and choose to optionally seed their images, mute users, and edit parts of your own profile page to have an avatar and description:

An example of a post with comments (selected randomly, identities blacked out):

Random stuff

The ZeroNet node running on your machine enforces size limits for sites. This prevents sites that you have accessed from consuming all your disk space. When a site starts reaching the size limit it appears in the main Hello page left sidebar in a special section:

When the site goes past the limit it will no longer download updates. Visiting the site will give a popup asking to increase the limit. Approving that popup will resume downloads for the site until it hits the next limit.

There's a sidebar on the right of pages that can be accessed by dragging the top right 0 icon to the left:

This sidebar gives statistics on the number of nodes seeding the site, the size limit, and various other things.

When you visit a ZeroNet site you starting seeding that sites content and receive automatic updates when the content changes. This can be disabled on the main Hello page on a per site basis. Sites can have "optional files" which you don't automatically seed. They are downloaded on demand and are often used for user generated content or larger files. You can choose to seed a sites optional files in the right hand sidebar for that site. There are also "Big Files" which are treated specially. These are large files like videos and are also optionally seeded. The Files tab of the Hello page lists optional and big files that you are seeding:

Sites can be set to allow cloning by end users. This creates a copy of the site with no content. An example of a site that does this is ZeroBlog which you can clone to create your own blog. This extends to ZeroMet itself. You can clone the ZeroMe user interface site and modify that to get a customized look but it still uses the data from the existing hubs.

There's a bunch of other interesting sites on ZeroNet. The ZeroTalk forums, various blogs, ZeroMail for encrypted email like service, etc. Be aware that the psuedo-anonymous use of identities can make for content you might not agree with and much spam. Use of 'Mute' is useful here.

Also be aware that it is 'psuedo-anonymous' not 'anonymous'. You create an identity and that identity is not tied to you in the real world but people can track what you do to that identity. Content is added to sites and distributed to other nodes. If you like a post or add content to some site then anyone who decides to dig into the data of that site can see that your identity liked or posted that content. It is possible to have multiple identities if you want to keep aspects of your ZeroNet usage separate but that's a topic for another post.

Overall ZeroMe is a nice microblogging system. It's user friendly, has a nice design and has tools for muting and "Non Real Name" identities. It, along with ZeroNet, is actively developed and supported by the ZeroNet developer.

Decentralized Websites with ZeroNet

2015-01-15T21:00:00+13:00

ZeroNet is a new project that aims to deliver a decentralized web. It uses a combination of bittorrent, a custom file server and a web based user interface to do this and manages to provide a pretty useable experience.

Users run a ZeroNet node and do their web browsing via the local proxy it provides. Website addresses are public keys, generated using the same algorithm as used for bitcoin addresses. A request for a website key results in the node looking in the bittorrent network for peers that are seeding the site. Peers are selected and ZeroNet connects to the peer directly to a custom file server that it implements. This is used to download the files required for the site. Bittorrent is only used for selecting peers, not for the site contents.

Once a site is retrieved the node then starts acting as a peer serving the sites content to users. The more users browsing your site, the more peers become available to provide the data. If the original site goes down the remaining peers can still serve the content.

Site updates are done by the owner making changes and then signing these changes with the private key for the site address. It then starts getting distributed to the peers that are seeding it.

Browsing is done through a standard web browser. The interface uses Websockets to communicate with the local node and receive real time information about site updates. The interface uses a sandboxed iframe to display websites.

Running

ZeroNet is open source and hosted on github. Everything is done through the one zeronet.py command. To run a node:

$ python zeronet.py
...output...

This will start the node and the file server. A check is made to see if the file server is available for connections externally. If this fails it displays a warning but the system still works. You won't seed sites or get real time notification of site updates however. The fix for this is to open port 15441 in your firewall. ZeroNet can use UPNP to do this automatically but it requires a MiniUPNP binary for this to work. See the --upnpc command line switch for details.

The node can be accessed from a web browser locally using port 43110. Providing a site address as the path will access a particular ZeroNet site. For example, 1EU1tbG9oC1A8jz2ouVwGZyQ5asrNsE4Vr is the main 'hello' site that is first displayed. To access it you'd use the URL http://127.0.0.1:43110/1EU1tbG9oC1A8jz2ouVwGZyQ5asrNsE4Vr.

Creating a site

To create a site you first need to shut down your running node (using ctrl+c will do it) then run the siteCreate command:

$ python zeronet.py siteCreate
...
- Site private key: ...private key...
- Site address: ...site address...
...
- Site created!

You should record the private key and address as you will need them when updating the site. The command results in a data/address directory being created, where 'address' is the site address that siteCreate produced. Inside that is a couple of default files. One of these, content.json, contains JSON data listing the files contained within the site and signing information. This gets updated automatically when you sign your site after doing updates. If you edit the title key in this file you can give your site a title that appears in the user interface instead of the address.

Another flie that gets modified during this site creation process is the sites.json file in the data directory. It contains the list of all the sites and some metadata about them.

If you visit http://127.0.0.1:43110/siteaddress in your browser, where siteaddress is the address created with siteCreate, then you'll see the default website that is created. If your node is peering successfully and you access this address from another node it will download the site, display it, and start seeding it. This is how the site data spreads through the network.

Updating a site

To change a site you must first store your files in the data/siteaddress directory. Any HTML, CSS, JavaScript, etc can be put here. It's like a standard website root directory. Just don't delete the config.json file that's there. Once you've added, modified or removed files you run the siteSign command:

$ python zeronet.py siteSign siteaddress
- Signing site: siteaddress...
Private key (input hidden):

Now you enter the private key that was displayed (and hopefully you saved) when you ran siteCreate. The site gets signed and information stored in config.json. To publish these changes to peers seeding the site:

$ python zeronet.py sitePublish siteaddress
...publishes to peers...

If your node is running it will serve the files from the running instance. If it is not then the sitePublish command will continue running to serve the files.

Deleting a site

You can pause seeding a site from the user interface but you can't delete it. To do that you must shutdown the node and delete the sites data/siteaddress directory manually. You will also need to remove its entry from data/sites.json. When you restart the node it will no longer appear.

Site tips

Because the website is displayed in a sandboxed iframe there are some restrictions in what it can do. The most obvious is that only relative URLs work in anchor elements. If you click on an absolute URL it does nothing. The sandboxed iframe has the allow-top-navigation option which means you can link to external pages or other ZeroNet sites if you use the target attribute of the anchor element and set it to _top. So this will work:

<a href="http://bluishcoder.co.nz/" target="_top">click me</a>

But this will not:

<a href="http://bluishcoder.co.nz/">click me</a>

Dynamic websites are supported, but requires help using centralized services. The ZeroNet node includes an example of a dynamic website called 'ZeroBoard'. This site allows users to enter a message in a form and it's published to a list of messages which all peering nodes will see. It does this by posting the message to an external web application that the author runs on the standard internet. This web app updates a file inside the sites ZeroNet directory and then signs it. The result is published to all peers and they automatically get the update through the Websocket interface.

Although this works it's unfortunate the it relies on a centralized web application. The ZeroNet author has posted that they are looking at decentralized ways of doing this, maybe using bitmessage or some other system. Something involving peer to peer WebRTC would be interesting.

Conclusion

ZeroNet seems to be most similar to tor, i2p or freenet. Compared to these it lacks the anonymity and encryption aspects. But it decentralizes the site content which tor and i2p don't. Freenet provides decentralization too but does not allow JavaScript in sites. ZeroNet does allow JavaScript but this has the usual security and tracking concerns.

Site addresses are in the same format as bitcoin addresses. It should be possible to import the private key into bitcoin and then bitcoins sent to the public address of a site would be accessed by the site owner. I haven't tested this but I don't see why it couldn't be made to work. Maybe this could be leveraged somehow to enable a web payment method.

ZeroNet's lack of encyption or obfuscation of the site contents could be a problem. A peer holds the entire site in a local directory. If this contains malicious or illegal content it can be accidentally run or viewed. Or it could be picked up in automated scans and the user held responsible. Even if the site originally had harmless content the site author could push an update out that contains problematic material. That's a bit scary.

It's early days for the project and hopefully some of these issues can be addressed. As it is though it works well, is very useable, and is an interesting experiement on decentralizing websites. Some links for more information: