Installing Java

The main turn off with Freenet for me is that it’s written in Java. Once you’ve installed it you’ll see for yourself how CPU-intensive it actually is – much more than Tor (which on the opposite isn’t too demanding). Anyway, installing Java is now a bit trickier than before, because Oracle got their reference implementation removed from official distribution repositories, but OpenJDK works just as well so that’s what we’ll pick.
apt-get install openjdk-6-jdk
(that will be about 170 MiB of stuff to install)

Installing Freenet

Even though the headless server instructions there are a bit scary, I found them quite sufficient. It’s not that complicated to install after all (also, maybe after getting warmed up by Tor’s installation I’m a little biased, too ), so I’ll just list the commands you need to type (obviously, edit the pathes), and they should be pretty self-explanatory:
cd /where/you/want/to/install/it
mkdir someFolder
cd someFolder
wget https://downloads.freenetproject.org/alpha/installer/freenet07.tar.gz --no-check-certificate
tar xzf freenet07.tar.gz
Now we need to create a user for Freenet, give it a password and assign them the freenet folder:
useradd someUserName
passwd someUserName
[enter the password twice]
chown someUserName freenet -R

First Freenet launch, configuration

Connect to the machine where you just installed Freenet as this someUserName, go to the freenet folder and run run.sh:
cd /where/you/want/to/install/it/someFolder/freenet
./run.sh
FYI (actually, more for my personal archives ^^), here’s the output:

Enabling the auto-update feature
Detecting tcp-ports availability...
Downloading update.sh
Downloading wrapper_Linux.zip
Downloading freenet-stable-latest.jar
Downloading freenet-ext.jar
Downloading the JSTUN plugin
Downloading the UPnP plugin
Downloading seednodes.fref
Installing cron job to start Freenet on reboot...
Installed cron job.
Starting Freenet 0.7...
Please visit http://127.0.0.1:8888/ to configure your node
Finished

Now, you need to connect to http://127.0.0.1:8888 to configure Freenet. If, as is likely since you’re reading this tutorial on console-based installation, it’s not on the same computer as where you are now, you have 2 options:
- follow this to allow external IPs (preferably, just a specific one: yours ) to connect to your Freenet node
- create an SSH tunnel to your machine (see the second half of this old post), and then when your browser is configured to use it (NB: don’t forget to remove “127.0.0.1″ from the exclusion list, in Firefox called “no proxy for”), going to http://127.0.0.1:8888 will actually go to your server.

Well, that’s pretty much it, now just configure it from the GUI as you wish. Here are a few hints, though:

• You should increase the RAM available to Freenet. By default it’s 128, it would be a good idea to at least double it. Actually 512 MiB would be even nicer. Note that you’ll have to restart Freenet in order for the change to be applied. The RAM can also be configured manually in wrapper.conf:
  wrapper.java.initmemory=60
wrapper.java.maxmemory=512
• You’ll probably want to review input and output bandwidth settings (although you already had the opportunity to do so during the first launch wizard)
• You’ll want to increase the datastore size (this helps the network) and the client cache (this helps you ), because during the first launch wizard you aren’t really given much choice for some reason… Respectively 50GiB and 25GiB could be fine for instance, although of course you can increase that much more. Note however that whenever you change those settings Freenet will rebuild its database, which means it will be somewhat slower for a while.
• Finally, it’s probably a good idea to install some social plugins such as WebOfTrust and Freetalk (both are official plugins), which will give you access to discussion boards. Other plugins of interest: Sone (kind of a social network), Library (search engine), Freenet Search (search engine too).

Bonus: solving the “There isn’t enough entropy available on your system… Freenet won’t start until it can gather enough.” error

Freenet seems to have massive needs for entropy, and if you happen to run out of it, it might take some time to spontaneously replenish. A quick way to boost this is to generate lots of disk access. I saw the suggestion of running find / >/dev/null, but it didn’t help a lot for me, maybe because my HD isn’t that filled. What worked great was to either copy a big directory, or even simpler, to download some large file, like: wget ftp://ftp.free.fr/mirrors/ftp.ubuntu.com/dvd/current/precise-dvd-amd64.iso

Bonus 2: restarting/stopping the node without using the web GUI

The ./run.sh script can be run with arguments, such as ./run.sh restart or ./run.sh stop

Dealing with package manager stuff

The first step is to configure the package manager to use Tor’s repository, because distribution repositories are, as you know, always outdated. On this part, the Tor project documentation (for Ubuntu and Debian) is good enough (NB: here is the page for Fedora / CentOS / SUSE). Basically:
nano /etc/apt/sources.list
and at the bottom, add (for Ubuntu 11.10):
deb http://deb.torproject.org/torproject.org oneiric main
You can now proceed to installation:
apt-get install tor tor-geoipdb

Editing the configuration file (torrc)

Installing Tor this way also starts it, but the default configuration is not to relay any traffic. We now need to configure is as a relay. The configuration file to edit is usually, as mentioned there, /etc/tor/torrc:
nano /etc/tor/torrc
In this file, the settings you’ll want to look at are:

• uncomment Log notice file /var/log/tor/notices.log if you want to avoid a notice in ARM (see below for what ARM is)
• uncomment ControlPort 9051 so that ARM can be used to control and monitor Tor
• uncomment ORPort and (optionally but recommended, I guess) change its value to some random available port. That’s the port that will be used by other nodes to connect to yours
• optionally uncomment Nickname and give a name to your node
• set RelayBandwidthRate and RelayBandwidthBurst. Those are the maximum bandwidth that can be used by Tor constantly and in burst, respectively. For instance:

  RelayBandwidthRate 5000 KB  # Throttle traffic to 100KB/s (800Kbps)
  RelayBandwidthBurst 7000 KB # But allow bursts up to 200KB/s (1600Kbps)

  Beware those values are in Bytes, not bits.

• if you have a traffic quota (and don’t want to blow it), you’ll want to configure AccountingMax and AccountingStart
• optionally, you can put your name and (spam protected) e-mail, or even your GPG fingerprint, in ContactInfo. According to the comments it can be used to contact you in case your node goes wild or something.
• optionally, you can also configure DirPort to mirror directory information on this port.
• finally, the most important part: choose whether you want to be just a relay or an exit node. To be just a relay, set:
  ExitPolicy reject *:*
  To be an exit node, you can either leave at is, or replace the default exit policy (in order to do the later, do read the instructions carefully). Note that being an exit can bring you trouble because there’s always a risk of abuse, so at least do check your country’s law, the law of your host’s country, and your host’s TOS. But I guess you’re already aware of that anyway.

Additionally, if you want to remotely use your node to enter the Tor network to surf yourself, well, I suppose it can be configured using SocksPort, SocksListenAddress, SocksPolicy accept and SocksPolicy reject, but I haven’t figured out how to do that yet :/

ARM: a terminal-friendly GUI

Last step, let’s install a nice piece of helper software called ARM (for “anonymizing relay monitor”), which is basically a nice console-based GUI to monitor and have some control over Tor. On Ubuntu 11.10 it’s as simple as:
apt-get install tor-arm

In order to use ARM, you’ll have to enable Tor control port first. You already entered it in the configuration file (if you followed the guide properly, at least), so we just need to apply the configuration:
/etc/init.d/tor reload

Now you can just type arm to launch ARM and view some neat information about your node, like the traffic graph, inbound and outbound nodes, etc.

Bonus: what’s a stable node, what’s a guard node?

A little bonus, because that’s something that puzzled me for a while: on some Tor node listings, you can see that servers have flags, notably some servers are flagged as “guard” and some are flagged as “stable”. If you’re curious about what it means, here it is, right from the specification:

"Guard" -- A router is a possible 'Guard' if its Weighted Fractional
Uptime is at least the median for "familiar" active routers, and if
its bandwidth is at least median or at least 250KB/s.

	To calculate weighted fractional uptime, compute the fraction
	of time that the router is up in any given day, weighting so that
	downtime and uptime in the past counts less.

	A node is 'familiar' if 1/8 of all active nodes have appeared more
	recently than it, OR it has been around for a few weeks.

"Stable" -- A router is 'Stable' if it is active, and either its Weighted
MTBF is at least the median for known active routers or its Weighted MTBF
corresponds to at least 7 days. Routers are never called Stable if they are
running a version of Tor known to drop circuits stupidly.  (0.1.1.10-alpha
through 0.1.1.16-rc are stupid this way.)

	To calculate weighted MTBF, compute the weighted mean of the lengths
	of all intervals when the router was observed to be up, weighting
	intervals by $\alpha^n$, where $n$ is the amount of time that has
	passed since the interval ended, and $\alpha$ is chosen so that
	measurements over approximately one month old no longer influence the
	weighted MTBF much.

	[XXXX what happens when we have less than 4 days of MTBF info.]

Sources (other than those already mentioned in the text):

• Tor – Community Ubuntu Documentation
• Ubuntu howto for CLI install of Tor relay?
• Torstatus: What does stable server mean?

The steps are actually fairly simple:

• find out the name of your LVM volume(s) and group(s): pvs
• let’s say you have a group named “vg00″ with 50 GiB free space, and we want to create a volume named “espace1″ which would take all this space. First we check what volumes are already in that group: lvdisplay /dev/vg00
  Then we create the “espace1″ volume: lvcreate -L50G -nespace1 vg00
• now we format it, say, in ext3: mkfs -t ext3 -m 0 -v /dev/vg00/espace1
  Note that the “-m 0″ means that we claim back the 5% (“-m 5″) space usually reserved for root. This is good for a storage partition, but not a good idea for a system partition.
• then we create the mounting point, for instance: mkdir /home/espace1
• and all we have to do now is to mount it: mount -t ext3 /dev/vg00/espace1 /home/espace1

That’s it, your new partition is ready. You can also check out the results using df -h. You’ll probably want to make sure it’s automatically mounted at boot time, too: for this just add a line in /etc/fstab (nano /etc/fstab). For instance here’s our fstab file (note the last line with our new partition):

/dev/xvda1      /               ext3    defaults        1 1
/dev/xvda2      none            swap    sw
/dev/vg00/usr   /usr            xfs     defaults        0 2
/dev/vg00/var   /var            xfs     defaults        0 2
/dev/vg00/home  /home           xfs     defaults,usrquota       0 2
devpts          /dev/pts        devpts  gid=5,mode=620  0 0
none            /proc           proc    defaults        0 0
none            /tmp    tmpfs   defaults        0 0
/dev/vg00/espace1  /home/espace1    ext3   defaults     0 2

Finally, to destroy it, you just need to unmount it first, so:
umount /dev/vg00/espace1 => unmount
lvremove /dev/vg00/espace1 => delete the volume

This time that’s all, the circle of life of the volume is complete Here are the sources I used, in the last one you’ll also find how to create the whole LVM thing, not just a volume within it like we did here:

• Ubuntux – Edit fstab to mount partition at startup
• LVM How-to – 11.7. Creating a logical volume
• Walker News – How To Create Linux LVM In 3 Minutes

127.0.0.1 – - [27/Jul/2011:09:55:22 +0200] “OPTIONS * HTTP/1.0″ 200 136 “-” “Apache (internal dummy connection)”
127.0.0.1 – - [27/Jul/2011:09:55:23 +0200] “OPTIONS * HTTP/1.0″ 200 136 “-” “Apache (internal dummy connection)”
127.0.0.1 – - [27/Jul/2011:09:55:24 +0200] “OPTIONS * HTTP/1.0″ 200 136 “-” “Apache (internal dummy connection)”
127.0.0.1 – - [27/Jul/2011:09:55:52 +0200] “OPTIONS * HTTP/1.0″ 200 136 “-” “Apache (internal dummy connection)”
127.0.0.1 – - [27/Jul/2011:09:55:54 +0200] “OPTIONS * HTTP/1.0″ 200 136 “-” “Apache (internal dummy connection)”
127.0.0.1 – - [27/Jul/2011:09:55:55 +0200] “OPTIONS * HTTP/1.0″ 200 136 “-” “Apache (internal dummy connection)”
127.0.0.1 – - [27/Jul/2011:09:55:56 +0200] “OPTIONS * HTTP/1.0″ 200 136 “-” “Apache (internal dummy connection)”
127.0.0.1 – - [27/Jul/2011:09:55:58 +0200] “OPTIONS * HTTP/1.0″ 200 136 “-” “Apache (internal dummy connection)”
127.0.0.1 – - [27/Jul/2011:09:56:05 +0200] “OPTIONS * HTTP/1.0″ 200 136 “-” “Apache (internal dummy connection)”
127.0.0.1 – - [27/Jul/2011:09:56:06 +0200] “OPTIONS * HTTP/1.0″ 200 136 “-” “Apache (internal dummy connection)”

I remember we already had issues with these internal dummy things on the previous server, but because software there was getting old our response to the problem was just, well, to move to the new server with all updated software and stuff. This time I decided to try to deal with the problem for real.

After some quick searches, I found out that internal dummy connections are used to wake up child processes. I’m not sure how all this works, but the key point here is that in the end, sometimes those calls may hit some of your “real” pages, and may also corrupt their cached version. Long story short, the request may end up more CPU-demanding than it should. That problem seems to have been solved in Apache 2.2.6, but although we are running a much later version I thought it wouldn’t hurt to still apply the work-around. Basically, all you need to do is to make sure that the dummy connections get a 403 error. So, just 2 lines in your configuration files (NB: it requires mod_rewrite to be enabled) (source):

RewriteCond %{HTTP_USER_AGENT} ^.*internal\ dummy\ connection.*$ [NC]
RewriteRule .* – [F,L]

And voilà, internal dummy connections should now be processed effortlessly. Don’t forget to apply configuration changes.

References (if you need more details you’ll find quite a few there):

• ASF Bugzilla – Bug 41796 – Internal Dummy Connection should process effortlessly
• Inventing Labs – Apache’s ‘internal dummy connection’
• Httpd Wiki – InternalDummyConnection
• Racker Hacker – Apache 2.2: internal dummy connection

Linux s15429750 2.6.32-25-generic-pae #45-Ubuntu SMP Sat Oct 16 21:01:33 UTC 2010 i686 GNU/Linux
Ubuntu 10.04.2 LTS

Welcome to Ubuntu!
* Documentation: https://help.ubuntu.com/

System information as of Sat Jun 11 09:54:52 CEST 2011

System load: 0.86 Processes: 109
Usage of /home: 84.6% of 12.99GB Users logged in: 1
Memory usage: 42% IP address for eth0: 82.165.148.117
Swap usage: 1%

Graph this data and manage this system at https://landscape.canonical.com/

6 packages can be updated.
3 updates are security updates.

I eventually found out that this is due to some junk (some copy of the welcome message) making its way into /etc/motd.tail, which is shown at the end of the “real” welcome message. So just empty this file (or replace it with whatever content you wish) to get rid of the double message.

On a side note, you can modify what’s output by motd by editing the scripts in /etc/update-motd.d. A dirty way to disable it is to set those scripts as non-executable: chmod -x * but maybe add back the header to display kernel version: chmod +x 00-header

Sources:

• Bug #660470 in Ubuntu: “Double Welcome Screen since Upgrade from 10.04 to 10.10 server”
• [ubuntu] Server 10.04: How to Eradicate the MOTD System? [Archive] – Ubuntu Forums

As I mentioned previously (twice already), when you rent a cloud server at 1and1, and you install Linux on it (well, at least Ubuntu), you obtain a weird partitioning scheme. Such as:

# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/xvda1            3.7G  1.2G  2.6G  31% /
none                  490M  140K  490M   1% /dev
none                  500M  8.0K  500M   1% /dev/shm
none                  500M     0  500M   0% /tmp
none                  500M   60K  500M   1% /var/run
none                  500M     0  500M   0% /var/lock
none                  500M     0  500M   0% /lib/init/rw
/dev/mapper/vg00-usr  4.0G  441M  3.6G  11% /usr
/dev/mapper/vg00-var  4.0G  215M  3.8G   6% /var
/dev/mapper/vg00-home 4.0G  4.3M  4.0G   1% /home

So, in total much less than the 100 GB you get in the smallest case. I tried to follow their (tiny) FAQ on the subject, but seriously this wasn’t worth going farther than:

# fdisk -l

Disk /dev/xvda: 100.0 GB, 100000595968 bytes
255 heads, 63 sectors/track, 12157 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

    Device Boot      Start         End      Blocks   Id  System
/dev/xvda1               1         487     3911796   83  Linux
/dev/xvda2             488         731     1959930   82  Linux swap / Solaris
/dev/xvda3             732       12157    91779345   8e  Linux LVM

(I’ll be trying to provide full output as I guess that’s what’s most helpful if you’re struggling to do just what I wanted to do, i.e. just get the GBs you paid for)
I just tried to mount that big /dev/xvda3 thing, but it failed:

# cd mnt
# ls
# mkdir part1
# mount /dev/xvda3 /mnt/part1
mount: unknown filesystem type 'LVM2_member'

But this error was not in vain, for searching for it brought me the solution: that thing is an LVM volume, and there’s a particular way to deal with it.
First you can (should!) check out your LVM volumes *and groups*:

# pvs
  PV         VG   Fmt  Attr PSize  PFree
  /dev/xvda3 vg00 lvm2 a-   87.52g 75.52g

Here it’s simple, we have one volume, /dev/xvda3, which contains one group, vg00, which has a size of 87.52GiB (among which 75.52GiB unassigned aka free). Looks familiar? You bet, remember earlier:

/dev/mapper/vg00-usr  4.0G  441M  3.6G  11% /usr
/dev/mapper/vg00-var  4.0G  215M  3.8G   6% /var
/dev/mapper/vg00-home 4.0G  4.3M  4.0G   1% /home

Now you’ll want to check out what’s inside that group:

# lvdisplay /dev/vg00
  --- Logical volume ---
  LV Name                /dev/vg00/usr
  VG Name                vg00
  LV UUID                ElBhAu-r2PF-g1sV-UICK-T7Ao-TZmC-uKv0sj
  LV Write Access        read/write
  LV Status              available
  # open                 1
  LV Size                4.00 GiB
  Current LE             1024
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           251:0

  --- Logical volume ---
  LV Name                /dev/vg00/var
  VG Name                vg00
  LV UUID                gz0jhl-7uzG-1wqj-b1Ok-Mqjk-rWlC-ZHBBhy
  LV Write Access        read/write
  LV Status              available
  # open                 1
  LV Size                4.00 GiB
  Current LE             1024
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           251:1

  --- Logical volume ---
  LV Name                /dev/vg00/home
  VG Name                vg00
  LV UUID                2C9KvC-Dcmx-4C6N-T7cg-NzAr-WVOi-KS2xc1
  LV Write Access        read/write
  LV Status              available
  # open                 1
  LV Size                4.00 GiB
  Current LE             1024
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           251:2

It’s like the jackpot, because from all the posts I read about LVM (see sources at the end), it seems that the hard part with those LVM volumes is to mount them. Here (on that 1and1 cloud server) we’re totally in luck: they’re already mounted! We just need to find out how to resize them, and this looks fairly easy: lvextend and lvreduce… Now, before going further, I’d like to warn you about something: extending is most likely totally harmless (and super-fast), reducing is probably more risky, or at least more tedious (you need to unmount the logical volume to do it “safely”, and I’m not sure how safe this is).
So the winning strategy, in my opinion, is maybe to try extending an LVM with 1GiB just to see if it works fine, and then wait until you have a better idea of which volume needs space before extending them again (maybe also wait till that volume gets low on free space – but don’t wait too much, it would be stupid to lose data because of lack of assigned space while there’s so much free ^^). Avoid extending the volumes alternatively and regularly to avoid fragmenting them. Avoid assigning too much too fast to avoid having to reduce a volume later (you’ll see below how scary it sounds when you try it ).
So, here are the 2 commands and their outputs:

# lvextend -L+1G /dev/vg00/home
  Extending logical volume home to 5.00 GiB
  Logical volume home successfully resized

# lvreduce -L-1G /dev/vg00/home
  WARNING: Reducing active and open logical volume to 4.00 GiB
  THIS MAY DESTROY YOUR DATA (filesystem etc.)
Do you really want to reduce home? [y/n]: n
  Logical volume home NOT reduced
  Command failed with status code 5.

(yeah, I aborted that one ^^)

Now that you’ve extended the volume, you’ll have to extend the filesystem first. Something you should know before going on if you’re doing this on 1and1 like me: their cloud servers use XFS, and you can’t shrink an XFS file system so once you’ve expanded it there’s no easy way back. Here we go, for instance for /home (the LVM volume needs to be mounted):

# xfs_growfs /home
meta-data=/dev/mapper/vg00-home  isize=256    agcount=8, agsize=131072 blks
         =                       sectsz=512   attr=0
data     =                       bsize=4096   blocks=1048576, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0
log      =internal               bsize=4096   blocks=2560, version=1
         =                       sectsz=512   sunit=0 blks, lazy-count=0
realtime =none                   extsz=65536  blocks=0, rtextents=0
data blocks changed from 1048576 to 1310720

One last thing: let’s check how the free space lost the GiB we assigned:

# pvs
  PV         VG   Fmt  Attr PSize  PFree
  /dev/xvda3 vg00 lvm2 a-   87.52g 74.52g

And… back to the beginning:

# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/xvda1            3.7G  1.1G  2.7G  30% /
none                  490M  140K  490M   1% /dev
none                  500M     0  500M   0% /dev/shm
none                  500M     0  500M   0% /tmp
none                  500M   60K  500M   1% /var/run
none                  500M     0  500M   0% /var/lock
none                  500M     0  500M   0% /lib/init/rw
/dev/mapper/vg00-usr  4.0G  441M  3.6G  11% /usr
/dev/mapper/vg00-var  4.0G  216M  3.8G   6% /var
/dev/mapper/vg00-home 5.0G  4.4M  5.0G   1% /home

Success!

Sources (I didn’t really use all of them, but they might be useful to dig deeper – I tried to put the most relevant ones at the top):

• LinuxQuestions – mount: unknown filesystem type ‘LVM2_member’
• Mounting a Linux LVM volume
• LVM How-to: Reducing a logical volume
• LVM How-to: Extending a logical volume
• Resizing an LVM partition, True Blade Systems, Inc.
• Accessing a Fedora Logical Volume from Ubuntu
• PissedOffAdmins – mount: unknown filesystem type ‘LVM2_member’
• How to mount LVM volumes, help!
• LVM and RAID

Update (15 Jan 2011): checking the queries bringing people here and reproducing them in Google, I was able to find a FAQ in the UK FAQ just about that… I really wonder why the heck the 1&1 FAQ is so totally different from one country to another! Anyway, here’s the link: 1&1 Webhosting FAQ | How can I increase a logical volume?