Clockfort's Tech Blog

There is like, zero documentation for doing this on the Internet. No one else wants to install Gentoo in a VServer guest on a Debian Itanium2 host? Lame.

This quick overview contains many things specific to my personal setup; therefore:

• You will want to check your local gentoo mirror to see what the current ia64 tarball is.
• You will want to change the name, hostname, and network address of your vserver
• You can use whatever directories you want; the actual files will go in /var/lib/vservers/<hostname>/ , (and /etc/vservers/<hostname>/ ) and you can delete the stage3 tarball after you’re done.
• So you don’t get confused, “jolt” is the name of my debian host machine, and “coffee” is the vserver guest I am creating.

#Install the Vserver kernel and utilities if you already have not.

jolt:/space/vserver# aptitude install linux-image-vserver-mckinley util-vserver vserver-debiantools

#Reboot to boot into new kernel.

jolt:/space/vserver# reboot

#Are you running the new kernel? It should say “vserver” in there somewhere.

jolt:/space/vserver# uname -a
Linux jolt 2.6.26-2-vserver-mckinley #1 SMP Thu Nov 5 07:44:36 UTC 2009 ia64 GNU/Linux

#Get the latest Stage3 tarball from a mirror close to you.

jolt:/space/vserver/base-images/# wget http://mirrors.rit.edu/gentoo/releases/ia64/current-stage3/stage3-ia64-20091229.tar.bz2

#Build the vserver.  –context is just a unique number you set from 1-49152, non-inclusive. Everything else is relatively self explanatory.

jolt:/space/vserver/base-images/# vserver coffee build –context 1253 –hostname coffee –interface eth2:129.21.50.66/24 –initstyle gentoo -m template — -d gentoo -t /space/vserver/base-images/stage3-ia64-20091229.tar.bz2

jolt:/space/vserver/base-images/# cd /var/lib/vservers/coffee
jolt:/var/lib/vservers/coffee# cp -L /etc/resolv.conf ./etc/resolv.conf
jolt:/var/lib/vservers/coffee# chroot ./ /bin/bash
jolt / # env-update
>>> Regenerating /etc/ld.so.cache…
jolt / # source /etc/profile
jolt / # export PS1=”(chroot) $PS1″
(chroot) jolt / # emerge –sync
(chroot) jolt / # eselect profile list
Available profile symlink targets:
[1] default/linux/ia64/10.0 *
[2] default/linux/ia64/10.0/desktop
[3] default/linux/ia64/10.0/developer
[4] default/linux/ia64/10.0/server
[5] hardened/linux/ia64/10.0
(chroot) jolt / # eselect profile set 4
(chroot) jolt / # cp /usr/share/zoneinfo/US/Eastern /etc/localtime
(chroot) jolt / # nano /etc/make.conf
(chroot) jolt / # cat /etc/make.conf
#Gentoo Itanium VServer make.conf Created: 2010-01-14 by Clockfort (devnull@remove_this_part_for_spam_reasons.csh.rit.edu)
CFLAGS=”-O2 -mtune=mckinley -pipe”
CXXFLAGS=”-O2 -pipe”
# WARNING: Changing your CHOST is not something that should be done lightly.
# Please consult http://www.gentoo.org/doc/en/change-chost.xml before changing.
CHOST=”ia64-unknown-linux-gnu”
MAKEOPTS=”-j5″

# Portage Options
EMERGE_DEFAULT_OPTS=”–ask –verbose –tree –jobs=5″
PORTAGE_ELOG_CLASSES=”info warn error log”
PORTAGE_ELOG_SYSTEM=”save”
AUTOCLEAN=”yes”
FEATURES=”parallel-fetch userfetch collision-protect buildpkg”
PORT_LOGDIR=/var/log/portage

# Portage Sync/Download Locations
GENTOO_MIRRORS=”http://mirrors.rit.edu/gentoo http://www.gtlib.gatech.edu/pub/gentoo ftp://ftp.wallawalla.edu/pub/mirrors/ftp.gentoo.org http://lug.mtu.edu/gentoo/”
SYNC=”rsync://rsync.namerica.gentoo.org/gentoo-portage”

# Use Flags
USE=”$USE symlink” # Kernel
USE=”$USE mmx sse sse2″ # CPU
USE=”$USE pam ssl” # Authentication
USE=”$USE bash-completion” # Completion
USE=”$USE X gtk svg qt3support” # X support
USE=”$USE xulrunner” # native browser integration
USE=”$USE jpeg png” #image support
#FEATURES=”${FEATURES} candy”

(chroot) jolt / # passwd
#Ignore the scanelf “unaligned access” error you get, it won’t hurt anyone, just slow things down a bit.
(chroot) jolt / # emerge syslog-ng
(chroot) jolt / # rc-update add syslog-ng default
#Remove reference to /proc/kmsg
(chroot) jolt / # nano /etc/syslog-ng/syslog-ng.conf
(chroot) jolt / # emerge vixie-cron
(chroot) jolt / # rc-update add vixie-cron default
(chroot) jolt / # exit
jolt:/var/lib/vservers/coffee# vserver coffee start
jolt:/var/lib/vservers/coffee# vserver coffee enter
See also:
http://linux-vserver.org/Installation_on_Debian
http://www.gentoo.org/proj/en/vps/vserver-howto.xml
http://linux-vserver.org/Frequently_Asked_Questions
http://www.gentoo.org/doc/en/handbook/

I decided to redo the NFS setup I previously had on my ol’ Itanium2 machine.

How to test to see which NFS block size is right for your setup:

#Testing write speeds:
mount hostname:/remote_folder/ /mnt/local_folder/ -o rw,wsize=1024
time dd if=/dev/zero of=/mnt/local_folder/write_test bs=16k count=16k
umount /mnt/local_folder/
#Testing read speeds:
mount hostname:/remote_folder/ /mnt/local_folder/ -o ro,rsize=1024
time dd if=/mnt/local_folder/write_test of=/dev/null bs=16k
umount /mnt/local_folder/

Go through and change wsize and rsize to 1024, 2048, 4096, 8192, 16384, 32768. MAKE SURE to unmount after every test, as otherwise caching may cause you to see erroneously high speeds for subsequent tests.

Additional performance boosters:
Use “async” and “noatime” options when mounting your remote NFS directory. Async can be a little dangerous, as the data is not immediately flushed to disk on the server, but if the server is rather reliable, it shouldn’t be a problem. “noatime” prevents the access time for the files accessed over NFS from being updated. POSIX says you should update the access time every time you read a file, but doing a write operation to a disk every time you need to read from a file is time-costly and, frankly, rather silly from a performance perspective.

If you can, allow jumbo packets between the two machines you are using NFS between. Beware, however, that these oversized packets must be supported by the server, the client, and every network switch and router in-between in order for that to function properly. If you know for sure that is the case, then go for it, and you’ll probably see an increase in throughput, as the nfs packets (especially the larger ones) will be fragmented into far fewer IP packets. Also beware that other vendors implementations of NFS differ than Linux’s slightly, for instance Apple OS X clients seem to only be able to connect to NFS servers with “insecure” as an /etc/export option for that particular NFS directory.

Test Machine Specs:
2x 1.5GHz Itanium2 processors
10GB ECC DDR
3x 74GB 15k RPM U320 SCSI drives in RAID5
Gigabit ethernet

I am somewhat confused as to why my own NFS write performance is so appalling, while the read performance approaches the theoretical maximums of a gigabit ethernet connection.
Any advice on improving write performance over NFS would be much appreciated.

(Graph is here if you’re reading this post without images)