Clockfort's Tech Blog

ISBN: 9781593271862
The Book of Xen: A Practical Guide for the System Administrator

Most Xen documentation on the Internet can be a tad focused on the single-computer, single-admin personal-use Xen administration case. This book, thankfully, is not. This is definitely the book to keep on your shelf if you require tips and tricks for setting up your own VPS hosting service, with its world full of  malevolent users needing to be kept in their place, quotas for bandwidth, disk I/O, CPU time, and memory usage, and allowing your users to configure their own instances without you having to step in every time they blow out their /boot partitions.

There are plenty of concepts covered in here for other use-cases (besides just hosting your own VPS provider) as well, including remote-mounting disks over NFS/iSCSI/AoE, migrating live Xen instances across a cluster of servers, and backing up disk images and machine states.

The Book of Xen provides a fair and balanced view of Xen management; that is to say, while it it does talk often about the many distro-specific ways of easily bootstrapping and configuring a new virtual server (like Debian’s debootstrap, Red Hat’s virt-install, or even creating images in Citrix XenServer) it also covers vendor and distro-neutral ways of performing all the required installation and management tasks. The Book of Xen is also fair in that it also goes on to describe the use and configuration of Microsoft, BSD, and Solaris Xen dom0 and domUs as well, with the caveat that support for Xen is weak and upcoming on such platforms as FreeBSD, and that HVM is required for many of these more exotic operating systems like “Microsoft Windows”, as there are no Xen hooks in the Windows kernel.

I particularly liked the Book of Xen’s first chapter, which, unrelated to the rest of the book’s sysadmin-oriented content, was a good overview of the technical underpinnings of the Xen hypervisor platform, and how it interacts with the hardware and virtualized machines from a very low-level perspective. As it is stated later in the book, and something that I agree with, the authors believe that one must know a technology, how it works, and its more basic manual and command line tools, before ever trusting a GUI or web interface to do the same. It will also surely aid debugging later when something goes wrong, as the administrator will have a good idea as to where the problem might lie.

All in all, I liked the book and would recommend it to anyone setting up their own Xen servers, however, I wished it would have had more information about Xen on the Intel Itanium (which is touched upon in the book as being a supported platform, but not talked about further) and I wish it had talked more about some of the topics they covered, like giving users access to their own Xen management consoles, in the common situation where there are many physical machines that a user’s instance could be on, a situation which completely broke their offered solutions for this situation and others.

There is like, zero documentation for doing this on the Internet. No one else wants to install Gentoo in a VServer guest on a Debian Itanium2 host? Lame.

This quick overview contains many things specific to my personal setup; therefore:

• You will want to check your local gentoo mirror to see what the current ia64 tarball is.
• You will want to change the name, hostname, and network address of your vserver
• You can use whatever directories you want; the actual files will go in /var/lib/vservers/<hostname>/ , (and /etc/vservers/<hostname>/ ) and you can delete the stage3 tarball after you’re done.
• So you don’t get confused, “jolt” is the name of my debian host machine, and “coffee” is the vserver guest I am creating.

#Install the Vserver kernel and utilities if you already have not.

jolt:/space/vserver# aptitude install linux-image-vserver-mckinley util-vserver vserver-debiantools

#Reboot to boot into new kernel.

jolt:/space/vserver# reboot

#Are you running the new kernel? It should say “vserver” in there somewhere.

jolt:/space/vserver# uname -a
Linux jolt 2.6.26-2-vserver-mckinley #1 SMP Thu Nov 5 07:44:36 UTC 2009 ia64 GNU/Linux

#Get the latest Stage3 tarball from a mirror close to you.

jolt:/space/vserver/base-images/# wget http://mirrors.rit.edu/gentoo/releases/ia64/current-stage3/stage3-ia64-20091229.tar.bz2

#Build the vserver.  –context is just a unique number you set from 1-49152, non-inclusive. Everything else is relatively self explanatory.

jolt:/space/vserver/base-images/# vserver coffee build –context 1253 –hostname coffee –interface eth2:129.21.50.66/24 –initstyle gentoo -m template — -d gentoo -t /space/vserver/base-images/stage3-ia64-20091229.tar.bz2

jolt:/space/vserver/base-images/# cd /var/lib/vservers/coffee
jolt:/var/lib/vservers/coffee# cp -L /etc/resolv.conf ./etc/resolv.conf
jolt:/var/lib/vservers/coffee# chroot ./ /bin/bash
jolt / # env-update
>>> Regenerating /etc/ld.so.cache…
jolt / # source /etc/profile
jolt / # export PS1=”(chroot) $PS1″
(chroot) jolt / # emerge –sync
(chroot) jolt / # eselect profile list
Available profile symlink targets:
[1] default/linux/ia64/10.0 *
[2] default/linux/ia64/10.0/desktop
[3] default/linux/ia64/10.0/developer
[4] default/linux/ia64/10.0/server
[5] hardened/linux/ia64/10.0
(chroot) jolt / # eselect profile set 4
(chroot) jolt / # cp /usr/share/zoneinfo/US/Eastern /etc/localtime
(chroot) jolt / # nano /etc/make.conf
(chroot) jolt / # cat /etc/make.conf
#Gentoo Itanium VServer make.conf Created: 2010-01-14 by Clockfort (devnull@remove_this_part_for_spam_reasons.csh.rit.edu)
CFLAGS=”-O2 -mtune=mckinley -pipe”
CXXFLAGS=”-O2 -pipe”
# WARNING: Changing your CHOST is not something that should be done lightly.
# Please consult http://www.gentoo.org/doc/en/change-chost.xml before changing.
CHOST=”ia64-unknown-linux-gnu”
MAKEOPTS=”-j5″

# Portage Options
EMERGE_DEFAULT_OPTS=”–ask –verbose –tree –jobs=5″
PORTAGE_ELOG_CLASSES=”info warn error log”
PORTAGE_ELOG_SYSTEM=”save”
AUTOCLEAN=”yes”
FEATURES=”parallel-fetch userfetch collision-protect buildpkg”
PORT_LOGDIR=/var/log/portage

# Portage Sync/Download Locations
GENTOO_MIRRORS=”http://mirrors.rit.edu/gentoo http://www.gtlib.gatech.edu/pub/gentoo ftp://ftp.wallawalla.edu/pub/mirrors/ftp.gentoo.org http://lug.mtu.edu/gentoo/”
SYNC=”rsync://rsync.namerica.gentoo.org/gentoo-portage”

# Use Flags
USE=”$USE symlink” # Kernel
USE=”$USE mmx sse sse2″ # CPU
USE=”$USE pam ssl” # Authentication
USE=”$USE bash-completion” # Completion
USE=”$USE X gtk svg qt3support” # X support
USE=”$USE xulrunner” # native browser integration
USE=”$USE jpeg png” #image support
#FEATURES=”${FEATURES} candy”

(chroot) jolt / # passwd
#Ignore the scanelf “unaligned access” error you get, it won’t hurt anyone, just slow things down a bit.
(chroot) jolt / # emerge syslog-ng
(chroot) jolt / # rc-update add syslog-ng default
#Remove reference to /proc/kmsg
(chroot) jolt / # nano /etc/syslog-ng/syslog-ng.conf
(chroot) jolt / # emerge vixie-cron
(chroot) jolt / # rc-update add vixie-cron default
(chroot) jolt / # exit
jolt:/var/lib/vservers/coffee# vserver coffee start
jolt:/var/lib/vservers/coffee# vserver coffee enter
See also:
http://linux-vserver.org/Installation_on_Debian
http://www.gentoo.org/proj/en/vps/vserver-howto.xml
http://linux-vserver.org/Frequently_Asked_Questions
http://www.gentoo.org/doc/en/handbook/

Edit: As of 2010, Sun has corrected themselves and now state that OpenBSD is only supported with virtualization processor extensions enabled.

VirtualBox, while I like the interface and it has a few features that are lacking in VMWare and other virtualization products, has many issues.
First off, it doesn’t quite implement the x86 instruction architecture perfectly.
Secondly, this is terrible:
VirtualBox Trouble Ticket #192
VirtualBox Trouble Ticket #639

Basically, Sun has insisted for years that VirtualBox emulates OpenBSD perfectly and yet, it has been completely broken for years and they have not only not fixed the bugs in their own application, but have also not stopped insisting that it works just fine. Theo has posted a rant or two about how Virtualbox’s emulation sucks, and how he has wasted time tracking down non-existant bugs because of users repeatedly filing bug reports for segfaulting software when it is in fact just VirtualBox’s shitty emulation; I’m starting to see why he has this opinion.

Seriously, do not use VirtualBox to emulate OpenBSD (or !any! Unix other than Solaris). Everything is broken. Go try QEMU or VMWare.
But, if you want to try it anyway, here’s the settings you’ll NEED to have. Other settings are left up to your own options.:

General
OS type: OpenBSD
System
VT-x/AMD-V: Enabled*
PAE/NX: Enabled
Nested Paging: Disabled
Display
Video memory: 32MB
Hard Disks
IDE Controller Type: PIIX4
Network
Adapter: Intel PRO/1000 T Server

*obviously this is not possible if your host OS/CPU does not support it. OpenBSD segfaults less with it enabled. It runs(…ish) with it disabled.

tl;dr: VirtualBox is an excellent desktop virtual machine software, especially with its new features, like its 3D-rendering pass through technologies, and many-CPU SMP support. It is a terrible server virtual machine software, and is buggy, cannot run any unix distribution properly, except Sun’s own Solaris 10 kool-aid.

Awesome. I’m using my laptop’s fingerprint sensor to log into my server. Fingerprint Reader –> Laptop/Linux –> VMWare browser plugin W/USB device forwarding –> Server/Linux –> VMWare/Windows.

… makes me wonder why /everyone/ isn’t doing it this way :-)