Clockfort's Tech Blog

If you’re trying to use Teensy-Loader on recent Fedora builds (12 & 13)
You’ll need to get:

gtk2.i686

gtk2-engines.i686
(Solves “Gtk-WARNING **: Unable to locate theme engine in module_path: “clearlooks”")

PackageKit-gtk-module.i686
(Solves Gtk-Message: Failed to load module “pk-gtk-module”: libpk-gtk-module.so: cannot open shared object file: No such file or directory)

libcanberra.i686
libcanberra-gtk2.i686 (Solves Gtk-Message: Failed to load module “canberra-gtk-module”: libcanberra-gtk-module.so: cannot open shared object file: No such file or directory)

Except for canberra, most all of these packages are in the default install of i686 Fedora builds, just not in the x86_64 ones.

You’ll also want:
sudo yum -y install avr-binutils avr-gcc avr-libc avr-libc-docs avr-gdb avra
in order to actually compile C code for the Teeny board itself.
(GDB is optional, but nice to have, frankly)

So, normally, backup software is going to read 90+% of your machine’s hard drive, and push that data over the network to someplace else.

However, in the process of doing this, it is going to steamroll your operating system’s filesystem cache.

There are two improvements that can be had here, in my reckoning, and I’ve seen neither in implementation.

1) Have the backup software back up the files that are in the operating system’s cache FIRST. Not only does this mean that important, often-changed files get priority, it means that backups will be slightly faster, as this data will come from memory, rather than from the disk.
This would probably require some way of getting at a list of what files the kernel/caching-daemon/whatever has in memory, which I’m not sure exists at the moment.

2) Have the backup software use a different way of accessing the disk such that the file caching daemon does not cache the files that the backup software is reading. This way, rather than steamrolling over the carefully-laid out filesystem cache of the system’s most often and/or most recently used files, the uncommon never-accessed-normally files don’t suddenly get pushed into filesystem cache when the backup software accesses them. This would lead to general system speedup, as “better” files from disk would be cached, rather than rarely-used junk.
This could be as simple as adding an oflag option to the kernel (i.e. fcntl.h in linux) that says “don’t cache this please”, and then using this when calling the open function in the backup application.

… Just some musings I had while talking to Russ on our way driving to California.

Security Warrior by Cyrus Peikari and Anton Chuvakin
ISBN: 9780596005450

This book is an excellent introduction into the world of computer security. I was a bit surprised at the contents; the book features many more offensive techniques, like reverse engineering binaries, performing successful stack/heap overflows, attacks on a variety of server/network platforms, and defeating IDS/forensic technologies. I had initially expected the book to be more focused on security defense, which is covered, but certainly not in a typical ratio. I wouldn’t complain though, because as is stated in this book several times, a good offense is a good defense. For instance, upon introducing stack overflows, the authors wisely quip how a company could save a great deal of money and embarrassment if its employees found such vulnerabilities before they leak into the wild.

If I did have one bad thing to say about Security Warrior, it’s that I happen to know quite a bit about its entire first section already, so I found parts quite tiresome. Having already read such texts as Chris Eagle’s “The Ida Pro Book”, this book’s section on disassembly seemed a paltry introduction in comparison; however, it seems this amount would be about right to gently introduce someone to the subject, were they not already aware of this field of computer security knowledge.

All in all, Security Warrior is a good introductory text to a wide variety of computer security related topics, and hopefully the reader will leave interested in implementing at least a few of the defensive strategies listed, or want to become more familiar with some of the more interesting attack vectors. Further reading/knowledge will be needed other than the information found here in order to do useful security work, but, Security Warrior certainly at least gets the ball rolling and the interest piqued.

ISBN: 9781593271862
The Book of Xen: A Practical Guide for the System Administrator

Most Xen documentation on the Internet can be a tad focused on the single-computer, single-admin personal-use Xen administration case. This book, thankfully, is not. This is definitely the book to keep on your shelf if you require tips and tricks for setting up your own VPS hosting service, with its world full of  malevolent users needing to be kept in their place, quotas for bandwidth, disk I/O, CPU time, and memory usage, and allowing your users to configure their own instances without you having to step in every time they blow out their /boot partitions.

There are plenty of concepts covered in here for other use-cases (besides just hosting your own VPS provider) as well, including remote-mounting disks over NFS/iSCSI/AoE, migrating live Xen instances across a cluster of servers, and backing up disk images and machine states.

The Book of Xen provides a fair and balanced view of Xen management; that is to say, while it it does talk often about the many distro-specific ways of easily bootstrapping and configuring a new virtual server (like Debian’s debootstrap, Red Hat’s virt-install, or even creating images in Citrix XenServer) it also covers vendor and distro-neutral ways of performing all the required installation and management tasks. The Book of Xen is also fair in that it also goes on to describe the use and configuration of Microsoft, BSD, and Solaris Xen dom0 and domUs as well, with the caveat that support for Xen is weak and upcoming on such platforms as FreeBSD, and that HVM is required for many of these more exotic operating systems like “Microsoft Windows”, as there are no Xen hooks in the Windows kernel.

I particularly liked the Book of Xen’s first chapter, which, unrelated to the rest of the book’s sysadmin-oriented content, was a good overview of the technical underpinnings of the Xen hypervisor platform, and how it interacts with the hardware and virtualized machines from a very low-level perspective. As it is stated later in the book, and something that I agree with, the authors believe that one must know a technology, how it works, and its more basic manual and command line tools, before ever trusting a GUI or web interface to do the same. It will also surely aid debugging later when something goes wrong, as the administrator will have a good idea as to where the problem might lie.

All in all, I liked the book and would recommend it to anyone setting up their own Xen servers, however, I wished it would have had more information about Xen on the Intel Itanium (which is touched upon in the book as being a supported platform, but not talked about further) and I wish it had talked more about some of the topics they covered, like giving users access to their own Xen management consoles, in the common situation where there are many physical machines that a user’s instance could be on, a situation which completely broke their offered solutions for this situation and others.

There is like, zero documentation for doing this on the Internet. No one else wants to install Gentoo in a VServer guest on a Debian Itanium2 host? Lame.

This quick overview contains many things specific to my personal setup; therefore:

• You will want to check your local gentoo mirror to see what the current ia64 tarball is.
• You will want to change the name, hostname, and network address of your vserver
• You can use whatever directories you want; the actual files will go in /var/lib/vservers/<hostname>/ , (and /etc/vservers/<hostname>/ ) and you can delete the stage3 tarball after you’re done.
• So you don’t get confused, “jolt” is the name of my debian host machine, and “coffee” is the vserver guest I am creating.

#Install the Vserver kernel and utilities if you already have not.

jolt:/space/vserver# aptitude install linux-image-vserver-mckinley util-vserver vserver-debiantools

#Reboot to boot into new kernel.

jolt:/space/vserver# reboot

#Are you running the new kernel? It should say “vserver” in there somewhere.

jolt:/space/vserver# uname -a
Linux jolt 2.6.26-2-vserver-mckinley #1 SMP Thu Nov 5 07:44:36 UTC 2009 ia64 GNU/Linux

#Get the latest Stage3 tarball from a mirror close to you.

jolt:/space/vserver/base-images/# wget http://mirrors.rit.edu/gentoo/releases/ia64/current-stage3/stage3-ia64-20091229.tar.bz2

#Build the vserver.  –context is just a unique number you set from 1-49152, non-inclusive. Everything else is relatively self explanatory.

jolt:/space/vserver/base-images/# vserver coffee build –context 1253 –hostname coffee –interface eth2:129.21.50.66/24 –initstyle gentoo -m template — -d gentoo -t /space/vserver/base-images/stage3-ia64-20091229.tar.bz2

jolt:/space/vserver/base-images/# cd /var/lib/vservers/coffee
jolt:/var/lib/vservers/coffee# cp -L /etc/resolv.conf ./etc/resolv.conf
jolt:/var/lib/vservers/coffee# chroot ./ /bin/bash
jolt / # env-update
>>> Regenerating /etc/ld.so.cache…
jolt / # source /etc/profile
jolt / # export PS1=”(chroot) $PS1″
(chroot) jolt / # emerge –sync
(chroot) jolt / # eselect profile list
Available profile symlink targets:
[1] default/linux/ia64/10.0 *
[2] default/linux/ia64/10.0/desktop
[3] default/linux/ia64/10.0/developer
[4] default/linux/ia64/10.0/server
[5] hardened/linux/ia64/10.0
(chroot) jolt / # eselect profile set 4
(chroot) jolt / # cp /usr/share/zoneinfo/US/Eastern /etc/localtime
(chroot) jolt / # nano /etc/make.conf
(chroot) jolt / # cat /etc/make.conf
#Gentoo Itanium VServer make.conf Created: 2010-01-14 by Clockfort (devnull@remove_this_part_for_spam_reasons.csh.rit.edu)
CFLAGS=”-O2 -mtune=mckinley -pipe”
CXXFLAGS=”-O2 -pipe”
# WARNING: Changing your CHOST is not something that should be done lightly.
# Please consult http://www.gentoo.org/doc/en/change-chost.xml before changing.
CHOST=”ia64-unknown-linux-gnu”
MAKEOPTS=”-j5″

# Portage Options
EMERGE_DEFAULT_OPTS=”–ask –verbose –tree –jobs=5″
PORTAGE_ELOG_CLASSES=”info warn error log”
PORTAGE_ELOG_SYSTEM=”save”
AUTOCLEAN=”yes”
FEATURES=”parallel-fetch userfetch collision-protect buildpkg”
PORT_LOGDIR=/var/log/portage

# Portage Sync/Download Locations
GENTOO_MIRRORS=”http://mirrors.rit.edu/gentoo http://www.gtlib.gatech.edu/pub/gentoo ftp://ftp.wallawalla.edu/pub/mirrors/ftp.gentoo.org http://lug.mtu.edu/gentoo/”
SYNC=”rsync://rsync.namerica.gentoo.org/gentoo-portage”

# Use Flags
USE=”$USE symlink” # Kernel
USE=”$USE mmx sse sse2″ # CPU
USE=”$USE pam ssl” # Authentication
USE=”$USE bash-completion” # Completion
USE=”$USE X gtk svg qt3support” # X support
USE=”$USE xulrunner” # native browser integration
USE=”$USE jpeg png” #image support
#FEATURES=”${FEATURES} candy”

(chroot) jolt / # passwd
#Ignore the scanelf “unaligned access” error you get, it won’t hurt anyone, just slow things down a bit.
(chroot) jolt / # emerge syslog-ng
(chroot) jolt / # rc-update add syslog-ng default
#Remove reference to /proc/kmsg
(chroot) jolt / # nano /etc/syslog-ng/syslog-ng.conf
(chroot) jolt / # emerge vixie-cron
(chroot) jolt / # rc-update add vixie-cron default
(chroot) jolt / # exit
jolt:/var/lib/vservers/coffee# vserver coffee start
jolt:/var/lib/vservers/coffee# vserver coffee enter
See also:
http://linux-vserver.org/Installation_on_Debian
http://www.gentoo.org/proj/en/vps/vserver-howto.xml
http://linux-vserver.org/Frequently_Asked_Questions
http://www.gentoo.org/doc/en/handbook/

ISBN: 9781886411951
How Not to Program in C++: 111 Broken Programs and 3 Working Ones, or Why Does 2+2=5986

Do you enjoy puzzles? Do you enjoy debugging other people’s code? If so, you’ll enjoy this book.

If reference materials or traditional educational coding books were like newspapers, then this book would be the crossword puzzle page. Just like any crossword puzzle, some sections are harder than others, and the puzzles in this book are no exception.  If you can’t quite figure out the subtle differences between pointers, addresses, double pointers, etc, then this listing of the plethora of ways that you can get yourself into trouble while programming in C++ will likely prove too difficult to solve at parts. Without a doubt though, anyone who fully knows C++ (and C, which is also covered in this book) and all its intricacies will not find most of the puzzles exceptionally difficult.

That being said, the book still proves a good read for anyone of higher programming skill; any reader will be scratching his or her head to find the truly subtle ways that the all-too-simple-looking sample programs have been broken. Luckily, there are helpful, very optional hints and answers in the back of the book one can read in order to get thinking on the right track without spoiling too much of the fun. Additionally, throughout the book’s puzzles, there are amusing programmer-related debugging horror stories and funny programming quips that provide a brief smile, even when one is wracking one’s brains out to find the misplaced comma, subtly misspelled keyword, missing semicolon, or devious memory misallocation that is keeping the answer of the problem elusive.

ISBN: 9780596101510

This epically-titled O’Reilly book is a well-organized collection of network configuration tips, stories, and common “gotchas”, as told by a self-admitted grouchy old network admin to younger, wet-behind-the-ears network administrators.

The author, in a move uncommon to most networking manuals, just cuts to the chase and says what needs to be said. Everything is told from a Cisco perspective, with Cisco terminology, and the only hint of variation allowed for is the occasional explanation when something is radically (or subtly, in some way that would ruin everything when you least expect it) different between CatOS and IOS. This is undeniably a good thing; it keeps the book short, and realistically, Cisco is one of the forefront leaders in the enterprise network market.

A wide range of topics are talked about – possible problems that you could run into with auto-negotiation on your fast-ethernet network, how to configure spanning tree or etherchannel, getting QoS to work properly, and a whole host of topics one should know when creating one’s own medium-to-large sized network. Even for those that already know how to implement these features, the author explains exactly when someone would want to use these features and how they evolved, and how they ought to be properly used.

I would recommend this book to anyone interested in enterprise networking – ranging from the relatively professionally uninitiated like myself (I only do networking administration for Computer Science House at the Rochester Institute of Technology) to those who are just transitioning from networking classes to actual jobs, who will benefit perhaps the most from the book’s tips and tricks from a person in industry, or even the average middle-aged network administrator, who may find a large portion of the book a snooze, but likely still pick up a trick or two that was previously unknown.

I found an undocumented feature in Cisco IOS 11.2.
The command “show interface description” exists in newer versions, so I typed it in out of habit. Even though it doesn’t tab-complete it, or know what that option to the sh int command is, it still works… Albeit badly, and it gives terribly formatted output (All on one line, no spaces between names, no blank spaces where unlabeled ports are, etc)

It still works though! :-)

Makes me wonder how many undocumented features exist in versions of IOS earlier than they are supposedly “released”.


drpepper#sh ver
Cisco Internetwork Operating System Software
IOS (tm) C2900XL Software (C2900XL-HS-M), Version 11.2(8.10)SA6,
MAINTENANCE INTERIM SOFTWARE
Copyright (c) 1986-1902 by cisco Systems, Inc.
Compiled Fri 15-Feb-02 09:47 by devgoyal
Image text-base: 0x00003000, data-base: 0x0020E278
...(rest of output truncated for brevity)
drpepper#sh int ?
FastEthernet FastEthernet IEEE 802.3
Null Null interface
VLAN Switch VLAN Virtual Interface
accounting Show interface accounting
crb Show interface routing/bridging info
irb Show interface routing/bridging info
link-trap Show interface traps on no link


drpepper#sh int desc
Blacktea LinkCSH UplinkJolt ManagementRESNETRESNETRESNETRESNET


I got very bored in a far-too-easy networking class and ended up coding a very memory efficient version of Conway’s Game of Life.
Of note, it uses bit-twiddling/bit-packing to access individual bits on byte-accessible memory for the gameboard, and it uses a sort of buffer-thing to further reduce memory usage, rather than push temporary changes into an entire other temporary game board. I just made it up as I went along. The idea behind being this memory efficient is so that I can plop this on a microcontroller (possibly CSH’s Big Infosys?) and just let ‘er rip on a LED matrix :-)

Source is up at http://github.com/clockfort/Life/

I decided to redo the NFS setup I previously had on my ol’ Itanium2 machine.

How to test to see which NFS block size is right for your setup:

#Testing write speeds:
mount hostname:/remote_folder/ /mnt/local_folder/ -o rw,wsize=1024
time dd if=/dev/zero of=/mnt/local_folder/write_test bs=16k count=16k
umount /mnt/local_folder/
#Testing read speeds:
mount hostname:/remote_folder/ /mnt/local_folder/ -o ro,rsize=1024
time dd if=/mnt/local_folder/write_test of=/dev/null bs=16k
umount /mnt/local_folder/

Go through and change wsize and rsize to 1024, 2048, 4096, 8192, 16384, 32768. MAKE SURE to unmount after every test, as otherwise caching may cause you to see erroneously high speeds for subsequent tests.

Additional performance boosters:
Use “async” and “noatime” options when mounting your remote NFS directory. Async can be a little dangerous, as the data is not immediately flushed to disk on the server, but if the server is rather reliable, it shouldn’t be a problem. “noatime” prevents the access time for the files accessed over NFS from being updated. POSIX says you should update the access time every time you read a file, but doing a write operation to a disk every time you need to read from a file is time-costly and, frankly, rather silly from a performance perspective.

If you can, allow jumbo packets between the two machines you are using NFS between. Beware, however, that these oversized packets must be supported by the server, the client, and every network switch and router in-between in order for that to function properly. If you know for sure that is the case, then go for it, and you’ll probably see an increase in throughput, as the nfs packets (especially the larger ones) will be fragmented into far fewer IP packets. Also beware that other vendors implementations of NFS differ than Linux’s slightly, for instance Apple OS X clients seem to only be able to connect to NFS servers with “insecure” as an /etc/export option for that particular NFS directory.

Test Machine Specs:
2x 1.5GHz Itanium2 processors
10GB ECC DDR
3x 74GB 15k RPM U320 SCSI drives in RAID5
Gigabit ethernet

I am somewhat confused as to why my own NFS write performance is so appalling, while the read performance approaches the theoretical maximums of a gigabit ethernet connection.
Any advice on improving write performance over NFS would be much appreciated.

(Graph is here if you’re reading this post without images)