Back in November last year I moved from working as a consultant in Red Hat‘s Global Professional Services divison, back to being a member of Engineering. The first project I’ve been involved in is the $100 Laptop, in particular looking at how we can run OS images without needing the real hardware. We quickly identified QEMU as being a reasonable starting point on which to build out a basic ‘Laptop Simulator’. The base QEMU engine is command line driven, and although there is KQEMU and GTK QEMU provide some form of management frontend, neither was entirely satisfactory.
VMWare GSX server has a basic API which lets one create and control virtual machines from Perl. The API, however, was clearly tacked onto the admin UI as an afterthought, and while the capabilities it exposes are very useful, one is frustrated by the bits not exposed to the API. Anticipating the need to construct autmated test harnesses to control QEMU virtual machines, it was clear that any management tool for QEMU must treat its API as a first-class peer to the actual UI. In addition exposing the API as Perl limits the level of interoperability for developers who might want to write their tools in C, or Python, or Java. Being an active member of the DBus community, the answer to this goal was clear:
- Write a management API as a DBus service
- Write a completely separate admin UI as a DBus client
After a few weeks hard work, we now have not only a DBus service for managing QEMU, but a general purpose Python admin UI providing similar levels of functionality to that of VMWare GSX server, a command line Perl tool for quickly launching pre-built OS images with a virtual hardware configuration aligned to that of the $100 laptop, and the beginnings of an automated test suite. The full code trees will be opened up to the community over the next few weeks, but as a taster of what’s to come, there is a flash movie here
Making it through my SpamAssassin filter this week, was a Nigeria 419er, with an entertaining entry line…
Dear/sir/ma
In the name of Almighty Allah, The Merciful, the Master of the day of
Judgment I greet you May The Almighty Allah Give You The Wisdom To
Understand My Predicament
Sadly it then deterioated into the regular spiel of “you can get $15 million if you send us your bank account details”.
On my way out to Boston, I left from Heathrow terminal 4, where they’re currently trialing a new full body scanner as an alternative to walking through the metal detectors at the security checkpoints. They have notices up informing you that they are trialling a new body scanner and that one might be ‘randomly’ selected to go through the new scanner. If you decline to take part in the trial they will perform a full hand search.
Of course the signs explaining this don’t mention that this scanner will quite literally ‘see through clothes’, leaving nothing at all to the imagination. Having previously read The Register’s article on this trail, I’ve seen examples of the explicit images produced. So just how randomly are the participants selected ? Well, in the 15 minutes I was standing in line waiting, approximately 10 people were selected to through the scanner, all attractive women, around the age of 20 – 30, travelling alone. Apparently a far from random selection process for this ‘see through clothes’ scanner.
It has been just over a year & 1/2 since I first blogged about DTrace suggesting that a similar tool would be very valuable to the Linux community. Well after a few long email threads, it turned out that a significant number of people within Red Hat agreed with this assessment and so in partnership with IBM and Intel the SystemTAP project came into life at the start of 2005. Starting with the previously developed KProbes dynamic instrumentation capability a huge amount of work has been done building out a high level language and runtime for safely, efficiently & reliably probing the kernel. It has seen a limited ‘technology preview’ in RHEL-4, and with its inclusion in the forthcoming Fedora Core 5 it will be exposed to a much wider community of users & potential developers.
On the very same day as Dave Jones was looking at the Fedora boot process via static kernel instrumentation, I was (completely co-incidentally ) playing around using SystemTAP to instrument the boot process. The probe I wrote looked at file opens, process fork/execve to enable a hierarchical view of startup to be pieced together. A simplified version of the script looked like:
global indent
function timestamp() {
return string(gettimeofday_ms()) . indent[pid()] . " "
}
function proc() {
return string(pid()) . " (" . execname() . ")"
}
function push(pid, ppid) {
indent[pid] = indent[ppid] . " "
}
function pop(pid) {
delete indent[pid]
}
probe kernel.function("sys_clone").return {
print(timestamp() . proc() . " forks " . string(retval()). "\n")
push(retval(), pid())
}
probe kernel.function("do_execve") {
print(timestamp() . proc() . " execs " . kernel_string($filename) . "\n")
}
probe kernel.function("sys_open") {
if ($flags & 1) {
print(timestamp() . proc() . " writes " . user_string($filename) . "\n")
} else {
print(timestamp() . proc() . " reads " . user_string($filename) . "\n")
}
}
probe kernel.function("do_exit") {
print(timestamp() . proc() . " exit\n")
pop(pid())
}
A few tricks later it was running during boot, and having analysed the results with Perl one can display a summary of how many files each init script opened
1 init read 90 write 30 running...
251 init read 29 write 0 run 23.08s
252 rc.sysinit read 1035 write 45 run 22.91s
274 start_udev read 355 write 128 run 15.10s
286 start_udev read 91 write 0 run 1.90s
287 MAKEDEV read 91 write 0 run 1.88s
291 udevstart read 177 write 124 run 3.95s
614 usleep read 2 write 0 run 1.05s
649 udev-stw.modules read 84 write 5 run 1.23s
701 dmraid read 111 write 0 run 1.07s
748 rc read 235 write 13 run 14.57s
753 S10network read 111 write 16 run 2.85s
833 S12syslog read 44 write 3 run 0.43s
844 S25netfs read 87 write 1 run 1.51s
861 S55cups read 31 write 2 run 1.70s
878 S55sshd read 52 write 1 run 0.86s
892 S97messagebus read 31 write 2 run 0.44s
900 S98NetworkManager read 96 write 10 run 0.58s
910 S98NetworkManagerDispatcher read 92 write 3 run 0.67s
921 S98avahi-daemon read 29 write 0 run 0.41s
929 S98haldaemon read 31 write 2 run 4.20s
955 S99local read 17 write 1 run 0.16s
There are so many other interesting ways to analyse the data collected at boot which I don’t have space for in this blog, so I’ve put all the information (including how to run SystemTAP during boot) up on my Red Hat homepage
With all the talk these days around Xen, one could be forgiven for thinking it is the only open source virtualization technology out there. This is certainly not the case, however, User Mode Linux has been providing the ability to run the Linux kernel in userspace for years, and more recently QEmu has matured to the point where it can reliably run pretty much any i?86 / x86_64 / PPC operating system. Each of these systems uses a different approach for virtualization; Xen uses hypervisor to mutliplex hardware access amongst multiple guest OS’; User Mode Linux consists of the a number of kernel patches to enable one to run the Linux kernel as a regular user space process; QEmu provides a userspace CPU emulator, with an optional kernel accelerator. The latter is particularly intersting in the realm of testing because it allows one to mix architectures, ie emulate a PPC guest on a x86 host, or vica-verca.
Setting up a Fedora Core 4 guest
So how does one go about setting up a guest OS running in QEmu ? For this post, I’ll outline the steps for setting up a minimal Fedora Core 4 instance on x86.
Creating a local install tree
While one could install directly from the Fedora download site, it is preferrable to setup a local YUM repository containing a Fedora Core install tree. The first step is to download the master ISO images to /mnt/isos/fc4/i386:
# mkdir -p /mnt/isos/fc4/i386
# cd /mnt/isos/fc4/i386
# wget http://download.fedora.redhat.com/pub/fedora/linux/core/4/i386/iso/FC4-i386-disc1.iso
# wget http://download.fedora.redhat.com/pub/fedora/linux/core/4/i386/iso/FC4-i386-disc2.iso
# wget http://download.fedora.redhat.com/pub/fedora/linux/core/4/i386/iso/FC4-i386-disc3.iso
# wget http://download.fedora.redhat.com/pub/fedora/linux/core/4/i386/iso/FC4-i386-disc4.iso
Now its time to unpack all the files to a convenient directory, forming the master installation image. For this guide, setup the install tree to be /mnt/distros/fc4/i386.
# mkdir -p /mnt/distros/fc4/i386
# mkdir /mnt/loopback
# cd /mnt/distros/fc4/i386
# for i in 1 2 3 4
do
mount /mnt/isos/fc4/i386/FC4-i386-disc${i}.iso /mnt/loopback -o loop
cp -af /mnt/loopback/* .
umount /mnt/loopback
done
If one is tight for disk space, the ISO images can be discarded now, although its a wise idea to at least burn them to a CD for future reference. The final step is to make the install tree available via Apache. This can be achieved by dropping a single file into /etc/httpd/conf.d containing:
Alias /distros/ /mnt/distros/
<Location /distros>
Allow from all
Options +Indexes
</Location>
Installing QEmu packages
Thomas Chung provides RPM builds of QEmu and the kernel accelerator module for Fedora Core 4 systems, so download the RPMs which match the kernel currently running in your host system, and install them now
# cd /root
# wget -r -l 1 http://fedoranews.org/tchung/qemu/fc4/0.7.2/2.6.13_1.1532_FC4/
# cd fedoranews.org/tchung/qemu/0.7.2/2.6.13_1.1532_FC4
# rpm -ivh *.i386.rpm
Preparing a Kickstart file
If one plans to install multiple guest OS images, its worthwhile creating kickstart file containing the desired anconda options. The system-config-kickstart program provides a simple GUI for doing this. At the minimum change the following options:
- Basic configuration: clear the ‘reboot after installation’ checkbox
- Basic configuration: select your timezone, keyboardk, and default language
- Basic configuration: choose a suitable root password
- Installation method: select ‘HTTP’ and enter ‘10.0.2.2’ for the ‘HTTP server’, and ‘/distros/fc4/i386’ as the ‘HTTP directory’
- Partition information: create a single ext3 partition on /, and make it fill to maximum size.
- Network configuration: add a single ethernet device, and use DHCP
- Firewall configuration: disable firewall, and disable SELinux
- Package configuration: select whichever package groups are desired.
Save the finished kickstart file to /mnt/distros/fc4/i386/qemu-ks.cfg
Creating and installing a guest
It is now time to create a QEmu guest OS. First of all prepare a disk image to use as the root filesystem, making sure its large enough to hold the package selection choose in the kickstart. As a rule of thumb, 1 GB is enough for a minimal install, 2 GB if X is added, and 6 GB for everything.
# mkdir /mnt/guests
# cd /mnt/guests
# qemu-img qemu-fc4-i386.img 2G
Now launch a virtual machine with this root filesystem, and booting the anaconda CD installer. The VM must have at least 256 MB of memory, otherwise anaconda will crash during installation.
# qemu -user-net -hda qemu-fc4-i386.img -boot d -cdrom /mnt/distros/fc4/i386/images/boot.iso -m 256
When the initial anaconda boot prompt is displayed, request a kickstart install by specifying
linux ks=http://10.0.2.2/distros/fc4/i386/qemu-ks.cfg
If all goes to plan, in about 10-15 minutes time anaconda will have completed installation of the guest OS. When it is done, shutdown the virtual machine – simply close its window once anaconda has shutdown.
Using the guest OS
Now, to use the virtual machine one can simply launch it with
# qemu -user-net -m 256m qemu-fc4-i386.img
With the userspace networking support, when inside the guest, the host machine can be accessed using the IP address 10.0.2.2. For convenience it may be easier to run the guest OS headless, and access it over a SSH connection. To do this, launch it with the following options
# qemu -user-net -m 256m -nographic -redir tcp:8000::22
The once booted, the guest can be accessed via SSH on port 8000
# ssh -p 8000 localhost