Making it through my SpamAssassin filter this week, was a Nigeria 419er, with an entertaining entry line…
Dear/sir/ma
In the name of Almighty Allah, The Merciful, the Master of the day of
Judgment I greet you May The Almighty Allah Give You The Wisdom To
Understand My Predicament
Sadly it then deterioated into the regular spiel of “you can get $15 million if you send us your bank account details”.
On my way out to Boston, I left from Heathrow terminal 4, where they’re currently trialing a new full body scanner as an alternative to walking through the metal detectors at the security checkpoints. They have notices up informing you that they are trialling a new body scanner and that one might be ‘randomly’ selected to go through the new scanner. If you decline to take part in the trial they will perform a full hand search.
Of course the signs explaining this don’t mention that this scanner will quite literally ‘see through clothes’, leaving nothing at all to the imagination. Having previously read The Register’s article on this trail, I’ve seen examples of the explicit images produced. So just how randomly are the participants selected ? Well, in the 15 minutes I was standing in line waiting, approximately 10 people were selected to through the scanner, all attractive women, around the age of 20 – 30, travelling alone. Apparently a far from random selection process for this ‘see through clothes’ scanner.
It has been just over a year & 1/2 since I first blogged about DTrace suggesting that a similar tool would be very valuable to the Linux community. Well after a few long email threads, it turned out that a significant number of people within Red Hat agreed with this assessment and so in partnership with IBM and Intel the SystemTAP project came into life at the start of 2005. Starting with the previously developed KProbes dynamic instrumentation capability a huge amount of work has been done building out a high level language and runtime for safely, efficiently & reliably probing the kernel. It has seen a limited ‘technology preview’ in RHEL-4, and with its inclusion in the forthcoming Fedora Core 5 it will be exposed to a much wider community of users & potential developers.
On the very same day as Dave Jones was looking at the Fedora boot process via static kernel instrumentation, I was (completely co-incidentally ) playing around using SystemTAP to instrument the boot process. The probe I wrote looked at file opens, process fork/execve to enable a hierarchical view of startup to be pieced together. A simplified version of the script looked like:
global indent
function timestamp() {
return string(gettimeofday_ms()) . indent[pid()] . " "
}
function proc() {
return string(pid()) . " (" . execname() . ")"
}
function push(pid, ppid) {
indent[pid] = indent[ppid] . " "
}
function pop(pid) {
delete indent[pid]
}
probe kernel.function("sys_clone").return {
print(timestamp() . proc() . " forks " . string(retval()). "\n")
push(retval(), pid())
}
probe kernel.function("do_execve") {
print(timestamp() . proc() . " execs " . kernel_string($filename) . "\n")
}
probe kernel.function("sys_open") {
if ($flags & 1) {
print(timestamp() . proc() . " writes " . user_string($filename) . "\n")
} else {
print(timestamp() . proc() . " reads " . user_string($filename) . "\n")
}
}
probe kernel.function("do_exit") {
print(timestamp() . proc() . " exit\n")
pop(pid())
}
A few tricks later it was running during boot, and having analysed the results with Perl one can display a summary of how many files each init script opened
1 init read 90 write 30 running...
251 init read 29 write 0 run 23.08s
252 rc.sysinit read 1035 write 45 run 22.91s
274 start_udev read 355 write 128 run 15.10s
286 start_udev read 91 write 0 run 1.90s
287 MAKEDEV read 91 write 0 run 1.88s
291 udevstart read 177 write 124 run 3.95s
614 usleep read 2 write 0 run 1.05s
649 udev-stw.modules read 84 write 5 run 1.23s
701 dmraid read 111 write 0 run 1.07s
748 rc read 235 write 13 run 14.57s
753 S10network read 111 write 16 run 2.85s
833 S12syslog read 44 write 3 run 0.43s
844 S25netfs read 87 write 1 run 1.51s
861 S55cups read 31 write 2 run 1.70s
878 S55sshd read 52 write 1 run 0.86s
892 S97messagebus read 31 write 2 run 0.44s
900 S98NetworkManager read 96 write 10 run 0.58s
910 S98NetworkManagerDispatcher read 92 write 3 run 0.67s
921 S98avahi-daemon read 29 write 0 run 0.41s
929 S98haldaemon read 31 write 2 run 4.20s
955 S99local read 17 write 1 run 0.16s
There are so many other interesting ways to analyse the data collected at boot which I don’t have space for in this blog, so I’ve put all the information (including how to run SystemTAP during boot) up on my Red Hat homepage
With all the talk these days around Xen, one could be forgiven for thinking it is the only open source virtualization technology out there. This is certainly not the case, however, User Mode Linux has been providing the ability to run the Linux kernel in userspace for years, and more recently QEmu has matured to the point where it can reliably run pretty much any i?86 / x86_64 / PPC operating system. Each of these systems uses a different approach for virtualization; Xen uses hypervisor to mutliplex hardware access amongst multiple guest OS’; User Mode Linux consists of the a number of kernel patches to enable one to run the Linux kernel as a regular user space process; QEmu provides a userspace CPU emulator, with an optional kernel accelerator. The latter is particularly intersting in the realm of testing because it allows one to mix architectures, ie emulate a PPC guest on a x86 host, or vica-verca.
So how does one go about setting up a guest OS running in QEmu ? For this post, I’ll outline the steps for setting up a minimal Fedora Core 4 instance on x86.
While one could install directly from the Fedora download site, it is preferrable to setup a local YUM repository containing a Fedora Core install tree. The first step is to download the master ISO images to /mnt/isos/fc4/i386:
# mkdir -p /mnt/isos/fc4/i386 # cd /mnt/isos/fc4/i386 # wget http://download.fedora.redhat.com/pub/fedora/linux/core/4/i386/iso/FC4-i386-disc1.iso # wget http://download.fedora.redhat.com/pub/fedora/linux/core/4/i386/iso/FC4-i386-disc2.iso # wget http://download.fedora.redhat.com/pub/fedora/linux/core/4/i386/iso/FC4-i386-disc3.iso # wget http://download.fedora.redhat.com/pub/fedora/linux/core/4/i386/iso/FC4-i386-disc4.iso
Now its time to unpack all the files to a convenient directory, forming the master installation image. For this guide, setup the install tree to be /mnt/distros/fc4/i386.
# mkdir -p /mnt/distros/fc4/i386
# mkdir /mnt/loopback
# cd /mnt/distros/fc4/i386
# for i in 1 2 3 4
do
mount /mnt/isos/fc4/i386/FC4-i386-disc${i}.iso /mnt/loopback -o loop
cp -af /mnt/loopback/* .
umount /mnt/loopback
done
If one is tight for disk space, the ISO images can be discarded now, although its a wise idea to at least burn them to a CD for future reference. The final step is to make the install tree available via Apache. This can be achieved by dropping a single file into /etc/httpd/conf.d containing:
Alias /distros/ /mnt/distros/ <Location /distros> Allow from all Options +Indexes </Location>
Thomas Chung provides RPM builds of QEmu and the kernel accelerator module for Fedora Core 4 systems, so download the RPMs which match the kernel currently running in your host system, and install them now
# cd /root # wget -r -l 1 http://fedoranews.org/tchung/qemu/fc4/0.7.2/2.6.13_1.1532_FC4/ # cd fedoranews.org/tchung/qemu/0.7.2/2.6.13_1.1532_FC4 # rpm -ivh *.i386.rpm
If one plans to install multiple guest OS images, its worthwhile creating kickstart file containing the desired anconda options. The system-config-kickstart program provides a simple GUI for doing this. At the minimum change the following options:
Save the finished kickstart file to /mnt/distros/fc4/i386/qemu-ks.cfg
It is now time to create a QEmu guest OS. First of all prepare a disk image to use as the root filesystem, making sure its large enough to hold the package selection choose in the kickstart. As a rule of thumb, 1 GB is enough for a minimal install, 2 GB if X is added, and 6 GB for everything.
# mkdir /mnt/guests # cd /mnt/guests # qemu-img qemu-fc4-i386.img 2G
Now launch a virtual machine with this root filesystem, and booting the anaconda CD installer. The VM must have at least 256 MB of memory, otherwise anaconda will crash during installation.
# qemu -user-net -hda qemu-fc4-i386.img -boot d -cdrom /mnt/distros/fc4/i386/images/boot.iso -m 256
When the initial anaconda boot prompt is displayed, request a kickstart install by specifying
linux ks=http://10.0.2.2/distros/fc4/i386/qemu-ks.cfg
If all goes to plan, in about 10-15 minutes time anaconda will have completed installation of the guest OS. When it is done, shutdown the virtual machine – simply close its window once anaconda has shutdown.
Now, to use the virtual machine one can simply launch it with
# qemu -user-net -m 256m qemu-fc4-i386.img
With the userspace networking support, when inside the guest, the host machine can be accessed using the IP address 10.0.2.2. For convenience it may be easier to run the guest OS headless, and access it over a SSH connection. To do this, launch it with the following options
# qemu -user-net -m 256m -nographic -redir tcp:8000::22
The once booted, the guest can be accessed via SSH on port 8000
# ssh -p 8000 localhost
This posting provides a tutorial on providing a DBus service using the
Perl Net::DBus application bindings, which will later appear in as a tutorial
in the POD document for Net::DBus. The examples in this document will
be based on the code from the Music::Player distribution, which is a simple DBus service providing a music track player.
The first step in creating an object is to create a new package which
inherits from Net::DBus::Object. The Music::Player::Manager object
provides an API for managing the collection of music player backends for
different track types. To start with, lets create the skeleton of the
package & its constructor. The constructor of the super type,
Net::DBus::Object expects to be given to parameters, a handle to the
Net::DBus::Service owning the object, and a path under which the object
shall be exported. Since the manager class is intended to be a singleton
object, we can hard code the path to it within the constructor:
package Music::Player::Manager;
use base qw(Net::DBus);
sub new {
my $class = shift;
my $service = shift;
my $self = $class->SUPER::new($service, "/music/player/manager");
bless $self, $class;
return $self;
}
1;
Now, as mentioned, the manager with handle a number of different player
backends. So we need to provide methods for registering new backends,
and querying for backends capable of playing a particular file type. So
modifying the above code we add a hash table in the constructor, to
store the backends:
sub new {
my $class = shift;
my $service = shift;
my $self = $class->SUPER::new($service, "/music/player/manager");
$self->{backends} = {};
bless $self, $class;
return $self;
}
And now a method to register a new backend. This takes a Perl module
name and uses it to instantiate a backend. Since the backends are also
going to be DBus objects, we need to pass in a reference to the service
we are attached to, along with a path under which to register the
backend. We use the “get_service” method to retreieve a reference to the
service the manager is attached to, and attach the player backend to
this same service: When a method on DBus object is invoked, the first
parameter is the object reference ($self), and the remainder are the
parameters provided to the method call. Thus writing a method
implementation on a DBUs is really no different to normal object
oriented Perl (cf perltoot):
sub register_backend {
my $self = shift;
my $name = shift;
my $module = shift;
eval "use $module";
if ($@) {
die "cannot load backend $module: $@" ;
}
$self->{backends} = $module->new($self->get_service,
"/music/player/backend/$name");
}
Looking at this one might wonder what happens if the “die” method is
triggered. In such a scenario, rather than terminating the service
process, the error will be caught and propagated back to the remote
caller to deal with.
The player backends provide a method “get_track_types” which returns an
array reference of the music track types they support. We can use this
method to provide an API to allow easy retrieval of a backend for a
particular track type. This method will return a path with which the
backend object can be accessed
sub find_backend {
my $self = shift;
my $extension = shift;
foreach my $name (keys %{$self->{backends}}) {
my $backend = $self->{backends}->{$name};
foreach my $type (@{$backend->get_track_types}) {
if ($type eq $extension) {
return $backend->get_object_path;
}
}
}
die "no backend for type $extension";
}
Lets take a quick moment to consider how this method would be used to
play a music track. Now, we have an MP3 file which
we wish to play, so we search for the path to a backend, then retrieve
the object for it, and play the track:
...get the music player service...
# Ask for a path to a player for mp3 files
my $path = $service->find_backend("mp3");
# $path now contains '/music/player/backend/mpg123'
# and we can get the backend object
my $backend = $service->get_object($path);
# and finally play the track
$backend->play("/vol/music/beck/guero/09-scarecrow.mp3");
The code above is a complete working object, ready to be registered with
a service, and since the parameters and return values for the two
methods are both simple strings we could stop there. In some cases,
however, one might want to be more specific about data types expected
for parameters, for example signed vs unsigned integers. Adding explicit
data typing also makes interaction with other programming languages more
reliable. Providing explicit data type defintions for exported method is
known in the DBus world as “Introspection”, and it makes life much more
reliable for users of one’s service whom may be using a strongly typed
language such as C.
The first step in providing introspection data for a DBus object in
Perl, is to specify the name of the interface provided by the object.
This is typically a period separated string, by convention containing
the domain name of the application as its first component. Since most
Perl modules end up living on CPAN, one might use “org.cpan” as the
first component, followed by the package name of the module (replacing
:: with .), eg “org.cpan.music.player.manager”. If it is not planned to
host the module on CPAN, a personal/project domain might be used eg
“com.berrange.music.player.manager”. The interface for an object is
defined by loading the Net::DBus::Exporter module, providing the
interface as its first parameter. So the earlier code example would be
modified to look like:
package Music::Player::Manager;
use base qw(Net::DBus);
use Net::DBus::Exporter qw(com.berrange.music.player.manager)
Next up, it is neccessary to provide data types for the parameters and
return values of the methods. The Net::DBus::Exporter module provides a
method “dbus_method” for this purpose, which takes three parameter, the
name of the method being exported, an array reference of parameter
types, and an array reference of return types (the latter can be omitted
if there are no return values). This can be called at any point in the
module’s code, but by convention it is preferrable to associate calls to
“dbus_method” with the actual method implementation, thus:
dbus_method("register_backend", ["string", "string"]);
sub register_backend {
my $self = shift;
my $name = shift;
my $module = shift;
.. snipped rest of method body ...
}
And, thus:
dbus_method("find_backend", ["string"], ["string"])
sub find_backend {
my $self = shift;
my $extension = shift;
... snip method body...
}
Now that the objects have been written, it is time to define a service.
A service is nothing more than a well known name for a given API
contract. A contract can be thought of as a definition of a list of
object paths, and the corresponding interfaces they provide. So, someone
else could come along a provide an alternate music player implementation
using the Python or QT bindings for DBus, and if they provided the same
set of object paths & interfaces, they could justifiably register the
same service on the bus.
The Net::DBus::Service module provides the means to register a service.
Its constructor expects a reference to the bus object (an instance of
Net::DBus), along with the name of the service. As with interface names,
the first component of a service name is usually derived from a domain
name, and then suffixed with the name of the application, in our example
forming “org.cpan.Music.Player”. While some objects will be created on
the fly during execution of the application, others are created upon
initial startup. The music player manager object created earlier in this
tutorial is an example of the latter. It is typical to instantiate and
register these objects in the constructor for the service. Thus a
service object for the music player application would look like:
package Music::Player;
use base qw(Net::DBus::Service);
sub new {
my $class = shift;
my $bus = shift;
my $self = $class->SUPER::new($bus, "org.cpan.music.player");
bless $self, $class;
$self->{manager} = Music::Player::Manager->new($self);
return $self;
}
The final step in getting our service up and running is to connect it to
the bus. This brings up an interesting conundrum, does one export the
service on the system bus (shared by all users & processes on the
machine), or the session bus (one per user logged into a machine). In
some cases the answer, with only one of the two buses conceptually
making sense. In other cases, however, both the session & system bus are
valid. In the former one would use the “session” or
Net::DBus to get a handle to the desired bus, while in the latter case,
the “find” method would be used. This applies a heuristic to determine
the correct bus based on execution environment. In the case of the music
player, either bus is relevant, so the code to connect the service to
the bus would look like:
use Net::DBus;
my $bus = Net::DBus->find;
my $player = Music::Player->new($bus);
With the service attached to the bus, it is merely neccessary to run the
main event processing loop to listen out for & handle incoming DBus
messages. So the above code is modified to start a simple reactor:
use Net::DBus;
use Net::DBus::Reactor;
my $bus = Net::DBus->find;
my $player = Music::Player->new($bus);
Net::DBus::Reactor->main->run;
exit 0;
Saving this code into a script “/usr/bin/music-player.pl”, coding is
complete and the service ready for use by clients on the bus.
One might now wonder how best to start the service, particularly if it
is a service capable of running on both the system and session buses.
DBus has the answer in the concept of “activation”. What happens is that
when a client on the bus attempts to call a method, or register a signal
handler against, a service not currently running, it will first try and
start the service. Service’s which wish to participate in this process
merely need stick a simple service definition file into the directoy
“/usr/share/dbus-1/services”. The file should be named to match the
service name, with the file extension “.service” appended. eg,
“/usr/share/dbus-1/services/org.cpan.music.player.service” The file
contains two keys, first the name of the service, and second the name of
the executable used to run the service, or in this case the Perl script.
So, for our simple service the data file would contain:
[D-BUS Service]
Name=org.cpan.music.player
Exec=/usr/bin/music-player.pl