Test-AutoBuild is the oldest open source project of mine that I still actually work on. The original code dates all the way back to approx the year 2000, when Richard Jones and I were working at a now defunct dot-com called BiblioTech (random archive.org historical link). Rich wrote a script called “Rolling Build” which would continuously checkout & build all our software from CVS, publishing the results as RPMs. Thankfully the company allowed the script to be open sourced under the GPLv2+ and I used it as the basis for creating a project called Test-AutoBuild in ~2004. We expanded the code to cover many different SCM tools, maintain historical archives of builds to avoid rebuilding modules if no code had changed and many other things besides. I did a couple of releases a year for while, but it has been on the backburner for the last couple of years. With the increasing number of inter-related virtualization projects using to libvirt & KVM I decided it was time to put Test-AutoBuild back into use as a build server. Yes there are many of automated build systems in existence these days I could have chosen, but I was looking for an excuse to hack on mine again :-)
There was a quiet, mostly unannounced release 1.2.3 back at the start of August, primarily fix the utterly broken GIT support I originally wrote. A couple of weeks ago, immediately before going on holiday, I uploaded release 1.2.4 to the CPAN distribution page. Aside from fixing a number of horrible bugs, the 1.2.4 release brought in the ability to rsync the build results pages to a remote server, so now it is now possible to run the automated builds inside one (or more) private virtual machines and publish the results to a separate public webserver. The second major change was the incorporation of a new theme for the Test-AutoBuild project website and the build status pages. Previously we had used a pretty lame icon of a gear wheel as the logo and a fairly plain web site style. Looking around for some better ideas I happened to come across a proposal for a Fedora 10 Theme that was never taken up. Nicu Buculei and Máirín Duffy, who produced that artwork, were generous enough to grant me permission to use the graphics for Test-AutoBuild under the CC-BY-SA 3.0 and GPLv2+ licenses. Thus for the 1.2.4 release the status pages for the automated builds have been completely restyled and the project’s main website has been similarly updated.
With the 1.2.4 release out and updated RPMs pushed into Fedora, I’m now able to publish the results of the automated builds I run for nearly all the libvirt related virtualization projects. The builder currently runs in a Fedora 14 virtual machine. The plan is to install further virtual machines running important target OS, at the very least, Debian and one of the BSDs, so we can be sure we aren’t causing regressions in our codebases. If I’m feeling adventurous I might even setup a QEMU PPC instance to run some builds on a non-x86 architecture, though that will probably be painfully slow :-)
I’ve written before about how virtualization causes pain wrt keyboard handling and about the huge number of scancode/keycode sets you have to worry about. Following on from that investigative work I completely rewrote GTK-VNC’s keycode handling, so it is able to correctly translate the keycodes it receives from GTK on Linux, Win32 and OS-X, even when running against a remote X11 server on a different platform. In doing so I made sure that the tables used for doing conversions between keycode sets were not just big arrays of magic numbers in the code, as is common practice across the kernel or QEMU codebase. Instead GTK-VNC now has a CSV file containing the unadulterated mapping data along with a simple script to split out mapping tables. This data file and script has already been reused to solve the same keycode mapping problem in SPICE-GTK.
Fast-forward a year and a libvirt developer from Fujitsu is working on a patch to wire up QEMU’s “sendkey” monitor command to a formal libvirt API. The first design question is how should the API accept the list of keys to be injected to the guest. The QEMU monitor command accepts a list of keycode names as strings, or as keycode values as hex-encoded strings. The QEMU keycode values come from what I term the “RFB” codeset, which is just the XT codeset with a slightly unusual encoding of extended keycodes. VirtualBox meanwhile has an API which wants integer keycode values, from the regular XT codeset.
One of the problems with the XT codeset is that no one can ever quite agree on what is the official way to encode extended keycodes, or whether it is even possible to encode certain types of key. There is also a usability problem with having the API require a lowlevel hardware oriented keycode set as input, in that as an application developer you might know what Win32 virtual keycode you want to generate, but have no idea what the corresponding XT keycode is. It would be preferable if you could simply directly inject a Win32 keycode to a Windows guest, or directly inject a Linux keycode to a Linux guest, etc.
After a little bit of discussion we came to the conclusion that the libvirt API should accept an array of integer keycodes, along with a enum parameter specifying what keycode set they belong to. Internally libvirt would then translate from whatever keycode set the application used, to the keycode set required by the hypervisor’s own API. Thus we got an API that looks like:
typedef enum {
VIR_KEYCODE_SET_LINUX = 0,
VIR_KEYCODE_SET_XT = 1,
VIR_KEYCODE_SET_ATSET1 = 2,
VIR_KEYCODE_SET_ATSET2 = 3,
VIR_KEYCODE_SET_ATSET3 = 4,
VIR_KEYCODE_SET_OSX = 5,
VIR_KEYCODE_SET_XT_KBD = 6,
VIR_KEYCODE_SET_USB = 7,
VIR_KEYCODE_SET_WIN32 = 8,
VIR_KEYCODE_SET_RFB = 9,
VIR_KEYCODE_SET_LAST,
} virKeycodeSet;
int virDomainSendKey(virDomainPtr domain,
unsigned int codeset,
unsigned int holdtime,
unsigned int *keycodes,
int nkeycodes,
unsigned int flags);
As with all libvirt APIs, this is also exposed in the virsh command line tool, via a new “send-key” command. As you might expect, this accepts a list of integer keycodes as parameters, along with a keycode set name. If the keycode set is omitted, we are assuming use of the Linux keycode set by default. To be slightly more user friendly though, for the Linux, Win32 & OS-X keycode sets, we also support symbolic keycode names as an alternative to the integer values. These names are simply the name of the #define constant from corresponding header file.
Some examples of how to use the new virsh command are
# send three strokes 'k', 'e', 'y', using xt codeset
virsh send-key dom --codeset xt 37 18 21
# send one stroke 'right-ctrl+C'
virsh send-key dom KEY_RIGHTCTRL KEY_C
# send a tab, held for 1 second
virsh send-key --holdtime 1000 0xf
So when interacting with virtual guests you now have a choice of how to send fake keycodes. If you have a VNC or SPICE connection directly to the guest in question, you can inject keycodes over that channel, while if you have a libvirt connection to the hypervisor you can inject keycodes over that channel.