Innovation in virtualization management tools: libvirt for KVM and Xen in Fedora 7

For the last couple of years all the hype wrt open source virtualization has been about Xen. Unfortunately after several years Xen is still not upstream in LKML, the main codebase being a huge out of tree patch persistently stuck on obsoleted kernel versions. The Xen paravirt_ops implementation is showing promise, but its a long way off being a full solution since it doesn’t provide Dom0 or ia64/ppc/x86_64 yet. Then out of nowhere, 6 months ago, a newer contender arrived in the form of KVM almost immediately finding itself merged upstream. Now you can’t claim to be offerring state of the art virtualization without including both KVM and Xen. We had to have KVM in Fedora 7. With excellant forsight, when working to integrate Xen in Fedora Core 5, Daniel Veillard had the idea to create a technology independant management library for virtualization platforms. This is libvirt. The core idea was to provide a stable library API for application developers to target, insulating them from implementation specific APIs in the changing base virtualization platform. This API would be LGPL to allow both both the community and 3rd party vendors build applications.

On top of libvirt a suite of applications are evolving – virt-manager, virt-install and cobbler/koan – to name the three most popular thus far. Fast forward to Fedora 7 when we’re looking to introduce support for KVM into the distribution. We could simply have taken the approach of throwing in the kernel modules & QEMU code for KVM, sending out a fancy press release & letting users figure out the gory details. With libvirt though, we were in a position to do more…much, much more. So the precious few months between the end of our RHEL-5 GA work and the Fedora 7 release were spent hacking on libvirt to add in support for managing QEMU and KVM virtual machines.

The results speak for themselves. In Fedora 7 you can fire up the tools like virt-manager & virt-install and manage KVM virtual machines in exactly the same way you managed Xen virtual machines. You are free from being locked into Xen. You avoided having to learn new command sets. You are able to pick the best open source virtualization technology to accomplish the task at hand. This is Fedora innovation & vision at its best! No other distribution even comes close…yet… None of this technology has any serious distribution specific logic & we want everyone to reap the rewards. Thus we actively work with developers from other distributions, in particular Open Solaris and Debian to help them integrate (& thus take advantage of) libvirt and the applications built upon it.

So if you’re using Fedora 7 what are some of the things to look out for….

Choosing which hypervisor to use

Both KVM and Xen require root privileges since they need to talk to various kernel devices & drivers. Not everyone has root on their machine. With KVM you can allow unprivileged users the ability to create VMs by changing the ownership on /dev/kvm. The plain unaccelerated QEMU can be used a general purpose emulator too without any elevated privileges. libvirt uses ‘hypervisor URIs’ to choose between backend implementations.

• xen (or no explicit URI): this connects to Xen. root has full access, non-root has read only access
• qemu:///system: this connects to the main QEMU/KVM backend driver. root has full access, non-root has read only access.
• qemu:///session: this connects to the per-user private QEMU backend driver. Full access irrespective of user, since there is one instance per-user account.

All libvirt tools accept a --connect argument to choose the hypervisor URI to use. virt-manager & virt-install will try to pick either xen or qemu://system depending on which kernel you’re running. You can always o an explicit argument. So to use virsh as a non-root user with the per-user QEMU instance, virsh --connect qemu:///session.

Using virsh instead of xm

You may be familiar with Xen’s xm command line tool. The capabilities of this valuable administrative tool are also in virsh. The benefit being that it works with Xen and KVM/QEMU, as well as any future platforms libvirt is ported to. To use with Xen, just run virsh with no arguments. To use with KVM/QEMU add in --connect qemu:///system. The virsh help command shows the (huge!) list of commands at your disposal. Oh the VIRSH_DEFAULT_CONNECT_URI environment variable can be used if you don’t want to specify --connect everytime.

Changing the config of an existing guest

Every virtualization technology has a difference configuration file format (or even none at all – relying on command line args in QEMU case). As expected libvirt provides a consistent way to change the configuration of a guest virtual machine. Every guest has an XML document describing its setup. You can see this configuration by using the virsh dumpxml [guest] command. Save this XML to a file, edit it in VI and then use it to update the master config by running virsh define [xmlfile]. There’s a page describing the various XML elements. Afraid of looking at XML ? Well virsh provides commands for changing simple things like memory & CPU count, while virt-manager also allows disk and network devices to be added/removed.

Connecting virtual machines to the network

libvirt allows for many ways to connect a virtual machine to the real network, two of them stand out as the common case. First the traditional Xen approach is to setup a bridge device in the host OS with the physical ethernet device enslaved, along with zero or more guest interfaces. Xen ships with a shell script which munges networking on startup to put eth0 into a bridge, though it is preferrable to just use the standard initscripts to define the bridge & device. In libvirt we call this a “shared physical device” setup – your physical device is shared between the guest and the host in effect. Bridging works well if you are on a permanently wired LAN. It fails miserably if you have a dynamic network setup as typically found on laptops. So the second approach is to have a bridge device not connected to any physical, attach guests to this bridge, and then NAT their traffic out to the real physical device. In libvirt we call this a “virtual network” since its not seen outside of a single host, all LAN traffic appears to orginate from the single host. David Lutterkort describes the setup in a little more detail

We try to make the choice easy in virt-install – if there is a bridge device with a physical interface enslaved we setup a ‘shared physical device’ style config, otherwise we fallback to a ‘virtual network’ config. You can be explicit when choosing between the two using the --network bridge:eth0 (connect to bridge called eth0) or --network network:default (connect to virtual network called ‘default’). Virt-manager has an extra screen in the VM creation wizard to let you choose the networking setup.

I can only just scratch the surface of this in a blog posting, and of course the work on libvirt is far from complete. A major focus for Fedora 8 will be providing secure remote management. In parallel there’s ongoing work to expand coverage of virtual hardware configuration to audio, USB, more advanced serial consoles. We’d also welcome contributions from anyone interested in adding support for OpenVZ, User Mode Linux, VMWare and Virtual Box to libvirt…