Libvirt has long supported use of TLS for its remote API service, using the gnutls library as its backend. When negotiating a TLS session, there are a huge number of possible algorithms that could be used and the client & server need to decide on the best one, where “best” is commonly some notion of “most secure”. The preference for negotiation is expressed by simply having an list of possible algorithms, sorted best to worst, and the client & server choose the first matching entry in their respective lists. Historically libvirt has not expressed any interest in the handshake priority configuration, simply delegating the decision to the gnutls library on that basis that its developers knew better than libvirt developers which are best. In gnutls terminology, this means that libvirt has historically used the “DEFAULT” priority string.
The past year or two has seen a seemingly never ending stream of CVEs related to TLS, some of them particular to specific algorithms. The only way some of these flaws can be addressed is by discontinuing use of the affected algorithm. The TLS library implementations have to be fairly conservative in dropping algorithms, because this has an effect on consumers of the library in question. There is also potentially a significant delay between a decision to discontinue support for an algorithm, and updated libraries being deployed to hosts. To address this Fedora 21 introduced the ability to define the algorithm priority strings in host configuration files, outside of the library code. This system administrators can edit a file /etc/crypto-policies/config to change the algorithm priority for all apps using TLS on the host. After editting this file, the update-crypto-policies command is run to generate the library specific configuration files. For example, it populates /etc/crypto-policies/back-ends/gnutls.config In gnutls use of this file is enabled by specifying that an application wants to use the “@SYSTEM” priority string.
This is a good step forward, as it takes the configuration out of source code and into data files, but it has limited flexibility because it applies to all apps on the host. There can be two apps on a host which have mutually incompatible views about what the best algorithm priority is. For example, a web browser will want to be fairly conservative in dropping algorithms to avoid breaking access to countless websites. An application like libvirtd though, where there is a well known set of servers and clients to connect in any site, can be fairly aggressive in only supporting the very best algorithms. What is desired is a way to override the algorithm priorty per application. Now of course this can easily be done via the application’s own configuration file, and so libvirt has added a new parameter “tls_priority” to /etc/libvirt/libvirtd.conf
The downside of using the application’s own configuration, is that the system administrator has to go hunting through many different files to update each application. It is much nicer to have a central location where the TLS priority settings for all applications can be controlled. What is desired is a way for libvirt to be built such that it can tell gnutls to first look for a libvirt specific priority string, and then fallback to the global priority string. To address this patches were written for GNUTLS to extend its priority string syntax. It is now possible to for libvirt to pass “@LIBVIRT,SYSTEM” to gnutls as the priority. It will thus read /etc/crypto-policies/back-ends/gnutls.config first looking for an entry matching “LIBVIRT” and then looking for an entry matching “SYSTEM“. To go along with the gnutls change, there is also an enhancement to the update-crypto-policies tool to allow application specific entries to be included when generating the /etc/crypto-policies/back-ends/gnutls.config file. It is thus possible to configure the libvirt priority string by simply creating a file /etc/crypto-policies/local.d/gnutls-libvirt.config containing the desired string and re-running update-crypto-policies.
In summary, the libvirt default priority settings are now:
- RHEL-6/7 –
NORMAL – a string hard coded in gnutls at build time
- Fedora < 25 -
@SYSTEM – a priority level defined by sysadmin based on /etc/crypto-policies/config
- Fedora >= 25 –
@LIBVIRT,SYSTEM – a raw priority string defined in /etc/crypto-policies/local.d/gnutls-libvirt.config, falling back to /etc/crypto-policies/config if not present.
In all cases it is still possible to customize in /etc/libvirt/libvirtd.conf via the tls_priority setting, but it is is recommended to use the global system /etc/crypto-policies facility where possible.
The current previous libvirt website design dated from circa 2008 just a few years after the libvirt project started. We have grown alot of content since that time, but the overall styling and layout of the libvirt website has not substantially changed. Compared to websites for more recently launched projects, libvirt was starting to look rather outdated. So I spent a little time to come up with a new design for the libvirt website to bring it into the modern era. There were two core aspects to the new design, simplify the layout and navigation, and implement new branding.
From the layout / navigation POV we have killed the massive expanding menu that was on the left hand side of every page. It was not serving its purpose very effectively since it was requiring far too many clicks & page loads to access some of the most frequently needed content. The front page now has direct links to key pieces of content (as identified from our web access stats), while the rest of the pages are directly visible in a flat menu on the “docs” page. The download page has been overhauled to recognise the fact that libvirt is shipping more than just the core C library – we have language bindings, object model mappings, docs and test suites. Also new is a page directly targeting new contributors with information about how to get involved in the project and the kind of help we’re looking for. The final notable change is used of some jquery magic to pull in a feed of blog posts to the site front page.
From the branding POV, we have taken the opportunity to re-create the project logo. We sadly lost the original master vector artwork used to produce the libvirt logo eons ago, so only had a png file of it in certain resolutions. When creating docbook content, we did have a new SVG created that was intended to mirror the original logo, but it was quite crudely drawn. None the less it was a useful basis to start from to create new master logo graphics. As a result we now have an attractively rendered new logo for the project, available in two variants – a standard square(-ish) format
and in a horizontal banner format
With the new logo prepared, we took the colour palette and font used in the graphic and applied both to the main website content, bringing together a consistent style.
Libvirt website v1 (2006-2008)
Libvirt website v2 (2008-2016)
Libvirt website v3 (2016-)
I am pleased to announce that a new release of the libvirt-glib package, version 1.0.0, is now available from
https://libvirt.org/sources/glib/
The packages are GPG signed with
Key fingerprint: DAF3 A6FD B26B 6291 2D0E 8E3F BE86 EBB4 1510 4FDF (4096R)
Changes in this release:
- Switch to new release numbering scheme, major digit incremented each year, minor for each release, micro for stable branches (if any)
- Fix Libs.private variable in pkg-config file
- Fix git introspection warnings
- Add ability to set SPICE gl property
- Add support for virtio video model
- Add support for 3d accel property
- Add support for querying video model
- Add support for host device config for PCI devs
- Add docs for more APIs
- Avoid unused variable warnings
- Fix check for libvirt optional features to use pkg-config
- Delete manually written python binding. All apps should use PyGObject with gobject introspection.
- Allow schema to be NULL on config objects
- Preserve unknown devices listed in XML
- Add further test coverage
libvirt-glib comprises three distinct libraries:
- libvirt-glib – Integrate with the GLib event loop and error handling
- libvirt-gconfig – Representation of libvirt XML documents as GObjects
- libvirt-gobject – Mapping of libvirt APIs into the GObject type system
NB: While libvirt aims to be API/ABI stable forever, with libvirt-glib we are not currently guaranteeing that libvirt-glib libraries are permanently API/ABI stable. That said we do not expect to break the API/ABI for the forseeable future and will always strive avoid it.
Follow up comments about libvirt-glib should be directed to the regular libvir-list@redhat.com development list.
Thanks to all the people involved in contributing to this release.
This blog is part 7 of a series I am writing about work I’ve completed over the past few releases to improve QEMU security related features.
The live migration feature in QEMU allows a running VM to be moved from one host to another with no noticeable interruption in service and minimal performance impact. The live migration data stream will contain a serialized copy of state of all emulated devices, along with all the guest RAM. In some versions of QEMU it is also used to transfer disk image content, but in modern QEMU use of the NBD protocol is preferred for this purpose. The guest RAM in particular can contain sensitive data that needs to be protected against any would be attackers on the network between source and target hosts. There are a number of ways to provide such security using external tools/services including VPNs, IPsec, SSH/stunnel tunnelling. The libvirtd daemon often already has a secure connection between the source and destination hosts for its own purposes, so many years back support was added to libvirt to automatically tunnel the live migration data stream over libvirt’s own secure connection. This solved both the encryption and authentication problems at once, but there are some downsides to this approach. Tunnelling the connection means extra data copies for the live migration traffic and when we look at guests with RAM many GB in size, the number of data copies will start to matter. The libvirt tunnel only supports a tunnelling of a single data connection and in future QEMU may well wish to use multiple TCP connections for the migration data stream to improve performance of post-copy. The use of NBD for storage migration is not supported with tunnelling via libvirt, since it would require extra connections too. IOW while tunnelling over libvirt was a useful short term hack to provide security, it has outlived its practicality.
It is clear that QEMU needs to support TLS encryption natively on its live migration connections. The QEMU migration code has historically had its own distinct I/O layer called QEMUFile which mixes up tracking of migration state with the connection establishment and I/O transfer support. As mentioned in previous blog post, QEMU now has a general purpose I/O channel framework, so the bulk of the work involved converting the migration code over to use the QIOChannel classes and APIs, which greatly reduced the amount of code in the QEMU migration/ sub-folder as well as simplifying it somewhat. The TLS support involves the addition of two new parameters to the migration code. First the “tls-creds” parameter provides the ID of a previously created TLS credential object, thus enabling use of TLS on the migration channel. This must be set on both the source and target QEMU’s involved in the migration.
On the target host, QEMU would be launched with a set of TLS credentials for a server endpoint:
$ qemu-system-x86_64 -monitor stdio -incoming defer \
-object tls-creds-x509,dir=/home/berrange/security/qemutls,endpoint=server,id=tls0 \
...other args...
To enable incoming TLS migration 2 monitor commands are then used
(qemu) migrate_set_str_parameter tls-creds tls0
(qemu) migrate_incoming tcp:myhostname:9000
On the source host, QEMU is launched in a similar manner but using client endpoint credentials
$ qemu-system-x86_64 -monitor stdio \
-object tls-creds-x509,dir=/home/berrange/security/qemutls,endpoint=client,id=tls0 \
...other args...
To enable outgoing TLS migration 2 monitor commands are then used
(qemu) migrate_set_str_parameter tls-creds tls0
(qemu) migrate tcp:otherhostname:9000
The migration code supports a number of different protocols besides just “tcp:“. In particular it allows an “fd:” protocol to tell QEMU to use a passed-in file descriptor, and an “exec:” protocol to tell QEMU to launch an external command to tunnel the connection. It is desirable to be able to use TLS with these protocols too, but when using TLS the client QEMU needs to know the hostname of the target QEMU in order to correctly validate the x509 certificate it receives. Thus, a second “tls-hostname” parameter was added to allow QEMU to be informed of the hostname to use for x509 certificate validation when using a non-tcp migration protocol. This can be set on the source QEMU prior to starting the migration using the “migrate_set_str_parameter” monitor command
(qemu) migrate_set_str_parameter tls-hostname myhost.mydomain
This feature has been under development for a while and finally merged into QEMU GIT early in the 2.7.0 development cycle, so will be available for use when 2.7.0 is released in a few weeks. With the arrival of the 2.7.0 release there will finally be TLS support across all QEMU host services where TCP connections are commonly used, namely VNC, SPICE, NBD, migration and character devices.
In this blog series:
This blog is part 6 of a series I am writing about work I’ve completed over the past few releases to improve QEMU security related features.
A number of QEMU device models and objects use a character devices for providing connectivity with the outside world, including the QEMU monitor, serial ports, parallel ports, virtio serial channels, RNG EGD object, CCID smartcard passthrough, IPMI device, USB device redirection and vhost-user. While some of these will only ever need a character device configured with local connectivity, some will certainly need to make use of TCP connections to remote hosts. Historically these connections have always been entirely in clear text, which is unacceptable in the modern hostile network environment where even internal networks cannot be trusted. Clearly the QEMU character device code requires the ability to use TLS for encrypting sensitive data and providing some level of authentication on connections.
The QEMU character device code was mostly using GLib’s GIOChannel framework for doing I/O but this has a number of unsatisfactory limitations. It can not do vectored I/O, is not easily extensible and does not concern itself at all with initial connection establishment. These are all reasons why the QIOChannel framework was added to QEMU. So the first step in supporting TLS on character devices was to convert the code over to use QIOChannel instead of GIOChannel. With that done, adding in support for TLS was quite straightforward, merely requiring addition of a new configuration property (“tls-creds“) to set the desired TLS credentials.
For example to run a QEMU VM with a serial port listening on IP 10.0.01, port 9000, acting as a TLS server:
$ qemu-system-x86_64 \
-object tls-creds-x509,id=tls0,endpoint=server,dir=/home/berrange/qemutls \
-chardev socket,id=s0,host=10.0.0.1,port=9000,tls-creds=tls0,server \
-device isa-serial,chardev=s0
...other QEMU options...
It is possible test connectivity to this TLS server using the gnutls-cli tool
$ gnutls-cli --priority=NORMAL -p 9000 \
--x509cafile=/home/berrange/security/qemutls/ca-cert.pem \
127.0.0.1
In the above example, QEMU was running as a TCP server, and acting as the TLS server endpoint, but this matching is not required. It is valid to configure it to run as a TLS client if desired, though this would be somewhat uncommon.
Of course you can connect 2 QEMU VMs together, both using TLS. Assuming the above QEMU is still running, we can launch a second QEMU connecting to it with
$ qemu-system-x86_64 \
-object tls-creds-x509,id=tls0,endpoint=client,dir=/home/berrange/qemutls \
-chardev socket,id=s0,host=10.0.0.1,port=9000,tls-creds=tls0 \
-device isa-serial,chardev=s0
...other QEMU options...
Notice, we’ve changed the “endpoint” and removed the “server” option, so this second QEMU runs as a TCP client and acts as the TLS client endpoint.
This feature is available since the QEMU 2.6.0 release a few months ago.
In this blog series:
