ANNOUNCE: New libvirt project Go language bindings

I’m happy to announce that the libvirt project is now supporting Go language bindings as a primary deliverable, joining Python and Perl, as language bindings with 100% API coverage of libvirt C library. The master repository is available on the libvirt GIT server, but it is expected that Go projects will consume it via an import of the github mirror, since the Go ecosystem is heavilty github focused (e.g. godoc.org can’t produce docs for stuff hosted on libvirt.org git)

import (
    libvirt "github.com/libvirt/libvirt-go"
)

conn, err := libvirt.NewConnect("qemu:///system")
...

API documentation is available on the godoc website.

For a while now libvirt has relied on 3rd parties to provide Go language bindings. The one most people use was first created by Alex Zorin and then taken over by Kyle Kelly. There’s been a lot of excellent work put into these bindings, however, the API coverage largely stops at what was available in libvirt 1.2.2, with the exception of a few APIs from libvirt 1.2.14 which have to be enabled via Go build tags. Libvirt is now working on version 3.0.0 and there have been many APIs added in that time, not to mention enums and other constants. Comparing the current libvirt-go API coverage against what the libvirt C library exposes reveals 163 missing functions (out of 476 total), 367 missing enum constants (out of 847 total) and 165 missing macro constants (out of 200 total). IOW while there is alot already implemented, there was still a very long way to go.

Initially I intended to contribute patches to address the missing API coverage to the existing libvirt-go bindings. In looking at the code though I had some concerns about the way some of the APIs had been exposed to Go. In the libvirt C library there are a set of APIs which accept or return a “virTypedParameterPtr” array, for cases where we need APIs to be easily extensible to handle additions of an arbitrary extra data fields in the future. The way these APIs work is one of the most ugly and unpleasant parts of the C API and thus in language bindings we never expose the virTypedParameter concept directly, but instead map it into a more suitable language specific data structure. In Perl and Python this meant mapping them to hash tables, which gives application developers a language friendly way to interact with the APIs. Unfortunately the current Go API bindings have exposed the virTypedParameter concept directly to Go and since Go does not support unions, the result is arguably even more unpleasant in Go than it already is in C. The second concern is with the way events are exposed to Go – in the C layer we have different callbacks that are needed for each event type, but have one method for registering callbacks, requiring an ugly type cast. This was again exposed directly in Go, meaning that the Go compiler can’t do strong type checking on the callback registration, instead only doing a runtime check at time of event dispatch. There were some other minor concerns about the Go API mapping, such as fact that it needlessly exposed the “vir” prefix on all methods & constants despite already being in a “libvirt” package namespace, returning of a struct instead of pointer to a struct for objects. Understandably the current maintainer had a desire to keep API compatibility going forward, so the decision was made to fork the existing libvirt-go codebase. This allowed us to take advantage of all the work put in so far, while fixing the design problems, and also extending them to have 100% API coverage. The idea is that applications can then decide to opt-in to the new Go binding at a point in time where they’re ready to adapt their code to the API changes.

For users of the existing libvirt Go binding, converting to the new official libvirt Go binding requires a little bit of work, but nothing too serious and will simplify the code if using any of the typed parameter methods. The changes are roughly as follows:

• The “VIR_” prefix is dropped from all constants. eg libvirt.VIR_DOMAIN_METADATA_DESCRIPTION because libvirt.DOMAIN_METADATA_DESCRIPTION
• The “vir” prefix is dropped from all types. eg libvirt.virDomain becomes libvirt.Domain
• Methods returning objects now return a pointer eg “* Domain” instead of “Domain”, allowing us to return the usual “nil” constant on error, instead of a struct with no underlying libvirt connection
• The domain events DomainEventRegister method has been replaced by a separate method for each event type. eg DomainEventLifecycleRegister, DomainEventRebootRegister, etc giving compile time type checking of callbacks
• The domain events API now accepts a single callback, instead of taking a pair of callbacks – the caller can create an anonymous function to invoke multiple things if required.
• Methods accepting or returning typed parameters now have a formal struct defined to expose all the parameters in a manner that allows direct access without type casts and enables normal Go compile time type checking. eg the Domain.GetBlockIOTune method returns a DomainBlockIoTuneParameters struct
• It is no longer necessary to use Go compiler build tags to access functionality in different libvirt versions. Through the magic of conditional compilation, the binding will transparently build against every libvirt version from 1.2.0 through 3.0.0
• The binding can find libvirt via pkg-config making it easy to compile against a libvirt installed to a non-standard location by simply setting “PKG_CONFIG_PATH”
• There is 100% coverage of all APIs ^[1], constants and macros, verified by the libvirt CI system, so that it always keeps up with GIT master of the Libvirt C library.
• The error callback concept is removed from the binding as this is deprecated by libvirt due to not being thread safe. It was also redundant since every method already directly returns an error object.
• There are now explicit types defined for all enums and methods which take flags or enums now use these types instead of “uint32”, again allowing stronger compiler type checking

With the exception of the typed parameter changes adapting existing apps should be a largely boring mechanical conversion to just adapt to the renames.

Again, without the effort put in by Alex Zorin and Kyle Kelly & other community contributors, creation of these new libvirt-go bindings would have taken at least 4-5 weeks instead of the 2 weeks effort put into this. So there’s a huge debt owed to all the people who previously contributed to libvirt Go bindings. I hope that having these new bindings with guaranteed 100% API coverage will be of benefit to the Go community going forward.

[1] At time of writing this is a slight lie, as i’ve not quite finished the virStream and virEvent callback method bindings, but this will be done shortly.