Providing IPv6 connectivity to virtual guests with libvirt and KVM

With World IPv6 Day last week there have been a few people trying to setup IPv6 connectivity for their virtual guests with libvirt and KVM. For those whose guests are using bridged networking to their LAN, there is really not much to say on the topic. If your LAN has IPv6 enabled and your virtualization host is getting a IPv6 address, then your guests can get IPv6 addresses in exactly the same manner, since they appear directly on the LAN. For those who are using routed / NATed networking with KVM, via the “default” virtual network libvirt creates out of the box, there is a little more work todo. That is what this blog posting will attempt to illustrate.

Network architecture

Before continuing further, it probably helps to have a simple network architecture diagram

• LAN Router A: This is the machine which provides IPv6 connectivity to machines on the wired LAN. In my case this is a LinkSys WRT 54GL, running OpenWRT, with an IPv6 tunnel + subnet provided by SIXXS
• Virt Host A/B: These are the 2 hosts which run virtual machines. They are initially getting their IPv6 addresses via stateless autoconf, thanks to the RADVD program on LAN Router A sending out route advertisements.
• Virtual Network A/B: These are the libvirt default virtual networks on Virt Hosts A/B respectively. Out of the box, they both come with a IPv4 only, NAT setup under 192.168.122.0/24 using bridge virbr0.
• Virt Guest A/B: These are the virtual machines running on Virt Host A
• Virt Guest C/D: These are the virtual machines running on Virt Host B

The goal is to provide IPv6 connectivity to Virt Guests A, B, C and D, such that every guest can talk to every other guest, every other LAN host and the internet. Every LAN host, should also be able to talk to every Virt Guest, and (firewall permitting) any machines on the entire IPv6 internet should be able to connect to any guest.

Initial network configuration

The initial configuration of the LAN is as follows (NB: my real network addresses have been substituted for fake ones). Details on this kind of setup can be found in countless howtos on the web, so I won’t get into details on how to configure your LAN/tunnel. All that’s important here is the network addressing in use

LAN Router A

The network interface for the AICCU tunnel to my SIXXS POP is called ‘sixxs’, and was assigned the address 2000:beef:cafe::1/64. The ISP also assigned a nice large subnet for use on the site 2000:dead:beef::/48. The interface for the Wired LAN is called ‘br-lan’. Individual networks are usually assigned a /64, so with the /48 assigned, I have enough addresses to setup 65336 different networks within my site. Skipping the ‘0’ network, for sake of clarity, we give the Wired LAN the second network, 2000:dead:beef:1::/64, and configure the interface ‘br-lan’ with the global IPv6 address 2000:dead:beef:1::1/64. It also has a link local address of fe80::200:ff:fe00:0/64. To allow hosts on the Wired LAN to get Ipv6 addresses, the router is also running the radvd daemon, advertising the network prefix. The radvd config on the router looks like

interface br-lan
{
  AdvSendAdvert on;
  AdvManagedFlag off;
  AdvOtherConfigFlag off;

  prefix 2000:dead:beef:1::/64
  {
    AdvOnLink on;
    AdvAutonomous on;
    AdvRouterAddr off;
  };
};

The address/route configuration looks like this

# ip -6 addr
1: lo: <LOOPBACK,UP>
 inet6 ::1/128 scope host
2: br-lan: <BROADCAST,MULTICAST,UP>
 inet6 fe80::200:ff:fe00:0/64 scope link
 inet6 2000:dead:beef:1::1/64 scope global
# ip -6 route
2000:beef:cafe::/64 dev sixxs  metric 256  mtu 1280 advmss 1220
2000:dead:beef:1::/64 dev br-lan  metric 256  mtu 1500 advmss 1440
fe80::/64 dev br-lan  metric 256  mtu 1500 advmss 1440
ff00::/8 dev br-lan  metric 256  mtu 1500 advmss 1440
default via 2000:beef:cafe:1 dev sixxs  metric 1024  mtu 1280 advmss 1220

Virt Host A

The network interface for the wired LAN is called ‘eth0’ and is getting an address via stateless autoconf. The NIC has a MAC address 00:11:22:33:44:0a, so it has a link-local IPv6 address of fe80::211:22ff:fe33:440a/64, and via auto-config gets a global address of 2000:dead:beef:1:211:22ff:fe33:440a/64. The address/route configuration looks like

# ip -6 addr
1: lo:  mtu 16436
    inet6 ::1/128 scope host
3: wlan0:  mtu 1500 qlen 1000
    inet6 2000:dead:beaf:1:211:22ff:fe33:440a/64 scope global dynamic
    inet6 fe80::211:22ff:fe33:440a/64 scope link
# ip -6 route
2000:dead:beaf:1::/64 dev wlan0  proto kernel metric 256 mtu 1500 advmss 1440
fe80::/64 dev wlan0  proto kernel  metric 256 mtu 1500 advmss 1440
default via fe80::200:ff:fe00:0 dev wlan0  proto static  metric 1024 mtu 1500 advmss 1440

Virt Host B

The network interface for the wired LAN is called ‘eth0’ and is getting an address via stateless autoconf. The NIC has a MAC address 00:11:22:33:44:0b, so it has a link-local IPv6 address of fe80::211:22ff:fe33:440b/64, and via auto-config gets a global address of 2000:dead:beef:1:211:22ff:fe33:440b/64.

# ip -6 addr
1: lo:  mtu 16436
    inet6 ::1/128 scope host
3: wlan0:  mtu 1500 qlen 1000
    inet6 2000:dead:beaf:1:211:22ff:fe33:440b/64 scope global dynamic
    inet6 fe80::211:22ff:fe33:440b/64 scope link
# ip -6 route
2000:dead:beaf:1::/64 dev wlan0  proto kernel metric 256 mtu 1500 advmss 1440
fe80::/64 dev wlan0  proto kernel  metric 256 mtu 1500 advmss 1440
default via fe80::200:ff:fe00:0 dev wlan0  proto static  metric 1024 mtu 1500 advmss 1440

Adjusted network configuration

Both Virt Host A and B have a virtual network, so the first thing that needs to be done is to assign a IPv6 subnet for each of them. The /48 subnet has enough space to create 65336 /64 networks, so we just need to pick a couple more. In this we’ll assign 2000:dead:beef:a::/64 to the default virtual network on Virt Host A, and 2000:dead:beef:b::/64 to Virt Host B.

LAN Router A

The first configuration task is to tell the LAN router how each of these new networks can be reached. This requires adding a static route for each subnet, directing traffic to the link local address of the respective Virt Host. You might think you can use the global address of the virt host, rather than its link local address. I had tried that at first, but strange things happened, so sticking to the link local addresses seems best

2000:dead:beef:a::/64 via fe80::211:22ff:fe33:440a dev br-lan  metric 256  mtu 1500 advmss 1440
2000:dead:beef:b::/64 via fe80::211:22ff:fe33:440b dev br-lan  metric 256  mtu 1500 advmss 1440

One other thing to beware of is that the ‘ip6tables’ FORWARD chain may have a rule which prevents forwarding of traffic. Make sure traffic can flow to/from the 2 new subnets. In my case I added a generic rule that allows any traffic that originates on br-lan, to be sent back on br-lan. Traffic from ‘sixxs’ (the public internet) is only allowed in if it is related to an existing connection, or a whitelisted host I own.

Virt Host A

Remember how all hosts on the wired LAN can automatically configure their own IPv6 addresses / routes using stateless autoconf, thanks to radvd. Well, unfortunately, now that the virt host is going to start routing traffic to/from the guest you can’t use autoconf anymore :-(  So the first job is to alter the host eth0 configuration, so that it uses a statically configured IPv6 address for eth0, or gets an address from DHCPv6. If you skip this  bit, things may appear to work fine at first, but next time you reconnect to the LAN, autoconf will fail.

Now that the host is not using autoconf, it is time to reconfigure libvirt to add IPv6 to the virtual network. To do this, we stop the virtual network, and then edit its XML

# net-destroy default
# net-edit default
....vi launches..

In the editor, we need to insert a snippet of XML giving details of the subnet assigned to this host, 2000:dead:beef:a::/64. Again we pick address ‘1’ for the host interface, virbr0.

<ip family='ipv6' address='2000:dead:beef:a::1' prefix='64'/>

After exiting the editor, simply start the network again

# net-start default

If all went to plan, virbr0 will now have an IPv6 address, and a route will be present. There will also be an radvd process running on the host advertising the network.

/usr/sbin/radvd --debug 1 --config /var/lib/libvirt/radvd/default-radvd.conf --pidfile /var/run/libvirt/network/default-radvd.pid-bin

If you look at the auto-generated configuration file, it should contain something like

interface virbr0
{
  AdvSendAdvert on;
  AdvManagedFlag off;
  AdvOtherConfigFlag off;

  prefix 2a01:348:157:1::1/64
  {
    AdvOnLink on;
    AdvAutonomous on;
    AdvRouterAddr off;
  };
};

Virt Host B

As with the previous Virt Host A, the first configuration task is to switch the host eth0 from using autoconf, to a static IPv6 address configuration, or DHCPv6. Then it is simply a case of running the same virsh commands, but using this host’s subnet, 2000:dead:beef:b::/64

<ip family='ipv6' address='2000:dead:beef:b::1' prefix='64'/>

The guest setup

At this stage, it should be possible to ping & ssh to the address of the ‘virbr0’ interface of each virt host, from anywhere on the wired LAN. If this isn’t working, or if ping works, but ssh fails, then most likely the LAN router has missing/incorrect routes for the new subnets, or there is a firewall blocking traffic on the LAN router, or Virt Host A/B. In particular check the FORWARD chain in ip6tables.

Assuming, this all works though, it should now be possible to start some guests on Virt Host A / B. In this case the guest will of course have a NIC configuration that uses the ‘default’ network:

<interface type='network'>
  <mac address='52:54:00:e6:1f:01'/>
  <source network='default'/>
</interface>

Starting up this guest, autoconfiguration should take place, resulting in it getting an address based on the virtual network prefix and the MAC address

# ip -6 addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436
inet6 ::1/128 scope host
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qlen 1000
inet6 2000:dead:beef:a:5054:ff:fee6:1f01/64 scope global dynamic
inet6 fe80::5054:ff:fee6:1f01/64 scope link
# ip -6 route
2000:dead:beef:a::/64 dev eth0  proto kernel  metric 256 mtu 1500 advmss 1440
fe80::/64 dev eth0  proto kernel  metric 256  mtu 1500 advmss 1440
default via fe80::211:22ff:fe33:440a dev eth0  proto kernel  metric 1024 mtu 1500 advmss 1440

A guest running on Host A ought to be able to connect to a guest on Host B, and vica-verca, and any host on the LAN should be able to connect to any guest.