My initial article on Networking configuration for KVM had examples that where very useful if you want to connect VMs to a specific network and maybe connect those between Hypervisor hosts.

As my Home Lab has grown I've also been experimenting with OpenStack which requires a lot of nodes, but seeing as only Nova (Compute / Hypervisors) have a reason to be on tin all the rest (Keystone / Identity, Glance / Image Storage, Swift / Object Storage, Cinder / Block Storage) can be VMs running on the the main storage machine on my Lab, the network gets a bit more complicated due to needing some VMs and Hypervisors having access to raw tagged networking (they have access to multiple 802.1Q VLANs). I have 3 separate Gigabit ports on each Hyp, and some have access to the world, others are dedicated for specific uses.

Translated into Linux networking, that means that one Hyp could have a plain ordinary bridge to an Ethernet device and see tagged frames (trunked) on that, and VMs attached to that bridge can handle their own VLANs. At the same time, another VM may require to see untagged frames on a specific VLAN, and the Hyp may also be the gateway/NAT router and/or DHCP server on that VLAN.

That would mean we need a tagged (trunk) bridge, untagged (single 802.1Q VLAN) bridges and the ability to connect anything to any of them. Although Debian/Ubuntu (or derivatives) provides a very comprehensive network configuration in /etc/networking/interfaces, this type of configuration is not something that is well documented and a lot of experimenting is needed to get it right. So here's my attempt at outlining the common scenarios...

Prerequisites

As before, you will need the bridge-utils and vlan packages installed for this to work.

Basic Trunk Bridge

This is really a repeat of previously described configuration where we create a bridge against an Ethernet device (eth1 in our example). This will pass VLAN tags, so for the purposes of this we'll make it an Isolated Bridge:

auto br1
iface br1 inet manual
        bridge_ports eth1
        bridge_fd 0
        bridge_waitport 0
        # stop forwarding on this interface
        up sysctl net.ipv4.conf.${IFACE}.forwarding=0
iface br1 inet6 manual
        up sysctl net.ipv6.conf.${IFACE}.disable_ipv6=1
        # stop forwarding on this interface
        up sysctl net.ipv6.conf.${IFACE}.forwarding=0

VMs can be attached to this or physical machines (eg. Hyps) on the physical network that the bridge is attached to (remembering you need a switch that will correctly pass tagged frames) and will need to handle their own VLAN tagging with interface configurations like:

auto eth1.1001
iface eth1.1001 inet static
        address xxx.yyy.zzz.2
        netmask 255.255.255.0
        network xxx.yyy.zzz.0
        broadcast xxx.yyy.zzz.255
        gateway xxx.yyy.zzz.1
iface eth1.1001 inet6 static
        address XXXX:YYYY:ZZZZ::2
        gateway XXXX:YYYY:ZZZZ::1
        netmask 64

Other devices connected to this network (eg. a physical host) will also need to handle VLAN tagging like this, or could have a similar Isolated bridge to pass traffic to VMs.

VLAN off Bridge

This would be a scenario where there was no bridges on specific VLANs and we just want the host (eg. Hyp) to have an address on a VLAN from the Isolated Bridge above. The syntax follows that used for normal network devices, in this case we use VLAN 31 off br1 above:

auto br1.31
iface br1.31 inet static
        bridge_fd 0
        bridge_waitport 0
        address xxx.yyy.zzz.2
        netmask 255.255.255.0
        network xxx.yyy.zzz.0
        broadcast xxx.yyy.zzz.255
        gateway xxx.yyy.zzz.1
iface br1.31 inet6 static
        address XXXX:YYYY:ZZZZ::2
        gateway XXXX:YYYY:ZZZZ::1
        netmask 64

Single VLAN Bridge off Trunk Bridge

Sometimes the scenario above isn't enough and we may need to attach some VMs to a specific VLAN without the VM (perhaps untrusted) doing it's own VLAN selection. This also allows the host (eg. Hyp) to have an address on the VLAN as above, in this case we use VLAN 121, create a dummy device off the main bridge for the VLAN (br1.121) and then attach the bridge to that device:

auto br1.121
iface br1.121 inet manual

auto vlan121br
iface vlan121br inet static
        bridge_ports br1.121
        bridge_fd 0
        bridge_waitport 0
        address xxx.yyy.zzz.2
        netmask 255.255.255.0
        network xxx.yyy.zzz.0
        broadcast xxx.yyy.zzz.255
iface vlan121br inet6 static
        address XXXX:YYYY:ZZZZ::2
        netmask 64

Variants of this are possible, eg. replace the addressing with the isolation lines and you have a bridge on the VLAN isolated from the host.