Glen Pitt-Pladdy :: BlogHome Lab Project: Network Bridges for KVM - NAT, Host-only, Isolated | |||
With building a Home Lab system for experimenting and skilling-up on new technologies, one area there is limited comprehensive examples are is all the different types of networks you're likely to want for Lab Hypervisors. Among my scenarios is:
One requirement I have is IPv6 must be fully working in all cases, including having NAT on NAT Bridges. This is important since as I described before, when simulating a real-world deployment and possibly also mixing Hardware devices, being able to NAT IPv6 to test real IPs is useful. In my case I'm using Debian Jessie for the host, but the concepts are transferable to other distros. For this all configuration is done in /etc/network/interfaces unless specified otherwise. Also note that the names of bridges are arbitrary. They could be "br0" or "FunkyNet", the OS really doesn't care. Packages RequiredYou should have the bridge-utils package installed for this which will give the necessary tools for administrating and debugging Linux network bridges. Additionally, if you have more than one Hypervisor then you probably want to start connecting bridges up between Hyps. If that's the case then we will be using 802.1Q VLANs to do this and you will need the vlan package as well. Basic Bridge to Ethernet DeviceThis is the standard bridge that everyone knows and is very widely used and documented. I shouldn't need to put this here, but it's useful for completeness. auto br0 Devices using this Bridge are on the same network as the port device (eth0 in this example). Isolated Network Bridge (no contact with anything)This is the opposite extreme from the basic bridge and here nothing (not even the host) can communicate to this bridge. auto visol_br0 This is useful where you have another VM doing routing between bridges, networks between application tiers and such like. Another possible scenario is to isolate something nasty (eg. you may be testing security software and don't want any test specimens leaking). Host-Only Network Bridge (no routing)In this case devices on the bridge are able to communicate with the host, but no further. This can be useful where the host provides some services into the network (eg. DHCP) but you want to minimise the risk of anything leaking as in the Isolated Bridge. auto vhost_br0 A special scenario here would be if you wanted to expose a small number of services to this network from outside without allowing general forwarding. A simple way of doing this is with rinetd which acts as a port-forwarder. As an example, I want to provide access to my apt-cacher-ng server to isolated hosts and all I need to add to /etc/rinetd.conf is: # bindadress bindport connectaddress connectport Now the host is re-serving apt-cacher-ng on the Host-Only network. NAT Network Host (NAT via host)This is very similar to Host-Only but we do allow forwarding (routing) of traffic. auto vnat_br0 In order to provide NAT to addresses in this network you will need some iptables/ip6tables rules to translate addresses to that of the host. Fortunately in newer versions of Linux you can NAT IPv6 for those special conditions (don't do this normally!) where it is useful: iptables --append FORWARD --in-interface $srcbridge --out-interface $dstbridge --proto tcp --source $srcipv4 --jump ACCEPT Obviously you will also need to have suitable flushing of tables and all the normal iptables stuff before hand. I've also taken the precaution of REJECTing port 25 (SMTP) traffic to avoid tests involving mail getting out of the sandbox. Another option would be to forward port 25 traffic to a mailserver (VM?) configured to accept all mail to all domains and put it all in one mailbox which can be useful for verifying mail sending for applications. Bridge Filtering and sysctlOne thing to watch out for is that it is possible to run Netfilter on traffic going through bridges. This could result in some unexpected behaviour where some things work and others don't and waste a load of time trying to figure out what is happening when some traffic goes across the bridge and others don't. The key file is /proc/sys/net/bridge/bridge-nf-call-iptables which if set to "1" passes traffic through Netfilter. To avoid these problems I have a /etc/sysctl.d/99-local-libvirt.conf file containing: net.bridge.bridge-nf-call-iptables=0 Joining up Hyps with VLANsThis will require a network switch that will pass the VLAN tags through. Unmanaged switches can be a bit variable with this - I've heard of them stripping VLAN tags, dropping tagged frames, having MTU problems with (larger) tagged frames, and some just let the frames straight through which is the one scenario that would actually work for this. I don't take chances with this and since very cheap "Smart" (not very, but they understand 802.1Q) switches are now available for marginally more than unmanaged switches, that's the solution I use. To join bridges using VLANs you just need to add one extra line to the config, in this case the example is for the isolated network to eth1 VLAN ID 632: auto visol_br0 You only need to add the bridge_ports line for the IPv4 part of the config and it will pass both IPv4 and IPv6. Depending on your switch, you may also have to manually configure it to pass the VLAN between the relevant ports that your Hyps are connected to. Adding this config doesn't affect the behaviour of the network, but it does link it to a physical network where other Hyps can also link bridges to it. Breaking out into the real worldAnother possibility is that you may want to mix physical devices (eg. staging a server before putting it into Colo) into your environment, or stretch networks across multiple Hypervisors. This is something I've done previously simply by using a real "bridge_port" to a physical network device (maybe a VLAN) to connect in the other devices. More fun with Bridges & VLANsFollowing this I've done another post taking VLANs with Linux Bridges further with various combinations of Trunked Bridges, Single VLAN Bridges off a Trunk Bridge etc. |
|||
|
This is a bunch of random thoughts, ideas and other nonsense, and is not intended to be taken seriously. I'm experimenting and mostly have no idea what I am doing with most of this so it should be taken with cuation and at your own risk. Intrustive technologies are minimised where possible. For the purposes of reducing abuse and other risks hCaptcha is used and has it's own policies linked from the widget.
Copyright Glen Pitt-Pladdy 2008-2023
|
|||