Glen Pitt-Pladdy :: BlogOpenWrt with native IPv6 on DG834 v2 (using AAISP) | |||
I used to run IPv6 here at home via a 6-to-4 tunnel to my ISP (AAISP), but after buying a new higher-end router, I found it blocked the 6-to-4 tunnel and my IPv6 was no more. Various other annoyances have come up with the router. While virtually all DSL routers appear to do ok on the basics (IPv4, NAT, DHCP etc.), the moment someone starts pushing their features it seems to move into the less tested (and more broken) parts of them. Annoyances I hit have included firewalling restrictions, IPv6 (as mentioned), lack of statistics, lack of good data via SNMP (or other ways of monitoring), lack of good logging and fault reporting, various user interface breakages, and much more. Admittedly this won't affect the average punter, but I'm not the average punter! Fed up with this, I decided I would look at the Open Source router firmware, and the one that has been making a lot of positive noise is OpenWrt. AAISP do native IPv6 on ADSL and this is my ideal - no more messing with tunnels. HardwareI wanted something sacrificial (cheap), but which would give good speed with an ADSL2+ modem. Wifi is not needed - my Wifi runs off a separate restricted branch of the internal network. There are few ADSL routers supported by OpenWrt, and many of the older, better supported routers seem to be stuck with ADSL1. One of the most popular routers around is the Netgear DG834 (or DG834G with wireless) features an ADSL2+ modem, and while not yet officially supported, on v1-v2 hardware things look promising provided you are prepared to build your own image. I picked up a DG834 v2 for £7.41 on eBay which for the money is hard to beat. A v3 would probably be better since according to Wikipedia it is faster on good lines (UPDATE: apparently due to the updated/modified bootloader, v3 has problems so stick with v2 for now unless you are brave - see my article on getting DG834v3 working with OpenWrt), but v4 and later changed the platform so I decided to avoid them. My v2 claims that it should be able to achieve 9984 kbps (ADSL1) on my line which is plenty of margin for the current 8 Mbps limit. From the charts that means that I should be looking at about 16 Mbps when we get ADSL2+ here in a few months. Useful LinksThere seems to be a lack of complete documentation on getting these up and running. This is understandable since they are not officially supported and only those brave enough to do their own build will get them running. These are places I found useful information:
BootloaderThe DG834 v1-v2 use the ADAM2 bootloader which needs to be modified to work. This is all very simple to achieve and afterwards the router still booted without problems like before. The instruction for modifying the bootloader and backing up the images are given in several of the links above including the OpenWrt DG834G Page. Make sure that you assign an address to ADAM2 as this will save lots of pain later when you try and FTP the images: router# echo "my_ipaddress 192.168.0.1" > /proc/sys/dev/adam2/environment BuildAs I was on a non-supported platform, I had to build my own image. I grabbed the latest 8.09 from Subversion on a Ubuntu Karmic virtual machine as the build platform. I started with Jonatahn McDowell's config and removed wireless support and added a few utilities that may be useful. Download: Build Config for OpenWrt on DG834 The menu configuration is build and run with: $ make menuconfig Adding packages in the main image is better than trying to add them later since there is very limited space for additional packages to be installed on a running system. Then it's just up to building the image which takes some time: $ make The images end up in the bin subdirectory. There is the option to dice up the image across 2 partitions in the Flash memory, or re-partition the Flash. Personally, I wanted to minimise changes made to the config so I diced up the image to work with the existing partitioning. How this works is that the ordering of the memory by ADAM to put the devices in the order of mtd2, mtd1, mtd0, mtd3, mtd4. The first 704KiB needs to go into mtd1, and the remainder into mtd0. OpenWrt will see the partitions in order so mtd1, mtd0 and mtd3 will be merged into it's partitions. To dice up the images for the partitions on the router as seen by ADAM2: $ dd if=openWrt-ar7-squashfs.bin of=ow-mtd1.bin count=720896 bs=1 Installing the imageThis was where I ran into the first problem, and it seems so have other people. The easy way of installing this is via the ADAM2 bootloader with ftp. The first trick is getting a connection to ADAM2 as there is a very small time window between boot and the OS loading where the ftp is available. It took a few tries (get ftp ready on the command line before you power on the router) to hit the window which seems to be a fraction of a second after the port light on the router comes on. This was where I ran into trouble. The connection stalled soon after trying to upload the image. I turned on hash printing and it made about 2 short bursts before it stopped. I tried some simple diagnostics with tshark, and it appears that there is probably some incompatibility between TCP stacks with Linux and ADAM2. I tried all the usual suspects for tweaks under /proc but nothing seemed to work. What does however work is FTPing the files under Windows, so I put them on a USB Flash drive and uploaded them under Windows. The instructions on Jonatahn McDowell's Blog are clear on how to FTP the images once you have a working machine to FTP them from. First bootAfter the first boot there is a wait while the system configures it's self for the first time, and then the Test light (Amber "tick" symbol) flasshes in "heartbeat". At this point it should be safe to telnet to the router and start configuration: $ telnet 192.168.0.1 The first thing is to set the password: router# passwd Then log out (Control-D is the quick way) and wait a few seconds while the ssh server starts. At that point you should be able to connect with ssh: $ ssh root@192.168.0.1 Now the real configuration can begin..... IPv4This is the LAN side of the router and defaults with my config to 192.168.0.1/24 in keeping with the defaults of the original firmware. AAISP will give you a routable block of IPs which allows me to run more than one machine on a real IP. Update the /etc/config/network to reflect your network settings (replace the ***'s): config 'interface' 'lan' IMPORTANT: As soon as these settings come into use, if they are wrong then you may loose the ability to connect to your router. Be very careful and make sure you get them right. To bring these settings into use, restart the networking (yes, you can do it other ways): router# /etc/init.d/network restart You will need to re-connect to your router if you changed anything that will affect your session. Likewise if you need to change the default DHCP configuration then you will find it in /etc/config/dhcp. IPv6I am using a completely static configuration for IPv6 so I didn't need to use radvd. I simply split my block of addresses (/64 - that's 18446744073709551616 individual addresses!) into a smaller one for this stage of routing. In reality it is completely feasible to run the full block from the router if you have a "flat" network (ie. everything comes directly off the router), but I have another stage of routing to the internal side of my network. The config now looks like this (replace the ***'s): config 'interface' 'lan' Again, you can restart the networking to bring the IPv6 address into operation. By default IPv6 forwarding doesn't seem to be enabled. Incomment the line in /etc/sysctl.conf: net.ipv6.conf.all.forwarding=1 Then start sysctl again to bring it into effect: # /etc/init.d/sysctl start ADSLThere seems to be limited documentation on the ADSL part which is understandable since few of the supported routers can do ADSL. First thing is to check if the hardware is around. You should get a /proc/avalanche/ directory where the status of the ADSL hardware is found. If you have connected the router to your line at this stage then all going well, it should come into sync and you should see some useful information. router# cat /proc/avalanche/avsar_modem_stats Next we need to configure the PPP interface (PPPoA for AAISP with a BT line). This is very simple and all that is needed is to put this in your /etc/config/network file: config 'interface' 'wan' You may need some different settings depending on how you connect. These are the standard for BT lines in the UK, but different last-mile providers have a different settings. To enable IPv6 on the PPP interface, you also need to add a line to /etc/ppp/options: +ipv6 At that point bring up the interface: router# ifup wan With a bit of luck you should be able to ping stuff in the outside world. If you have your IPv6 configured right then you should be able to ping stuff with IPv6 too (if you have been tunnelling your IPv6 up to now then remember to remove the tunnel endpoint in your AAISP control pages - click on the IPv6 address block to configure it). PPP boot fixOne thing that you will quickly discover is that out the box PPP will not come up on it's own. There is a patch to start PPP at boot - basically comment out one line, add another in /etc/init.d/network and it all starts working as expected. FirewallFirewalls have limited usefulness as Marcus Ranum points out - ultimately they have to let some traffic through which means there is always some exposure. The main benefit for me is to remove the background noise of random scans and other stuff that is going on all the time. OpenWrt provides it's own firewall configured in /etc/config/firewall and it may well be good for many people. The things I didn't like about is was that I could only find limited documentation, and I have a lot of rules which looked like they where going go be a pain to configure. The firewall does have a user script (/etc/firewall.user) so I commented out all the config except the lines to include the user script. Have a look at a cut down example of my rules (IPs removed to protect identities) if you want. IMPORTANT: Firewalling is one place that you can easily get yourself locked out of the router - I did and had to resort to a soldering iron to get the serial terminal going. I suggest you test your firewall by running something like: router# /etc/init.d/firewall restart; sleep 60; /etc/init.d/firewall stop This means that if you do lock yourself out then after 60 seconds the firewall will be stopped again. The way I locked myself out is having the rules to block private address ranges while running on a private address meant that the router started blocking it's own subnet. If you have moved onto real IPs then that will be avoided, but always other ways to get locked out. QoSThis is quite useful, but the default settings are likely to restrict the bandwidth badly on all but the slowest connections. Edit /etc/config/qos and adjust the "option upload" and "option download" settings to match your line, then restart qos: router# /etc/init.d/qos restart NTPFor this you need to install ntpclient: router# opkg update By default it will use ntp.org servers, but to use your ISP's (AAISP has a list of their servers) to configure it edit /etc/config/ntpclient adding your server(s) and disabling the existing ones. This only starts when PPP comes up so you may need to manually restart PPP: router# ifdown wan ADSL/PPP LEDAnnoyingly this doesn't work out the box. There do appear to be patches for it to work via hotplug (event driven which sounds good to me), but these don't seem to have maded it into the trunk. In the interests of minimal hassle, I cooked up a quick poller script which would take care of things. All this does is to poll for ADSL being up and ppp0 running. If ppp0 is running it turns off the ADSL LED since it is amber and much brighter than the PPP LED, other wise it sets the ADSL LED to match the line status. It also sets the PPP LED to run off the ppp0 network device so when PPP is up this green LED should be on (flickering with traffic) , if PPP is down but ADSL up then the amber ADSL LED should be on, and if nothing is up then the LEDs should be off. It's not perfect and only polls each second which means that there may be overlaps between LEDs or delays in the ADSL LED, but it is an easy workaround for the LEDs not working natively. Download: OpenWrt ADSL/PPP LED script You can either scp it to the router, or wget it directly to the router. Put this in /etc/init.d, make it executable, and then to enable it at boot run: router# /etc/init.d/local-adsl-led enable Then to start it either reboot the router or run: router# /etc/init.d/local-adsl-led start Backups with dirvishThis uses rsync over ssh to backup machines. Essentially all you need to do on the router is install rsync: router# opkg update And then if you use key based authentication for the dirvish backups, put the authorised key in /etc/dropbear/authorized_keys Download: OpenWrt dirvish backup config Monitoring with CactiNormally I use SNMP for monitoring (that's what it's meant for), but it's rather a large package for such a small device. Since we already have a webserver on board, I decided to use that instead, and just deliver raw data from various commands. This means that custom data input scripts are needed on Cacti to interpret this data, and this minimises the work the router has to do. Create a /www/cgi-bin/monitor directory on the router and drop the CGI Scripts in there (making them executable of course). On the Cacti host, put the Cacti Input Scripts in the Cacti scripts directory (these templates are designed for /usr/local/share/cacti/scripts), and import the Cacti template bundle. You will also need to create a cache directory on the Cacti host writable by the user Cacti runs under: cactihost# mkdir /var/local/cacti ADSL:PPP Connectivity:Network Stats:CPU Stats:Memory Stats:Load Average:Derived from standard Cacti Load Average templates Connection Tracking Stats:Cacti Templates:Also, to catch brief drops in ADSL and PPP between polling the router, you can add a monitoring script to /etc/init.d, and remember to enable it: And to flag PPP drops and ip-up script to go in /etc/ppp/ip-up.d: What the graphs look like:
Update 27/1/2010: AnnoyancesEverything has mostly been good, but a few small things that have come up: CPU Limited bandwidthAfter having thrashed it a bit, it seems that there is very high CPU usage under even moderate network traffic. See the CPU graph above - the Soft IRQ usage climbs to about 20% when it's only shifting about 1Mbit/s. I will see if I can investigate this further when I get some time, but it may just be that this is the way it is on this platform. Network falls back to 10Mb/s half duplexI discovered that the ethernet had dropped down in speed. With ethtool it looks like this: root@OpenWrt:~# ethtool eth0 Attempting to make it go at full speed, full duplex results in nothing happening, unless I disable autonegotiation at which point there is "bad stuff" in the logs: root@OpenWrt:~# ethtool -s eth0 speed 100 duplex full autoneg off And in the logs: Jan 27 19:39:55 OpenWrt user.info kernel: PHY: 0:10 - Link is Down I have to turn autonegotiation on again to stop it grumbling about "Trying 10/HALF". I suspect the ethernet driver here, but need to do more investigation. Even though it thinks the link is down, it remains pingable throughout which suggests a bug. Update 8/5/2010I have put up improved Cacti scripts and templates which take care of a load of little things that I have noticed running the previous set. They can now flag up brief drops in PPP and ADSL between samples, better representation of sync rates and other refinements. |
|||
|
This is a bunch of random thoughts, ideas and other nonsense, and is not intended to be taken seriously. I'm experimenting and mostly have no idea what I am doing with most of this so it should be taken with cuation and at your own risk. Intrustive technologies are minimised where possible. For the purposes of reducing abuse and other risks hCaptcha is used and has it's own policies linked from the widget.
Copyright Glen Pitt-Pladdy 2008-2023
|
|||
