Glen Pitt-Pladdy :: Blog

OpenWrt with native IPv6 on DG834 v2 (using AAISP)

I used to run IPv6 here at home via a 6-to-4 tunnel to my ISP (AAISP), but after buying a new higher-end router, I found it blocked the 6-to-4 tunnel and my IPv6 was no more.

Various other annoyances have come up with the router. While virtually all DSL routers appear to do ok on the basics (IPv4, NAT, DHCP etc.), the moment someone starts pushing their features it seems to move into the less tested (and more broken) parts of them. Annoyances I hit have included firewalling restrictions, IPv6 (as mentioned), lack of statistics, lack of good data via SNMP (or other ways of monitoring), lack of good logging and fault reporting, various user interface breakages, and much more. Admittedly this won't affect the average punter, but I'm not the average punter!

Fed up with this, I decided I would look at the Open Source router firmware, and the one that has been making a lot of positive noise is OpenWrt. AAISP do native IPv6 on ADSL and this is my ideal - no more messing with tunnels.

Hardware

I wanted something sacrificial (cheap), but which would give good speed with an ADSL2+ modem. Wifi is not needed - my Wifi runs off a separate restricted branch of the internal network. There are few ADSL routers supported by OpenWrt, and many of the older, better supported routers seem to be stuck with ADSL1.

One of the most popular routers around is the Netgear DG834 (or DG834G with wireless) features an ADSL2+ modem, and while not yet officially supported, on v1-v2 hardware things look promising provided you are prepared to build your own image.

I picked up a DG834 v2 for £7.41 on eBay which for the money is hard to beat. A v3 would probably be better since according to Wikipedia it is faster on good lines (UPDATE: apparently due to the updated/modified bootloader, v3 has problems so stick with v2 for now unless you are brave - see my article on getting DG834v3 working with OpenWrt), but v4 and later changed the platform so I decided to avoid them. My v2 claims that it should be able to achieve 9984 kbps (ADSL1) on my line which is plenty of margin for the current 8 Mbps limit. From the charts that means that I should be looking at about 16 Mbps when we get ADSL2+ here in a few months.

Useful Links

There seems to be a lack of complete documentation on getting these up and running. This is understandable since they are not officially supported and only those brave enough to do their own build will get them running.

These are places I found useful information:

Bootloader

The DG834 v1-v2 use the ADAM2 bootloader which needs to be modified to work. This is all very simple to achieve and afterwards the router still booted without problems like before.

The instruction for modifying the bootloader and backing up the images are given in several of the links above including the OpenWrt DG834G Page.

Make sure that you assign an address to ADAM2 as this will save lots of pain later when you try and FTP the images:

router# echo "my_ipaddress 192.168.0.1" > /proc/sys/dev/adam2/environment

Build

As I was on a non-supported platform, I had to build my own image.  I grabbed the latest 8.09 from Subversion on a Ubuntu Karmic virtual machine as the build platform. I started with Jonatahn McDowell's config and removed wireless support and added a few utilities that may be useful.

Download: Build Config for OpenWrt on DG834

The menu configuration is build and run with:

$ make menuconfig

Adding packages in the main image is better than trying to add them later since there is very limited space for additional packages to be installed on a running system.

Then it's just up to building the image which takes some time:

$ make

The images end up in the bin subdirectory. There is the option to dice up the image across 2 partitions in the Flash memory, or re-partition the Flash. Personally, I wanted to minimise changes made to the config so I diced up the image to work with the existing partitioning.

How this works is that the ordering of the memory by ADAM to put the devices in the order of mtd2, mtd1, mtd0, mtd3, mtd4. The first 704KiB needs to go into mtd1, and the remainder into mtd0. OpenWrt will see the partitions in order so mtd1, mtd0 and mtd3 will be merged into it's partitions.

To dice up the images for the partitions on the router as seen by ADAM2:

$ dd if=openWrt-ar7-squashfs.bin of=ow-mtd1.bin count=720896 bs=1
$ dd if=openWrt-ar7-squashfs.bin of=ow-mtd0.bin skip=720896 bs=1

Installing the image

This was where I ran into the first problem, and it seems so have other people. The easy way of installing this is via the ADAM2 bootloader with ftp. The first trick is getting a connection to ADAM2 as there is a very small time window between boot and the OS loading where the ftp is available.

It took a few tries (get ftp ready on the command line before you power on the router) to hit the window which seems to be a fraction of a second after the port light on the router comes on.

This was where I ran into trouble. The connection stalled soon after trying to upload the image. I turned on hash printing and it made about 2 short bursts before it stopped. I tried some simple diagnostics with tshark, and it appears that there is probably some incompatibility between TCP stacks with Linux and ADAM2. I tried all the usual suspects for tweaks under /proc but nothing seemed to work.

What does however work is FTPing the files under Windows, so I put them on a USB Flash drive and uploaded them under Windows. The instructions on Jonatahn McDowell's Blog are clear on how to FTP the images once you have a working machine to FTP them from.

First boot

After the first boot there is a wait while the system configures it's self for the first time, and then the Test light (Amber "tick" symbol) flasshes in "heartbeat".

At this point it should be safe to telnet to the router and start configuration:

$ telnet 192.168.0.1

The first thing is to set the password:

router# passwd

Then log out (Control-D is the quick way) and wait a few seconds while the ssh server starts. At that point you should be able to connect with ssh:

$ ssh root@192.168.0.1

Now the real configuration can begin.....

IPv4

This is the LAN side of the router and defaults with my config to 192.168.0.1/24 in keeping with the defaults of the original firmware.

AAISP will give you a routable block of IPs which allows me to run more than one machine on a real IP. Update the /etc/config/network to reflect your network settings (replace the ***'s):

config 'interface' 'lan'
        option 'ifname' 'eth0'
        option 'proto' 'static'
        option 'netmask' '***.***.***.***'
        option 'dns' ''
        option 'gateway' ''
        option 'ipaddr' '***.***.***.***'

IMPORTANT: As soon as these settings come into use, if they are wrong then you may loose the ability to connect to your router. Be very careful and make sure you get them right.

To bring these settings into use, restart the networking (yes, you can do it other ways):

router# /etc/init.d/network restart

You will need to re-connect to your router if you changed anything that will affect your session.

Likewise if you need to change the default DHCP configuration then you will find it in /etc/config/dhcp.

IPv6

I am using a completely static configuration for IPv6 so I didn't need to use radvd. I simply split my block of addresses (/64 - that's 18446744073709551616 individual addresses!) into a smaller one for this stage of routing. In reality it is completely feasible to run the full block from the router if you have a "flat" network (ie. everything comes directly off the router), but I have another stage of routing to the internal side of my network. The config now looks like this (replace the ***'s):

config 'interface' 'lan'
        option 'ifname' 'eth0'
        option 'proto' 'static'
        option 'netmask' '***.***.***.***'
        option 'dns' ''
        option 'gateway' ''
        option 'ipaddr' '***.***.***.***'
        option ip6addr ****:****:****:****:****::****/**

Again, you can restart the networking to bring the IPv6 address into operation.

By default IPv6 forwarding doesn't seem to be enabled. Incomment the line in /etc/sysctl.conf:

net.ipv6.conf.all.forwarding=1

Then start sysctl again to bring it into effect:

# /etc/init.d/sysctl start

ADSL

There seems to be limited documentation on the ADSL part which is understandable since few of the supported routers can do ADSL.

First thing is to check if the hardware is around. You should get a /proc/avalanche/ directory where the status of the ADSL hardware is found. If you have connected the router to your line at this stage then all going well, it should come into sync and you should see some useful information.

router# cat /proc/avalanche/avsar_modem_stats

Next we need to configure the PPP interface (PPPoA for AAISP with a BT line). This is very simple and all that is needed is to put this in your /etc/config/network file:

config 'interface' 'wan'
        option ifname ppp0
        option unit 0
        option proto pppoa
        option encaps vc
        option vpi 0
        option vci 38
        option username ADSLUSERNAME
        option password ADSLPASSWORD
        option keepalive 5,5

You may need some different settings depending on how you connect. These are the standard for BT lines in the UK, but different last-mile providers have a different settings.

To enable IPv6 on the PPP interface, you also need to add a line to /etc/ppp/options:

+ipv6

At that point bring up the interface:

router# ifup wan

With a bit of luck you should be able to ping stuff in the outside world. If you have your IPv6 configured right then you should be able to ping stuff with IPv6 too (if you have been tunnelling your IPv6 up to now then remember to remove the tunnel endpoint in your AAISP control  pages - click on the IPv6 address block to configure it).

PPP boot fix

One thing that you will quickly discover is that out the box PPP will not come up on it's own.  There is a patch to start PPP at boot - basically comment out one line, add another in /etc/init.d/network and it all starts working as expected.

Firewall

Firewalls have limited usefulness as Marcus Ranum points out - ultimately they have to let some traffic through which means there is always some exposure. The main benefit for me is to remove the background noise of random scans and other stuff that is going on all the time.

OpenWrt provides it's own firewall configured in /etc/config/firewall and it may well be good for many people. The things I didn't like about is was that I could only find limited documentation, and I have a lot of rules which looked like they where going go be a pain to configure.

The firewall does have a user script (/etc/firewall.user) so I commented out all the config except the lines to include the user script.

Have a look at a cut down example of my rules (IPs removed to protect identities) if you want.

IMPORTANT: Firewalling is one place that you can easily get yourself locked out of the router - I did and had to resort to a soldering iron to get the serial terminal going. I suggest you test your firewall by running something like:

router# /etc/init.d/firewall restart; sleep 60; /etc/init.d/firewall stop

This means that if you do lock yourself out then after 60 seconds the firewall will be stopped again.

The way I locked myself out is having the rules to block private address ranges while running on a private address meant that the router started blocking it's own subnet. If you have moved onto real IPs then that will be avoided, but always other ways to get locked out.

QoS

This is quite useful, but the default settings are likely to restrict the bandwidth badly on all but the slowest connections. Edit /etc/config/qos and adjust the "option upload" and "option download" settings to match your line, then restart qos:

router# /etc/init.d/qos restart

NTP

For this you need to install ntpclient:

router# opkg update
router# opkg install ntpclient

By default it will use ntp.org servers, but to use your ISP's (AAISP has a list of their servers) to configure it edit /etc/config/ntpclient adding your server(s) and disabling the existing ones.

This only starts when PPP comes up so you may need to manually restart PPP:

router# ifdown wan
router# ifup wan

ADSL/PPP LED

Annoyingly this doesn't work out the box. There do appear to be patches for it to work via hotplug (event driven which sounds good to me), but these don't seem to have maded it into the trunk.

In the interests of minimal hassle, I cooked up a quick poller script which would take care of things. All this does is to poll for ADSL being up and ppp0 running. If ppp0 is running it turns off the ADSL LED since it is amber and much brighter than the PPP LED, other wise it sets the ADSL LED to match the line status. It also sets the PPP LED to run off the ppp0 network device so when PPP is up this green LED should be on (flickering with traffic) , if PPP is down but ADSL up then the amber ADSL LED should be on, and if nothing is up then the LEDs should be off.

It's not perfect and only polls each second which means that there may be overlaps between LEDs or delays in the ADSL LED, but it is an easy workaround for the LEDs not working natively.

Download: OpenWrt ADSL/PPP LED script

You can either scp it to the router, or wget it directly to the router. Put this in /etc/init.d, make it executable, and then to enable it at boot run:

router# /etc/init.d/local-adsl-led enable

Then to start it either reboot the router or run:

router# /etc/init.d/local-adsl-led start

Backups with dirvish

This uses rsync over ssh to backup machines. Essentially all you need to do on the router is install rsync:

router# opkg update
router# opkg install rsync

And then if you use key based authentication for the dirvish backups, put the authorised key in /etc/dropbear/authorized_keys

Download: OpenWrt dirvish backup config

Monitoring with Cacti

Normally I use SNMP for monitoring (that's what it's meant for), but it's rather a large package for such a small device. Since we already have a webserver on board, I decided to use that instead, and just deliver raw data from various commands. This means that custom data input scripts are needed on Cacti to interpret this data, and this minimises the work the router has to do.

Create a /www/cgi-bin/monitor directory on the router and drop the CGI Scripts in there (making them executable of course).

On the Cacti host, put the Cacti Input Scripts in the Cacti scripts directory (these templates are designed for /usr/local/share/cacti/scripts), and import the Cacti template bundle.

You will also need to create a cache directory on the Cacti host writable by the user Cacti runs under:

cactihost# mkdir /var/local/cacti
cactihost# chgrp www-data /var/local/cacti
cactihost # chmod g+w /var/local/cacti

ADSL:
PPP Connectivity:
Network Stats:
CPU Stats:
Memory Stats:
Load Average:

Derived from standard Cacti Load Average templates

Connection Tracking Stats:
Cacti Templates:

Also, to catch brief drops in ADSL and PPP between polling the router, you can add a monitoring script to /etc/init.d, and remember to enable it:

And to flag PPP drops and ip-up script to go in /etc/ppp/ip-up.d:

What the graphs look like:

OpenWRT ADSL and PPP Status Cacti Graph

OpenWRT ADSL CRC Errors Cacti Graph

OpenWRT ADSL ADSL Errors Cacti Graph

OpenWRT ADSL Signal Quality Cacti Graph

OpenWRT ADSL Speeds Cacti Graph

OpenWRT CPU Usage Cacti Graph

OpenWRT Ethernet Bytes Cacti Graph

OpenWRT Ethernet Packets Cacti Graph

OpenWRT IP Conntrack Cacti Graph

OpenWRT Load Average Cacti Graph

OpenWRT Memory Usage Cacti Graph

OpenWRT PPP Bytes Cacti Graph

OpenWRT PPP Packets Cacti Graph

OpenWRT BusyBox via SSH

Update 27/1/2010: Annoyances

Everything has mostly been good, but a few small things that have come up:

CPU Limited bandwidth

After having thrashed it a bit, it seems that there is very high CPU usage under even moderate network traffic. See the CPU graph above - the Soft IRQ usage climbs to about 20% when it's only shifting about 1Mbit/s.

I will see if I can investigate this further when I get some time, but it may just be that this is the way it is on this platform.

Network falls back to 10Mb/s half duplex

I discovered that the ethernet had dropped down in speed. With ethtool it looks like this:

root@OpenWrt:~# ethtool eth0
Settings for eth0:
        Supported ports: [ TP AUI BNC MII FIBRE ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
        Supports auto-negotiation: Yes
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
        Advertised auto-negotiation: Yes
        Speed: 10Mb/s
        Duplex: Half
        Port: MII
        PHYAD: 16
        Transceiver: external
        Auto-negotiation: on
        Link detected: yes

Attempting to make it go at full speed, full duplex results in  nothing happening, unless I disable autonegotiation at which point there is "bad stuff" in the logs:

root@OpenWrt:~# ethtool -s eth0 speed 100 duplex full autoneg off

And in the logs:

Jan 27 19:39:55 OpenWrt user.info kernel: PHY: 0:10 - Link is Down
Jan 27 19:40:05 OpenWrt user.info kernel: Trying 100/HALF
Jan 27 19:40:05 OpenWrt user.info kernel: PHY: 0:10 - Link is Down
Jan 27 19:40:16 OpenWrt user.info kernel: Trying 10/FULL
Jan 27 19:40:16 OpenWrt user.info kernel: PHY: 0:10 - Link is Down
Jan 27 19:40:27 OpenWrt user.info kernel: Trying 10/HALF
Jan 27 19:40:27 OpenWrt user.info kernel: PHY: 0:10 - Link is Down
Jan 27 19:40:38 OpenWrt user.info kernel: Trying 10/HALF
...... repeats OpenWrt user.info kernel: Trying 10/HALF

I have to turn autonegotiation on again to stop it grumbling about "Trying 10/HALF". I suspect the ethernet driver here, but need to do more investigation.

Even though it thinks the link is down, it remains pingable throughout which suggests a bug.

Update 8/5/2010

I have put up improved Cacti scripts and templates which take care of a load of little things that I have noticed running the previous set. They can now flag up brief drops in PPP and ADSL between samples, better representation of sync rates and other refinements.

Comments:

Cecil Ward Image  27/06/2010 23:53 :: Cecil Ward

Very useful articles on the DG834v3 and IPv6-ness in general. Much appreciated.

Cecil Ward Image  27/06/2010 23:53 :: Cecil Ward

Very useful articles on the DG834v3 and IPv6-ness in general. Much appreciated.

Simon Image  23/12/2010 19:18 :: Simon

Thanks for the instructions, just followed them and now have native IPv6 up and running on my AAISP connection.

Simon Image  23/12/2010 19:18 :: Simon

Thanks for the instructions, just followed them and now have native IPv6 up and running on my AAISP connection.

Glen Pitt-Pladdy Image  23/12/2010 19:29 :: Glen Pitt-Pladdy

Well done! It's always good to hear success stories.

Glen Pitt-Pladdy Image  23/12/2010 19:29 :: Glen Pitt-Pladdy

Well done! It's always good to hear success stories.

Thomas Jepp Image  12/04/2011 15:19 :: Thomas Jepp

I cannibalised this guide and the official openwrt one and got this working on a wrt160nl and a zyxel in bridged mode, also on AAISP.

Thomas Jepp Image  12/04/2011 15:19 :: Thomas Jepp

I cannibalised this guide and the official openwrt one and got this working on a wrt160nl and a zyxel in bridged mode, also on AAISP.

Glen Pitt-Pladdy Image  12/04/2011 19:16 :: Glen Pitt-Pladdy

That's excellent! Good to see others having success.

Glen Pitt-Pladdy Image  12/04/2011 19:16 :: Glen Pitt-Pladdy

That's excellent! Good to see others having success.

MikeH Image  20/05/2011 17:47 :: MikeH

Seriously, did you really need to redact the OpenWRT and Busybox versions? How paranoid ARE you?

MikeH Image  20/05/2011 17:47 :: MikeH

Seriously, did you really need to redact the OpenWRT and Busybox versions? How paranoid ARE you?

Glen Pitt-Pladdy Image  20/05/2011 19:00 :: Glen Pitt-Pladdy

It takes less time to do than to think about... or comment about! So, yes, I'm paranoid, but not as paranoid as I am lazy :-)

Glen Pitt-Pladdy Image  20/05/2011 19:00 :: Glen Pitt-Pladdy

It takes less time to do than to think about... or comment about! So, yes, I'm paranoid, but not as paranoid as I am lazy :-)

Lee Shakespeare Image  17/09/2011 17:11 :: Lee Shakespeare

Just spent the last 24 hours sorting this out with a DG834v1, very useful, many thanks.

Lee Shakespeare Image  17/09/2011 17:11 :: Lee Shakespeare

Just spent the last 24 hours sorting this out with a DG834v1, very useful, many thanks.

Glen Pitt-Pladdy Image  17/09/2011 17:16 :: Glen Pitt-Pladdy

I'm glad to hear people are having success

Glen Pitt-Pladdy Image  17/09/2011 17:16 :: Glen Pitt-Pladdy

I'm glad to hear people are having success

tempest69 Image  03/01/2012 13:16 :: tempest69

Hi!

I'm a DG834Gv3 (pspboot+routertech firmware inside - full fonctioning) and a DG834v3 (no wifi - original firmware) owner.

With your method, i 've been able to try openwrt on the DG834Gv3 but : no wifi (that's not a secret) and the worse ethernet switch was not ok to!

I would like to put last openwrt version on the DG834v3, but only if switch and port based vlan is ok with it.

Any information about it?

Regards

Glen Pitt-Pladdy Image  03/01/2012 13:28 :: Glen Pitt-Pladdy

I never used VLANs on my DG834 v3 though I do remember reading about problems at the time I did this. Given it's been some time I suspect things may have moved on.

I'm no longer using DG834's as I've moved to FTTC so I don't think I can help you with this.




Are you human? (reduces spam)
Note: Identity details will be stored in a cookie. Posts may not appear immediately