I finally reached the end of my tether with the busybox syslogd and klogd. They keep truncating and mangling log lines and my mini_logcheck script keeps sending me mangled lines which don't match the exclusion patterns.

Enough! Switching to a syslog-ng3

There are two options on switching depending on if you are building a completely new image or want to switch over an existing image. The same instructions will likely be ok for syslog-ng also.

New build

Switchover is easy enough. If you are building your OpenWrt image from starters then in the menuconfig enable Administration => syslog-ng3 in the menuconfig and remove Base System => busybox => Configuration => System Logging Utilities => syslogd and klogd.

Build as usual and you should then have an image without the busybox loggers.

Existing build - switching live

OpenWrt has a packaging system with all of what you need already build, though do keep in mind the risks of installing binary packages.

# opkg update
# opkg install syslog-ng3

I also had to manually install libdbi:

# opkg install libdbi

Next, find the syslogd and klogd processes:

# ps |grep logd

And kill them:

# kill <syslogd pid> <klogd pid>

Then start syslog-ng3:

# /etc/init.d/syslog-ng start

Once you are happy with this then enable it to start at boot:

# /etc/init.d/syslog-ng enable

The only catch now is that the busybox syslogd and klogd are still installed and running and need to be disabled else they will start at boot. There are two approaches:

1. Delete /sbin/syslogd and /sbin/klogd - this simply removes the symlinks and you can add them in again later in you want to re-instate the busybox loggers
2. Edit /etc/init.d/boot and comment out the sections for syslogd and klogd. Again, easy to reinstate them later

What you do need to be aware of is that if you upgrade busybox at a later stage these changes may be overwritten.

Do not try and be smart and chmod -x the busybox loggers as they are symlinks to busybox and that will make busybox non-executable. All manner of pain will follow as without an executable busybox, most of the system (including chmod) will become unusable and the entire system will die. Yup - I found that one the hard way! :-)

At this point you should have a working logger. By default syslog-ng3 will do the job of both syslogd and klogd and I have not had any line mangling since switching to syslog-ng.

Log Rotation

One thing that syslog-ng3 doesn't do is log rotation - this needs to be done externally, but is easy enough to script up.

To keep compatibility with my mini_logcheck script which will try and read the final lines from previous log file when it's rotated, we will keep one generation uncompressed, and then however many compressed ones we specify (default 5).

Download my logspinner script and put it somewhere appropriate (/etc is probably best as it contains config). You will also need a working cron installation to run the logspinner. Your /etc/crontabs/root should have an entry something like this:

# m h dom mon dow command
37 6 * * * /etc/logspinner

Then restart cron:

# /etc/init.d/cron restart

That will run the script once a day, but if you need more regular rotations (eg. run small logfiles or lots of data hitting the logs) then you may want to run it more regularly.

You can edit the script and specify how large you want to allow the logs to get, and how many compressed generations you want to keep. By default log files are rotated when they get bigger than 64 blocks (typically 1024 bytes) and 5 compressed generations are kept. You can change that to suit your needs.