Glen Pitt-Pladdy :: Blog

Apache stats on Cacti (via SNMP)

Following on from the basics of SNMP I did previously, this article now adds my next set of SNMP extension scripts, config, and Cacti templates to monitor an Apache web server.

Apache Stats

Apache fortunately generates comprehensive stats internally and presents it under the url http://localhost/server-status?auto in a form that is easily parsable by scripts. All that is needed is to extract the stats.

Because a number of hits on the server will be needed for each statistic when snmpd is queried, it is a good idea to do some basic caching so that the stats retreval doesn't become the dominant load on an otherwise lightly loaded server.

The one thing that needs to be changed in Apache's config is to switch to extended stats. On Debian and Ubuntu systems edit the file /etc/apache2/mods-available/status.conf and add the directive "ExtendedStatus on" just before the Location section. Reload apache and it should be ready to go.

Getting Apache stats over SNMP

As the stats are available to the local machine (localhost or 127.0.0.1) over http, we once again avoid acrobatics discussed previously needed to get some data into snmpd. A single extension script is all that is needed.

To allow caching, I have created an additional directory: /var/local/snmp/cache

Set this to be owned by snmp so that it can write and modify files in here: chown snmp: /var/local/snmp/cache

Download: Apache statistics SNMP extension script

This script is in Python and you will also need the python-urlgrabber package (or at least that's what it's called in Debian / Ubuntu).

I place this script (make it executable first: chmod +x apache-stats) in /etc/snmp

In /etc/snmp/snmpd.conf add the following line (or others if you want to monitor them):

extend apache /etc/snmp/apache-stats

Once you have added all this in you can test apache-stats by running it from the command line with appropriate parameters, and via SNMP by appending the appropriate SNMP OID to the "snmpwalk" commands shown previously.

Cacti Templates

I have generated some basic Cacti Templates for these Apache stats.

Download: Cacti Templates for Apache statistics over SNMP

Simply import this template, and add the graphs you want to the appropriate device graphs in Cacti. It should just work if your SNMP is working correctly for that device (ensure other SNMP parameters are working for that device).

Graph Screen Shots

Apache Stats over SNMP on Cacti : Bytes per Request

Apache Stats over SNMP on Cacti : Bytes per Second

Apache Stats over SNMP on Cacti : CPU Load

Apache Stats over SNMP on Cacti : Scoreboard

Apache Stats over SNMP on Cacti : Workers

Update:

One person had pointed out that there has been some problems on some versions of Cacti with importing the templates. The template was generated with version 0.8.7b (from Debain Lenny).

The problem manifests it's self as "Cacti version does not exist" error, and appears to be cured by adding in this version, although in my version the file is actually global_arrays.php

The relevant array from my global_arrays.php / config_array.php:

$hash_version_codes = array(
    "0.8.4"  => "0000",
    "0.8.5"  => "0001",
    "0.8.5a" => "0002",
    "0.8.6"  => "0003",
    "0.8.6a" => "0004",
    "0.8.6b" => "0005",
    "0.8.6c" => "0006",
    "0.8.6d" => "0007",
    "0.8.6e" => "0008",
    "0.8.6f" => "0009",
    "0.8.6g" => "0010",
    "0.8.6h" => "0011",
    "0.8.6i" => "0012",
    "0.8.6j" => "0013",
    "0.8.7"  => "0014",
    "0.8.7a" => "0015",
    "0.8.7b" => "0016"
    );

Comments:

19/05/2010 05:25 :: Ajay Kajla

Hi,

The python script is giving following error. Can you please check.

./apache-stats

16

9216

Traceback (most recent call last):

File "./apache-stats.1", line 78, in ?

print params[param]

KeyError: 'CPULoad'

[root@ajay snmp]# ./apache-stats

16

9216

Traceback (most recent call last):

File "./apache-stats", line 78, in ?

print params[param]

KeyError: 'CPULoad'

19/05/2010 08:08 :: Glen Pitt-Pladdy

Thanks for pointing this out. Indeed, on some platforms (64bit?) Apache seems not to have a CPULoad parameter.

I have updated the script to handle missing parameters so hopefully this will solve the problem for you.

20/05/2010 18:50 :: Rob Moser

Hi Glen,

Doing something very similar today, and this was a great help; thanks.

I had two problems implementing this. 1) My installation of SNMP didn't have the MIB for NET-SNMP-EXTEND-MIB installed. Not your fault, obviously, but I thought I'd mention it here in case someone else following your advice has the same problem. If you test the SNMP extension with snmpwalk and get an error like "No Such Instance currently exists at this OID", have a look at the instructions for adding MIBs at www.net-snmp.org/docs/FAQ.html#How_do_I_add_a_MIB_

Problem 2 was that the graph and data templates for Bytes per Request didn't seem to import properly. It seemed to have something to do with the fact that it was using bytesPRErequest instead of bytesPERrequest (emphasis mine) in some places; I changed them all to PER, deleted all of the graphs and data sources and re-built them from the new templates, and it seems to be working fine.

20/05/2010 19:27 :: Glen Pitt-Pladdy

Hi Rob, Thanks for the feedback.

Interesting that your net-snmp didn't have that MIB. On my Debian installs they all did. The link you provided should hopefully do the trick for anyone else with this problem.

I think it's interesting that the pre/per thing caused problems. It was a typo that propagated from the Data template to the Graph template. I have to admit that I have had the occasional problem with templates with Cacti and generally clearing them out and re-importing seems to do the trick for me.

I have put up an updated version of the template with this corrected. Hopefully that will avoid any others with these problems.

Thanks!

20/05/2010 19:31 :: Rob Moser

I thought it was odd too; looked like it was done consistently throughout, so it shouldn't have bothered anyone but pedants like me. But of course I only really started looking at it after I had half fixed it, so I didn't have it in the original state to check. Its entirely possible that the typo had nothing to do with it, and re-loading things in Cacti is the only thing that I needed to fix my problem.

Thanks again for the handy walkthrough!

22/05/2010 00:03 :: Rob Moser

Turned out one of our machines didn't have python (and we don't want it to) so I ported this to perl. Saved a copy at docs.google.com/Doc?docid=0AT8aSke8tPgGZHZ2N3Bydl8xcHFtcHRzZGs&hl=en if its of any use to you...

23/05/2010 16:45 :: Glen Pitt-Pladdy

Nicely done!

I have to confess that doing stuff in Python is just an excuse for me to learn it :)

If anyone is unclear on how to get Rob's script working - you will need to add a #!/usr/bin/perl (or wherever your perl lives) at the top or modify your cron job to run this file with perl. Other than that, it looks the business to me and should be a drop-in replacement if you don't want to mess with Python.




Are you human? (reduces spam) Note: Identity details will be stored in a cookie. Posts may not appear immediately.