Glen Pitt-Pladdy :: BlogDovecot stats on Cacti (via SNMP) | |||
As promised in the comments about the Postfix stats article, this posting kicks off support for Dovecot stats with Cacti, extending the original Postfix only stats. This uses my Universal Log Analyser providing a plugin for Dovecot. You will need to get this installed and working before you can use the plugin here, and have it running from a cron job every 5 minutes (or whatever your sample times is for Cacti) so that it can parse the logs. At this stage I am only producing stats for IMAP, but with the nice consistent log format it should be easy to add POP3 and SIEVE later. Dovecot PluginThrow dovecot.pm in your plugin directory for the Universal Log Analyser, and add "dovecot" as a module to the command line so that this module gets loaded. Download: Universal Log Analyser plugin and Cacti template for Dovecot are on GitHub Not much more to it than that. The remainder of the article assumes your stats file is /var/local/snmp/mail so if it isn't you will need to tweak things to match your install. It's worth checking the stats file to verify that dovecot stats are in fact being picked up by the plugin. SNMP ScriptsFirst, ensure that your SNMP is configured and working as described in my SNMP basics article. These provide the link for snmpd to pick up the stats and assuming they are in /etc/snmp the config in /etc/snmp/snmpd.conf is: extend dovecotauth /etc/snmp/dovecot-stats-auth Extension scripts for snmpd are named dovecot-stats-* - put them in a suitable place.... like /etc/snmp You should be able to run these scripts manually and they should spit back the current info from the stats file. Remember to restart snmpd so that the new config is picked up and we should be ready to go. Cacti TemplateImport this into your Cacti and add graphs as usual. After that, assuming everything is working then after a couple data samples content should start to appear on the graphs. If not then check the data at each step: the stats file, SNMP scripts, snmpwalk from the Cacti server, check Cacti Poller log for errors, and try Cacti in debug mode for graphs and data sources to see if that shows anything. I will post example graphs once my ones are mature enough to have some useful data on them. The Graphs
|
|||
|
Disclaimer: This is a load of random thoughts, ideas and other nonsense and is not intended to be taken seriously. I have no idea what I am doing with most of this so if you are stupid and naive enough to believe any of it, it is your own fault and you can live with the consequences. More importantly this blog may contain substances such as humor which have not yet been approved for human (or machine) consumption and could seriously damage your health if taken seriously. If you still feel the need to litigate (or whatever other legal nonsense people have dreamed up now), then please address all complaints and other stupidity to yourself as you clearly "don't get it".
Copyright Glen Pitt-Pladdy 2008-2022
|
|||
Comments:
See some comments on the SNMP basics post regarding tweaking this plugin for different format (dedicated) logs on different systems.
hi,
i have this errors:
unknown dovecot: Feb 5 04:03:00 mail dovecot: POP3(xxx@domain.it): Disconnected: Logged out top=0/0, retr=0/0, del=0/0, size=0
unknown dovecot: Feb 8 15:22:58 mail dovecot: pop3-login: Aborted login (auth failed, 1 attempts): user=<postmaster@domain.it>, method=PLAIN, rip=XX.XXX.XXX.X, lip=XXX.XXX.X.XX
the version is:
20110626 (the last i guess).
thanks,
Daniele
Thanks for the report - I had not tested pop3 on a live setup as my servers only run imap so it is really useful to get these reports. I have released a new version tonight which should understand these lines and adds an additional "login Aborted" graph.
Let me know if you discover any further lines it does not understand.
Hi again,
seems only that two errors for dovecot (i will look later at postfix, but anyway strings does not go to stderr).
Instead, i've found a bug in the template, it look for "dovecotpop3isconnect" instead of "dovecotpop3disconnect".
I have already patched the file, but how can i send it to you?
Thanks,
Daniele
Good to hear that the scripts are working better.
Cacti can be have problems re-importing templates so it's best I make the changes in cacti and re-export the templates. I have found the typos you spotted (well done!) and have released a new template. The new template should have a MD5 checksum of 6e8f655721539da3d17c2a7f22e0b23d - if you keep getting the old one then you may need to force reload without cache (shift-F5 in many browsers).
Jul 25 18:22:03 XXXXX dovecot: auth-worker(default): mysql: Connected to localhost (XXXXX)
I don't speak enough PERL for adding the needed code. :(
Can you build an update, please?
Mny Tnx
Alex
Thanks for submitting the unhandled line. I've just released version 20120726 which should handle those lines.
Hi, I have one more unhandled line:
Jun 3 20:38:14 chaos dovecot: pop3-login: Login: user=<tomek>, method=PLAIN, rip=192.168.0.135, lip=xx.xx.xx.xx, mpid=24423, TLS, session=<ur1xRETeEADAqACH>
Please help :)
I think these relate to newer versions of dovecot (eg. in Debian wheezy) and with starting to upgrade my servers I'm starting to see similar. I will release an update shortly to handle the new formats/info in these lines and others. Busy testing it right now....
Hi, I have another unhandled line:
/etc/snmp/uloganalyser-plugin/dovecot.pm 20120501:192 /var/log/mail.log:7842 unknown dovecot: Jun 24 08:45:03 XXXXXXX dovecot: deliver(XX@XXX.XX): sieve: msgid=<XX@XXX.XX>: forwarded to <xx@xxx.xx>
Many thanks for your work :)
First of all, thanks for your great work! Below I have a few more examples of errors:
dovecot.pm 20121115:157 /var/log/mail_test.log:489 unknown dovecot: Sep 15 06:28:45 mc1 dovecot: imap-login: Login: user=<xxxxxx@xxxxxx>, method=PLAIN, rip=1.1.1.1, lip=1.1.1.1, mpid=27826, secured, session=<ROj3hWTmSQCsFtyb>
dovecot.pm 20121115:157 /var/log/mail_test.log:471 unknown dovecot: Sep 15 06:28:45 mc1 dovecot: pop3-login: Login: user=<xxxxx@xxxxxx>, method=PLAIN, rip=1.1.1.1, lip=1.1.1.1, mpid=27833, secured, session=<J0n3hWTmMACsFtyb>
dovecot.pm 20121115:157 /var/log/mail_test.log:389 unknown dovecot: Sep 15 06:28:01 mc1 dovecot: managesieve-login: Login: user=<xxxxx@xxxxxx>, method=PLAIN, rip=1.1.1.1, lip=1.1.1.1, mpid=27835, secured, session=<UUFQg2Tm8ACsFtya>
Actually this is all covered in my development version (been updating it for Wheezy) so I've done some extra testing on that and released that today. It should cover a whole load of things in newer versions of Dovecot. The main changes are dovecot.pm, dovecot-stats-deliver and of course the Cacti template which may be messy to update.
Thanks for the update! Most of the things work fine, besides the cacti template. Can I use the old cacti template with your new scripts?
Hello!
It seems my post here two days ago is not show so I'll try it again. I am running dovecot 2.1.17. The only graph that is drawn with the Dovecot stats on Cacti tarball 20130920 is the session-graph. Where could it start to check if all the needed stuff is found and paths are right?
Andreas
Glen,
I don't see a link to the cacti template in the article. Am I missing something?
Travis
Andreas - I am running 2.1.7 so the version is good. The data goes into the stats file specified (if you are using the example in the uloganalyser post then /var/local/snmp/mail). What is unique about the sessions is that they are picked up from doveadm, whereas everything else is from parsing log files which should be a big clue to what part is not working and where to investigate.
Travis - the template is in the tarball along with all the plugin (you will still need uloganalyser from the other article), scripts and other bits needed to set this up: cacti_host_template_dovecot.xml
Thanks Glen! I am not sure how I missed that before.
Thanks for all your work on this!
I'm trying to import into the tarball 20131027 into Cacti 0.8.7i and I'm getting "Error: XML: Hash version does not exist". All the data, graph and host templates import but there is no name on any of them.
Am I missing something obvious?
I have the same problem with the Cacti file for postfix also.
This is very likely because the version these were done on is 0.8.8a and your version doesn't know about that. It's possible the ugly stuff in my "Cacti hack for forward compatibility" article will help (or really mess things up), otherwise I expect you will have to upgrade.
Hi Glen,
I have found an issue with my dovecot log format (2.2.12):
dovecot.pm 20131027:419 /root/mail.log.0:191 unknown dovecot: Apr 29 23:59:59 pop01 dovecot: imap(pippo@pippo.com), session=<HPjxVzX4XgBdLtmW>: Disconnected: Logged out in=564 out=11430
dovecot.pm 20131027:419 /root/mail.log.0:194 unknown dovecot: Apr 29 23:59:59 pop01 dovecot: pop3(pippo@pippo.com), session=<98OajTX4eAC8Dqbj>: Disconnected: Logged out top=0/0, retr=0/0, del=0/0, size=0, bytes=12/43
the difference from my log and your "regex" for log out is that my log starts with session=
I understand that this problem can be fixed by editing this line in dovecot.pm:
elsif ( $line =~ s/^(Disconnected)[:\s]*// or $line =~ s/^(proxy)\([^\)]+\): disconnecting [\da-f\.:]+ // )
but I'm not able, can you help me?
Thanks
Hi Glen,
I solved the previous problem moving "session" at the end of the line but now your plugin is not any more able to count the in/out bytes.
Here a sample of my "Disconnected" log:
Apr 30 23:01:02 pop01eeh dovecot: imap(pippo@pippo.com): Disconnected: Logged out in=640 out=3883 session=<ACSb2Ej4iABtqHHz>
Apr 30 23:01:02 pop01eeh dovecot: pop3(pippo@pippo.com): Disconnected: Logged out top=0/0, retr=0/0, del=0/0, size=0, bytes=12/43, session=<jAaa2Ej4cAAuLOvd>
Can you help me?
Thanks
What version of Dovecot and distro are you running? How have you changed the log format?
Hi Glen,
I'm running Dovecot 2.2.12 build from source with a small change to logout format:
# Logout POP
pop3_logout_format = top=%t/%p, retr=%r/%b, del=%d/%m, size=%s, bytes=%i/%o, session=<%{session}>
# Logout IMAP
imap_logout_format = in=%i out=%o session=<%{session}>
I added the "session" to logout for a law requirement (match the login and logout session).
Hi, I's not practical for me to support customized log formats with this tool, however with some basic knowledge of Perl and regex you should be able to tweak the script to match your customization.
Sub analyse at module dovecot.pm doesn't return any data. Sub wrapup works. What i am doing wrong?
That's not really enough information to go on. Sub analyse is only called on new lines being found by uloganalyser, and then it filters the lines as per the regex. Currently working fine for me on Debian Wheezy, but if you are running something different or have changed log formats then you will need to debug and adapt the plugin for your needs.
Glen,
thanks again for such great tools, all running well here.
if I want to upgrade my installation to the latest scripts you have here:
is it just sufficient to download latest tarball above, OR, do I also need to re-download templates, and, re import over 'old' ones ?
I'll just start with tarball..
That depends a lot on where you are starting from. I've tried to keep the *-stats scripts as consistent as possible so newer ones just add more additional metrics. The uloganalyser plugins should have minimal impact between versions and at most just add more metrics for the *-stats scripts to pick up.
If Cacti templates have changed since the version you have been running then it's a much more tricky subject. Here weird stuff might happen when you import a newer templates. The safest way is to purge all old templates and graphs and start clean otherwise it's down to luck and manual clean-up afterwards if anything goes wrong. It's one area that I've seen some really messy stuff with Cacti.
Glen,
in the page above, 3 lines with loginmethod are doubled up, I think. in the download file, they are OK (but I screen scrapped from here, hence noticed)
thanks again for these pages and utilities,
Voytek
Thanks for pointing that out - hopefully all correct now
Great job.
Small glitch. The dovecot plugin auth section is not parsing correctly failed login.
Everything is tag under dovecot:auth:disallowedchar instead of unknow user and password mismatch.
May 28 12:34:48 xxxxxxxxxxxx dovecot: auth-worker(15299): sql(fdg5sgdg@fakedomain.com,111.111.111.111,<4Z5TsUZtP8zUbhFN>): unknown user
May 28 12:34:49 xxxxxxxxxxxx dovecot: auth-worker(15299): sql(fdg5sgdg@fakedomain.com,111.111.111.111,<4Z5TsUZtP8zUbhFN>): unknown user
May 28 12:34:51 xxxxxxxxxxxx dovecot: auth-worker(15299): sql(fdg5sgdg@fakedomain.com,111.111.111.111,<4Z5TsUZtP8zUbhFN>): unknown user
May 28 12:34:52 xxxxxxxxxxxx dovecot: auth-worker(15299): sql(fdg5sgdg@fakedomain.com,111.111.111.111,<4Z5TsUZtP8zUbhFN>): Password mismatch
I can provide more logs if needed
Thanks for the report. Are there any other lines that relate to these? I need to know in order to ensure that I avoid duplicate counting authentication failures. These could be something like "auth: sql(testuser,111.111.111.111): unknown user" so I need to work out if they are already being counted some other way (so I just need to ignore them) or if these are new scenarios that need counting. I'll get this fixed when I know which is is.
contact me by mail i'll give you a raw log file
I want to avoid sharing log files as it's fraught with legal issues. Can you just tell me if there are other log entries relating to these and what they are (redacted) so I can ensure we don't double count or miss out counting these events. If it's easier you can also raise a bug report with this info on GitHub. Thanks!
Github is a better place ;)
issue #12
many thanks for your work!
I have dovecot logging to it's own dovecot.log and not maillog, so, I run into separate output like
/etc/snmp/uloganalyser /var/log/dovecot.1 /var/log/dovecot.log /var/local/snmp/maild dovecot
that works, but then I noticed stats file is also specified in *-stats-* , so I also should edit all dovecot-stats-* files ?
STATS=/var/local/snmp/maild
I think it worked even before I edited stats to correct stats file ?
I have edited now to reflect correct file
on my newly installed Dovecot stats, I have 23 graphs on Cacti/Dovecot, BUT, only get data in just two, POP3 sessions and IMAP sessions,
and, only collect this data, how to troubleshoot this ?
Dovecot has standard log, EXCEPT, it's own log
# grep log_path *.conf
dovecot.conf:log_path = /var/log/dovecot.log
dovecot.conf: log_path = /var/log/sieve.log
# cat /var/local/snmp/maild
lastrun=1589267401
lastline=157520
lastposition=26251360
repeatline=May 12 17:09:53 pop3(mech-5@aaaaaa.com.au)<21683></khrtghghglI8OTMgHi>: Info: Disconnected: Logged out top=0/0, retr=0/0, del=0/57, size=11673542
dovecot:sessions:imap=30
dovecot:sessions:managesieve=0
dovecot:sessions:pop3=0
dovecot:users:imap=9
dovecot:users:managesieve=0
dovecot:users:pop3=0
lastinode=100664518
Since all this does is match log lines against patterns, the thing to determine is if there are lines matching any other scenarios being logged. From there I'd look at the path logs take. If you log via syslog then maybe you have filters or log levels or other config that excludes the logs that are being parsed.
Looking at the output of doveconf the only thing that are obvious to me that might make a difference is that I have "auth_verbose = yes" but otherwise everything else is default. You might also find that some of the default format configs have changed between versions and does not have some info that is collected for graphs.
I would personally start debugging by selecting one graph that was working before and then go back to the stats file for that, and then the plugin code that collects that and see if I can find log lines containing the patterns or a new equivalent. The plugins are written to make noise about anything they don't recognise so most likely the info is no longer logged.
Ah, sorry, I just noticed you posted 2 messages. Yes, if you need to change the location of the stats file then it will need changing everywhere.
The way this works is that the log analyser and plugins generate stats from the logs and put them in the specified file, then the individual *-stats-* files just grep the relevant stat out of the stats file for smtpd to return.