2011-08-30 07:15 :: Glen Pitt-Pladdy

See some comments on the SNMP basics post regarding tweaking this plugin for different format (dedicated) logs on different systems.

  2012-02-08 17:12 :: Daniele Palumbo

hi,

i have this errors:
unknown dovecot: Feb  5 04:03:00 mail dovecot: POP3(xxx@domain.it): Disconnected: Logged out top=0/0, retr=0/0, del=0/0, size=0
unknown dovecot: Feb  8 15:22:58 mail dovecot: pop3-login: Aborted login (auth failed, 1 attempts): user=<postmaster@domain.it>, method=PLAIN, rip=XX.XXX.XXX.X, lip=XXX.XXX.X.XX

the version is:
20110626 (the last i guess).

thanks,
Daniele

  2012-02-08 19:58 :: Glen Pitt-Pladdy

Thanks for the report - I had not tested pop3 on a live setup as my servers only run imap so it is really useful to get these reports. I have released a new version tonight which should understand these lines and adds an additional "login Aborted" graph.

Let me know if you discover any further lines it does not understand.

  2012-02-09 15:48 :: Daniele Palumbo

Hi again,

seems only that two errors for dovecot (i will look later at postfix, but anyway strings does not go to stderr).

Instead, i've found a bug in the template, it look for "dovecotpop3isconnect" instead of "dovecotpop3disconnect".
I have already patched the file, but how can i send it to you?

Thanks,
Daniele

  2012-02-09 16:08 :: Glen Pitt-Pladdy

Good to hear that the scripts are working better.

Cacti can be have problems re-importing templates so it's best I make the changes in cacti and re-export the templates. I have found the typos you spotted (well done!) and have released a new template. The new template should have a MD5 checksum of 6e8f655721539da3d17c2a7f22e0b23d - if you keep getting the old one then you may need to force reload without cache (shift-F5 in many browsers).

  2012-07-25 17:42 :: Alex

Jul 25 18:22:03 XXXXX dovecot: auth-worker(default): mysql: Connected to localhost (XXXXX)

I don't speak enough PERL for adding the needed code. :(
Can you build an update, please?

Mny Tnx
Alex

  2012-07-26 15:48 :: Glen Pitt-Pladdy

Thanks for submitting the unhandled line. I've just released version 20120726 which should handle those lines.

  2013-06-03 19:13 :: Tomek

Hi, I have one more unhandled line:
Jun  3 20:38:14 chaos dovecot: pop3-login: Login: user=<tomek>, method=PLAIN, rip=192.168.0.135, lip=xx.xx.xx.xx, mpid=24423, TLS, session=<ur1xRETeEADAqACH>
Please help :)

  2013-06-22 13:56 :: Glen Pitt-Pladdy

I think these relate to newer versions of dovecot (eg. in Debian wheezy) and with starting to upgrade my servers I'm starting to see similar. I will release an update shortly to handle the new formats/info in these lines and others. Busy testing it right now....

  2013-06-24 07:04 :: Alex

Hi, I have another unhandled line:

/etc/snmp/uloganalyser-plugin/dovecot.pm 20120501:192 /var/log/mail.log:7842 unknown dovecot: Jun 24 08:45:03 XXXXXXX dovecot: deliver(XX@XXX.XX): sieve: msgid=<XX@XXX.XX>: forwarded to <xx@xxx.xx>

Many thanks for your work :)

  2013-09-20 11:39 :: Vlad

First of all, thanks for your great work! Below I have a few more examples of errors:
dovecot.pm 20121115:157 /var/log/mail_test.log:489 unknown dovecot: Sep 15 06:28:45 mc1 dovecot: imap-login: Login: user=<xxxxxx@xxxxxx>, method=PLAIN, rip=1.1.1.1, lip=1.1.1.1, mpid=27826, secured, session=<ROj3hWTmSQCsFtyb>
dovecot.pm 20121115:157 /var/log/mail_test.log:471 unknown dovecot: Sep 15 06:28:45 mc1 dovecot: pop3-login: Login: user=<xxxxx@xxxxxx>, method=PLAIN, rip=1.1.1.1, lip=1.1.1.1, mpid=27833, secured, session=<J0n3hWTmMACsFtyb>
dovecot.pm 20121115:157 /var/log/mail_test.log:389 unknown dovecot: Sep 15 06:28:01 mc1 dovecot: managesieve-login: Login: user=<xxxxx@xxxxxx>, method=PLAIN, rip=1.1.1.1, lip=1.1.1.1, mpid=27835, secured, session=<UUFQg2Tm8ACsFtya>

  2013-09-21 07:26 :: Glen Pitt-Pladdy

Actually this is all covered in my development version (been updating it for Wheezy) so I've done some extra testing on that and released that today. It should cover a whole load of things in newer versions of Dovecot. The main changes are dovecot.pm, dovecot-stats-deliver and of course the Cacti template which may be messy to update.

  2013-10-07 12:13 :: Vlad

Thanks for the update! Most of the things work fine, besides the cacti template. Can I use the old cacti template with your new scripts?

  2013-10-22 14:59 :: Andreas

Hello!

It seems my post here two days ago is not show so I'll try it again. I am running dovecot 2.1.17. The only graph that is drawn with the Dovecot stats on Cacti tarball 20130920 is the session-graph. Where could it start to check if all the needed stuff is found and paths are right?

Andreas

  2013-10-23 14:13 :: Travis

Glen,

I don't see a link to the cacti template in the article.  Am I missing something?

Travis

  2013-10-24 18:55 :: Glen Pitt-Pladdy

Andreas - I am running 2.1.7 so the version is good. The data goes into the stats file specified (if you are using the example in the uloganalyser post then /var/local/snmp/mail). What is unique about the sessions is that they are picked up from doveadm, whereas everything else is from parsing log files which should be a big clue to what part is not working and where to investigate.

  2013-10-24 18:58 :: Glen Pitt-Pladdy

Travis - the template is in the tarball along with all the plugin (you will still need uloganalyser from the other article), scripts and other bits needed to set this up: cacti_host_template_dovecot.xml

  2013-10-27 01:48 :: Travis Brown

Thanks Glen! I am not sure how I missed that before.

  2013-11-06 14:36 :: Mitch

Thanks for all your work on this!

I'm trying to import into the tarball 20131027 into Cacti 0.8.7i and I'm getting "Error: XML: Hash version does not exist". All the data, graph and host templates import but there is no name on any of them.

Am I missing something obvious?

  2013-11-06 14:52 :: Mitch

I have the same problem with the Cacti file for postfix also.

  2013-11-06 18:55 :: Glen Pitt-Pladdy

This is very likely because the version these were done on is 0.8.8a and your version doesn't know about that. It's possible the ugly stuff in my "Cacti hack for forward compatibility" article will help (or really mess things up), otherwise I expect you will have to upgrade.

  2014-04-30 13:06 :: Alessio

Hi Glen,

I have found an issue with my dovecot log format (2.2.12):

dovecot.pm 20131027:419 /root/mail.log.0:191 unknown dovecot: Apr 29 23:59:59 pop01 dovecot: imap(pippo@pippo.com), session=<HPjxVzX4XgBdLtmW>: Disconnected: Logged out in=564 out=11430

dovecot.pm 20131027:419 /root/mail.log.0:194 unknown dovecot: Apr 29 23:59:59 pop01 dovecot: pop3(pippo@pippo.com), session=<98OajTX4eAC8Dqbj>: Disconnected: Logged out top=0/0, retr=0/0, del=0/0, size=0, bytes=12/43

the difference from my log and your "regex" for log out is that my log starts with session=

I understand that this problem can be fixed by editing this line in dovecot.pm:

elsif ( $line =~ s/^(Disconnected)[:\s]*// or $line =~ s/^(proxy)\([^\)]+\): disconnecting [\da-f\.:]+ // )

but I'm not able, can you help me?

Thanks

  2014-04-30 21:03 :: Alessio

Hi Glen,

I solved the previous problem moving "session" at the end of the line but now your plugin is not any more able to count the in/out bytes.

Here a sample of my "Disconnected" log:

Apr 30 23:01:02 pop01eeh dovecot: imap(pippo@pippo.com): Disconnected: Logged out in=640 out=3883 session=<ACSb2Ej4iABtqHHz>
Apr 30 23:01:02 pop01eeh dovecot: pop3(pippo@pippo.com): Disconnected: Logged out top=0/0, retr=0/0, del=0/0, size=0, bytes=12/43, session=<jAaa2Ej4cAAuLOvd>

Can you help me?
Thanks

  2014-04-30 21:17 :: Glen Pitt-Pladdy

What version of Dovecot and distro are you running? How have you changed the log format?

  2014-05-05 14:54 :: Alessio

Hi Glen,

I'm running Dovecot 2.2.12 build from source with a small change to logout format:

# Logout POP
pop3_logout_format = top=%t/%p, retr=%r/%b, del=%d/%m, size=%s, bytes=%i/%o, session=<%{session}>

# Logout IMAP
imap_logout_format = in=%i out=%o session=<%{session}>

I added the "session" to logout for a law requirement (match the login and logout session).

  2014-05-06 07:29 :: Glen Pitt-Pladdy

Hi, I's not practical for me to support customized log formats with this tool, however with some basic knowledge of Perl and regex you should be able to tweak the script to match your customization.

  2014-12-16 08:56 :: skeletor

Sub analyse at module dovecot.pm doesn't return any data. Sub wrapup works. What i am doing wrong?

  2014-12-22 15:54 :: Glen Pitt-Pladdy

That's not really enough information to go on. Sub analyse is only called on new lines being found by uloganalyser, and then it filters the lines as per the regex. Currently working fine for me on Debian Wheezy, but if you are running something different or have changed log formats then you will need to debug and adapt the plugin for your needs.

  2015-02-12 07:14 :: voytek

Glen,
thanks again for such great tools, all running well here.

if I want to upgrade my installation to the latest scripts you have here:

is it just sufficient to download latest tarball above, OR, do I also need to re-download templates, and, re import over 'old' ones ?

I'll just start with tarball..

  2015-02-12 20:18 :: Glen Pitt-Pladdy

That depends a lot on where you are starting from. I've tried to keep the *-stats scripts as consistent as possible so newer ones just add more additional metrics. The uloganalyser plugins should have minimal impact between versions and at most just add more metrics for the *-stats scripts to pick up.

If Cacti templates have changed since the version you have been running then it's a much more tricky subject. Here weird stuff might happen when you import a newer templates. The safest way is to purge all old templates and graphs and start clean otherwise it's down to luck and manual clean-up afterwards if anything goes wrong. It's one area that I've seen some really messy stuff with Cacti.

  2018-01-22 11:18 :: Voytek Eymont

Glen,

in the page above, 3 lines with loginmethod are doubled up, I think. in the download file, they are OK (but I screen scrapped from here, hence noticed)

thanks again for these pages and utilities,

Voytek

  2018-05-28 16:39 :: Yves

Great job.
Small glitch. The dovecot plugin auth section is not parsing correctly failed login.
Everything is tag under dovecot:auth:disallowedchar instead of unknow user and password mismatch.

May 28 12:34:48 xxxxxxxxxxxx dovecot: auth-worker(15299): sql(fdg5sgdg@fakedomain.com,111.111.111.111,<4Z5TsUZtP8zUbhFN>): unknown user
May 28 12:34:49 xxxxxxxxxxxx dovecot: auth-worker(15299): sql(fdg5sgdg@fakedomain.com,111.111.111.111,<4Z5TsUZtP8zUbhFN>): unknown user
May 28 12:34:51 xxxxxxxxxxxx dovecot: auth-worker(15299): sql(fdg5sgdg@fakedomain.com,111.111.111.111,<4Z5TsUZtP8zUbhFN>): unknown user
May 28 12:34:52 xxxxxxxxxxxx dovecot: auth-worker(15299): sql(fdg5sgdg@fakedomain.com,111.111.111.111,<4Z5TsUZtP8zUbhFN>): Password mismatch

I can provide more logs if needed

  2018-05-29 11:09 :: Yves

contact me by mail i'll give you a raw log file

  2018-05-30 14:58 :: Yves

Github is a better place ;)

issue #12