Re: [courier-users] logfile analysis
Am Donnerstag, 12. August 2004 09:38 schrieb Terrel Shumway: It's pretty scary to try to get something useful from courier's logs because there is no standard format for log messages. It's not that heavy...Maybe it could be easier :) First: does anyone have a greppy perl script or such that will make sense of the log files? On http://www.courier-mta.org is a link on cla (This one: http://zak.ecotroph.net/~anewton/cla/) and another log analyzer (that one: http://www.enesbe.com.au/cgi-bin/wiki.pl?EnesbeDownloads) which requires webmin. I've only tested cla and it produces something like that: ---snip--- == SMTP Daily Totals = SMTP Local DeliveryLocal SMTPRelay SMTP Broken 5XX Freemail Day Connects Deliveries SizeErrors Relays SizeErrors Pipes Errors Errors -- -- -- -- -- -- -- -- -- Aug 15 174 182 770202 0 0 0 4 10 4 0 Aug 14 410 546 2803820 0 0 0 2 0 2 0 Aug 13 7001028 5106002 0 29874 20 2 20 0 Aug 12 5041068 5382788 0 0 0 8 4 4 0 ---snap--- Well, it looks better in a console... I am willing to go through the code and standardize the current log messages if no one objects. (It won't be all at once, but I'll get to it a little at a time.) If you do this, create a patch of it, so everybody could think about, if he/she want's to destroy compatibility to logwatch (I don't use logwatch, so there is no need to be compatible to it) Dream BIG! -- Terrel HAND, Steffen --- SF.Net email is sponsored by Shop4tech.com-Lowest price on Blank Media 100pk Sonic DVD-R 4x for only $29 -100pk Sonic DVD+R for only $33 Save 50% off Retail on Ink Toner - Free Shipping and Free Gift. http://www.shop4tech.com/z/Inkjet_Cartridges/9_108_r285 ___ courier-users mailing list [EMAIL PROTECTED] Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] logfile analysis
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Thursday 12 August 2004 09:38, Terrel Shumway wrote: It's pretty scary to try to get something useful from courier's logs because there is no standard format for log messages. First: does anyone have a greppy perl script or such that will make sense of the log files? I am in the process of writing a small (and probably clumsy - I'm not much of a shell script genius) shell script. Not even sure if it'll work 100% with other shells than bash. www.sentinel.dk/cookbook/courier-mail-statistics.sh One should have geoip installed as well (not a requirement, though). The output it generates looks like this: /var/log/mail.log.4.gz (Jul 4 06:48:02 - Jul 11 06:46:40) (weekly report) Processed mail: Incoming: 4352, Outgoing: 498 Relay attempts: 17 Mail sent to hosted domain but with unknown user: 1190 IP addresses behind relay attempts (top ten): 62.42.15.137 ES, Spain 4 218.5.109.188CN, China 4 208.31.42.77 US, United States 4 61.249.159.157 KR, Korea, Republic of 1 222.101.168.37 KR, Korea, Republic of 1 211.180.125.238 KR, Korea, Republic of 1 210.64.169.222 TW, Taiwan 1 195.228.231.80 HU, Hungary 1 IP addresses behind user unknown attempts (probably spammers) (top ten): 80.98.168.39 HU, Hungary31 62.68.170.228HU, Hungary29 65.254.39.234US, United States 28 195.228.164.82 HU, Hungary25 82.100.12.61 CZ, Czech Republic 24 62.65.166.4 SK, Slovakia 21 62.168.123.42SK, Slovakia 21 209.51.152.146 US, United States 20 212.92.26.195HU, Hungary19 194.88.52.2 HU, Hungary19 B/R, - -- Frederik Dannemare | mailto:[EMAIL PROTECTED] http://qa.debian.org/developer.php?login=Frederik+Dannemare http://frederik.dannemare.net | http://www.linuxworlddomination.dk Key fingerprint: BB7B 078A 0DBF 7663 180A F84A 2D25 FAD5 9C4E B5A8 -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.5 (GNU/Linux) iD8DBQFBHc/9LSX61ZxOtagRAuVNAJ49Jusvzf6TViPS69aDJSeqb/Ks0ACgj6QJ J2umjS/NES3gEK96XUPqvHc= =GY9C -END PGP SIGNATURE- --- SF.Net email is sponsored by Shop4tech.com-Lowest price on Blank Media 100pk Sonic DVD-R 4x for only $29 -100pk Sonic DVD+R for only $33 Save 50% off Retail on Ink Toner - Free Shipping and Free Gift. http://www.shop4tech.com/z/Inkjet_Cartridges/9_108_r285 ___ courier-users mailing list [EMAIL PROTECTED] Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
[courier-users] logfile analysis
It's pretty scary to try to get something useful from courier's logs because there is no standard format for log messages. First: does anyone have a greppy perl script or such that will make sense of the log files? Second: does anyone object to coming up with a standard way to format log messages? Here are some ideas: timestamp host level program class [structured] freeform timestamp: maybe some iso format like 2004-08-12T23:54:52, but I really don't care as long as it's standard. It would be nice to record the year. level = (debug|info|status|warning|error|critical...) program = (courierd,esmtpd,courier-fax,courier-imap...) class = one of a standard set of message types structured = (key=value,...) where keys are [-a-zA-Z0-9_]+ and values have no whitespace or commas freeform = anything goes From looking at my log files, and the courier source (very quickly), I came up with these classes: status (message status: stuff that would go in access_log instead of error_log if we were talking about a web server) status-failure status-success status-success-delivered status-deferred status-rejected status-rejected-bofh status-rejected-bofh-sender status-rejected-bofh-spamtrap status-rejected-bofh-network (rbl, relay, bad mx...) status-rejected-bofh-content (e.g. spamassasin) status-cancelled status-smtp status-smtp-250 status-connect status-connect-closed error error-system (errors from system and standard libary calls) error-config (misconfigurations) error-communication (unexpected input or output from another courier program) error-protocol (protocol/rfc violations) error-network (unexpected network conditions) error-network-dns error-network-connect (refused, timed-out, closed, ...) error-internal (sumthin' ain't right) error-denied error-denied-baduser error-denied-badpassword error-external (error output from third-party programs) error-smtp error-smtp-517 ... trace (program logic tracing) Feedback please. I am willing to go through the code and standardize the current log messages if no one objects. (It won't be all at once, but I'll get to it a little at a time.) Dream BIG! -- Terrel --- SF.Net email is sponsored by Shop4tech.com-Lowest price on Blank Media 100pk Sonic DVD-R 4x for only $29 -100pk Sonic DVD+R for only $33 Save 50% off Retail on Ink Toner - Free Shipping and Free Gift. http://www.shop4tech.com/z/Inkjet_Cartridges/9_108_r285 ___ courier-users mailing list [EMAIL PROTECTED] Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] logfile analysis
Terrel Shumway wrote: It's pretty scary to try to get something useful from courier's logs because there is no standard format for log messages. First: does anyone have a greppy perl script or such that will make sense of the log files? I have a script at http://perlstalker.amigo.net/courier/scripts/courier-stats but it's more a stats thing then a real analyzer. It may get you started though. Second: does anyone object to coming up with a standard way to format log messages? Standards are good, IMO. [snip: format suggestion] -- Randy Smith Amigo.Net Systems Administrator 719-589-6100 / 888-759-4430 x 4185 http://www.amigo.net/ --- SF.Net email is sponsored by Shop4tech.com-Lowest price on Blank Media 100pk Sonic DVD-R 4x for only $29 -100pk Sonic DVD+R for only $33 Save 50% off Retail on Ink Toner - Free Shipping and Free Gift. http://www.shop4tech.com/z/Inkjet_Cartridges/9_108_r285 ___ courier-users mailing list [EMAIL PROTECTED] Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
