Re: [AMaViS-user] amavis-stats 0.1.20 problem - Is this the right place to ask?
On Jan 19, 2006, at 03:54 , Stephen Carter wrote: Stephen Carter Retrac Networking Limited www: http://www.retnet.co.uk Ph: +44 (0)7870 218 693 Fax: +44 (0)870 7060 056 CNA, CNE, CNS, CCNA, MCP Dale Walsh [EMAIL PROTECTED] 01/18/06 9:30 PM On Jan 18, 2006, at 14:38 , Stephen Carter wrote: I didn't find a current amavis-stats list or openly available e- mail address for dale walsh (at least not in the amavis-stats download) so I thought I'd ask here. I'm running SuSE 9.3 and have amavis-stats v 0.1.20 pretty much running fine, apart from the following php errors when I access the stats web page: Warning: Cannot modify header information - headers already sent by (output started at /usr/local/share/amavis-stats/amavis-stats.php: 277) in /usr/local/share/amavis-stats/includes/page_header.php on line 41 Warning: Cannot modify header information - headers already sent by (output started at /usr/local/share/amavis-stats/amavis-stats.php: 277) in /usr/local/share/amavis-stats/includes/page_header.php on line 47 Warning: Cannot modify header information - headers already sent by (output started at /usr/local/share/amavis-stats/amavis-stats.php: 277) in /usr/local/share/amavis-stats/includes/page_header.php on line 48 This occurs when some output is generated in the web interface before it outputs the html page. I did a little hunting around and found some general coding errors that can cause this and couldn't find anything similar in amavis- stats although I'm not a programmer so I wasn't counting on finding the problem there. Can someome tell me how to fix this problem? The alternative I have is to comment out the offending lines, but this disables what appears to be some no-cache directives in the original php code which I'd like to keep if possible. I see a small error in my logic where I don't test for the existence of virus data before calling rrd_graph, I'll correct this and have a solution soon but seeing as how you now have virus data you wont be experiencing this problem anymore. Thanks, Stephen Carter -- Dale Thanks Dale. Some time after I posted the e-mail I continued testing by sending through bogus e-mails, thus generating the required data and the errors went away. Although I didn't post the url, I suppose it was easy enough to guess! from the URL I can see how you installed it and where all of the files used for it are on your drive, this aids in debugging. One problem that remains which I'm still trying to track down though is that banned files are counted twice per e-mail. As a guess I think the amavis-stats code counts the number of 'BLOCKED Banned' instances it finds, as when I run a grep on the amavis log file I find 2 instances of this tag for each banned file e-mail it comes across. There also appears to be a unique processing ID (I think) that amavisd adds to each line in the log as it processes an e-mail which I can see has the same ID for each Banned instance found so I know it's for the same e-mail. I'm running amavisd-new v2.2.0 with a log level of 2 so will look into tweaking it to see if I can remove the 2nd instance of this appearing, although I like seeing the additional info that loglevel 2 gives, which helps when I very occasionally troubleshoot scoring. I don't believe the log_level is the issue since I've run it using log level 5 and it doesn't get confused. mustangrestomods:/SourceCache root# cat /etc/amavisd.conf|grep log_level = $log_level = 2; # (defaults to 0) mustangrestomods:/SourceCache root# The version of amavisd I'm using is actually the Maia Mailgaurd modified version so maybe that is playing a part in this somewhere.. This might have something to do with it however, amavis-stats checks the processing ID to ensure that it's not counted more than once so I can't see how this could be a problem. Also looking at the page, I'm not sure it's really counting the same processing ID twice but since I'm not sure what affect the maia mods have on it, it may just be a re-injection that your seeing because the function that catches Blocked BANNED also catches Blocked INFECTED and it only caught this once. --- This SF.net email is sponsored by: Splunk Inc. Do you grep through log files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! http://sel.as-us.falkag.net/sel?cmd=lnkkid=103432bid=230486dat=121642 ___ AMaViS-user mailing list AMaViS-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/amavis-user AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3 AMaViS-HowTos:http://www.amavis.org/howto/
Re: [AMaViS-user] amavis-stats 0.1.20 problem - Is this the right place to ask?
Dale Walsh wrote: On Jan 19, 2006, at 03:54 , Stephen Carter wrote: One problem that remains which I'm still trying to track down though is that banned files are counted twice per e-mail. As a guess I think the amavis-stats code counts the number of 'BLOCKED Banned' instances it finds, as when I run a grep on the amavis log file I find 2 instances of this tag for each banned file e-mail it comes across. There also appears to be a unique processing ID (I think) that amavisd adds to each line in the log as it processes an e-mail which I can see has the same ID for each Banned instance found so I know it's for the same e-mail. I'm running amavisd-new v2.2.0 with a log level of 2 so will look into tweaking it to see if I can remove the 2nd instance of this appearing, although I like seeing the additional info that loglevel 2 gives, which helps when I very occasionally troubleshoot scoring. I don't believe the log_level is the issue since I've run it using log level 5 and it doesn't get confused. mustangrestomods:/SourceCache root# cat /etc/amavisd.conf|grep log_level = $log_level = 2; # (defaults to 0) mustangrestomods:/SourceCache root# Hi Dale Like I already wrote you before, this really seems to BE a problem, I'm running at 3. But it doesn't come up in lower levels (at least not in 0). Here is a problematic snip: BTW: There's a bug in the statistics if one uses a debug level greater than 0 For some viruses it then happens that it gets reported twice. once as VIRUS and once as BANNED That happens because amavisd-new spits out some intermediate results in debug levels 1 and more. This might look like this: Jan 6 09:31:13 www.mindblow.ch /usr/sbin/amavisd[20255]: (20255-06) Blocked INFECTED (Worm.SomeFool.Gen-2), [62.2.174.133] - [EMAIL PROTECTED], quarantine: virus/virus-20060106-093112-20255-06.gz, Message-ID: [EMAIL PROTECTED], mail_id: YmNLvRSucV83, Hits: -, 524 ms server:/var/log # grep Jan 6 09:31:1 amavis.log Jan 6 09:31:12 www.mindblow.ch /usr/sbin/amavisd[20255]: (20255-06) LMTP::10024 /var/spool/amavis/tmp/amavis-20060106T054228-20255: - [EMAIL PROTECTED] Received: SIZE=34169 from server.mindblow.ch ([127.0.0.1]) by localhost (www.mindblow.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 20255-06 for [EMAIL PROTECTED]; Fri, 6 Jan 2006 09:31:12 +0100 (CET) Jan 6 09:31:12 www.mindblow.ch /usr/sbin/amavisd[20255]: (20255-06) Checking: YmNLvRSucV83 - [EMAIL PROTECTED] Jan 6 09:31:13 www.mindblow.ch /usr/sbin/amavisd[20255]: (20255-06) p.path BANNED:1 [EMAIL PROTECTED]: P=p005,L=1,M=multipart/report | P=p006,L=1/3,M=message/rfc822 | P=p007,L=1/3/1,M=multipart/mixed | P=p004,L=1/3/1/2,M=application/x-zip-compressed,T=zip,N=ps.zip | P=p008,L=1/3/1/2/1,T=exe,N=ps.rtf.scr, matching_key=(?i-xsm:\\.[^./]*[A-Za-z][^./]*\\.(exe|vbs|pif|lnk|scr|bat|cmd|com|cpl|dll)\\.?$) Jan 6 09:31:13 www.mindblow.ch /usr/sbin/amavisd[20255]: (20255-06) local delivery: - virus-quarantine, mbx=/var/spool/amavis/quarantine/virus/virus-20060106-093112-20255-06.gz Jan 6 09:31:13 www.mindblow.ch /usr/sbin/amavisd[20255]: (20255-06) Blocked INFECTED (Worm.SomeFool.Gen-2), [62.2.174.133] - [EMAIL PROTECTED], quarantine: virus/virus-20060106-093112-20255-06.gz, Message-ID: [EMAIL PROTECTED], mail_id: YmNLvRSucV83, Hits: -, 524 ms As far as I could find out, the p.path BANNED is just an intermediate information which should NOT be taken into account by the script.. ! Matt --- This SF.net email is sponsored by: Splunk Inc. Do you grep through log files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! http://sel.as-us.falkag.net/sel?cmd=lnkkid=103432bid=230486dat=121642 ___ AMaViS-user mailing list AMaViS-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/amavis-user AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3 AMaViS-HowTos:http://www.amavis.org/howto/
Re: [AMaViS-user] amavis-stats 0.1.20 problem - Is this the right place to ask?
On Jan 19, 2006, at 15:45 , Matthias Keller wrote: Dale Walsh wrote: On Jan 19, 2006, at 03:54 , Stephen Carter wrote: One problem that remains which I'm still trying to track down though is that banned files are counted twice per e-mail. As a guess I think the amavis-stats code counts the number of 'BLOCKED Banned' instances it finds, as when I run a grep on the amavis log file I find 2 instances of this tag for each banned file e-mail it comes across. There also appears to be a unique processing ID (I think) that amavisd adds to each line in the log as it processes an e-mail which I can see has the same ID for each Banned instance found so I know it's for the same e-mail. I'm running amavisd-new v2.2.0 with a log level of 2 so will look into tweaking it to see if I can remove the 2nd instance of this appearing, although I like seeing the additional info that loglevel 2 gives, which helps when I very occasionally troubleshoot scoring. I don't believe the log_level is the issue since I've run it using log level 5 and it doesn't get confused. mustangrestomods:/SourceCache root# cat /etc/amavisd.conf|grep log_level = $log_level = 2; # (defaults to 0) mustangrestomods:/SourceCache root# Hi Dale Like I already wrote you before, this really seems to BE a problem, I'm running at 3. But it doesn't come up in lower levels (at least not in 0). I've upped the log level to 5 on 4 different version of darwin/ amavisd-new and I'm not experiencing any problems with double counting. I have darwin 7.9 using amavisd-new-2.2.0 unmodified and darwin 7.9 using amavisd-new-2.2.0 modified with the advanced header patch and then darwin 8.4 using amavisd-new 2.3.3 unmodified and then darwin 8.4 using amavisd-new 2.3.3 using the advanced header patch. It might be the maia modification made to the amavisd-new app that are causing the problems because I've been unable to duplicate it using a standard amavisd-new or amavisd-new with the advanced header patch. I've wrapped amavisd-new up in a configure/Makefile package that will build, configure amavisd-new and include options to build amavis- milter, apply the advanced header patch if you want it and applying an OS specific patch and includes OS specific startup items all in one shot making installation easier requiring less user edits to get amavisd-new up and running. I've tested the install method on a couple of different OS's besides darwin and have installed amavis-stats on redhat using my amavisd-new install package and it doesn't experience any double counting so all I can conclude is it has to be some patch that is applied to amavisd- new that is causing the problem. Here is a problematic snip: BTW: There's a bug in the statistics if one uses a debug level greater than 0 For some viruses it then happens that it gets reported twice. once as VIRUS and once as BANNED That happens because amavisd-new spits out some intermediate results in debug levels 1 and more. This might look like this: Jan 6 09:31:13 www.mindblow.ch /usr/sbin/amavisd[20255]: (20255-06) Blocked INFECTED (Worm.SomeFool.Gen-2), [62.2.174.133] - [EMAIL PROTECTED], quarantine: virus/ virus-20060106-093112-20255-06.gz, Message-ID: [EMAIL PROTECTED], mail_id: YmNLvRSucV83, Hits: -, 524 ms server:/var/log # grep Jan 6 09:31:1 amavis.log Jan 6 09:31:12 www.mindblow.ch /usr/sbin/amavisd[20255]: (20255-06) LMTP::10024 /var/spool/amavis/tmp/amavis-20060106T054228-20255: - [EMAIL PROTECTED] Received: SIZE=34169 from server.mindblow.ch ([127.0.0.1]) by localhost (www.mindblow.ch [127.0.0.1]) (amavisd- new, port 10024) with LMTP id 20255-06 for [EMAIL PROTECTED]; Fri, 6 Jan 2006 09:31:12 +0100 (CET) Jan 6 09:31:12 www.mindblow.ch /usr/sbin/amavisd[20255]: (20255-06) Checking: YmNLvRSucV83 - [EMAIL PROTECTED] Jan 6 09:31:13 www.mindblow.ch /usr/sbin/amavisd[20255]: (20255-06) p.path BANNED:1 [EMAIL PROTECTED]: P=p005,L=1,M=multipart/report | P=p006,L=1/3,M=message/rfc822 | P=p007,L=1/3/1,M=multipart/mixed | P=p004,L=1/3/1/2,M=application/x-zip-compressed,T=zip,N=ps.zip | P=p008,L=1/3/1/2/1,T=exe,N=ps.rtf.scr, matching_key=(?i-xsm:\\. [^./]*[A-Za-z][^./]*\\.(exe|vbs|pif|lnk|scr|bat|cmd|com|cpl|dll)\\.? $) Jan 6 09:31:13 www.mindblow.ch /usr/sbin/amavisd[20255]: (20255-06) local delivery: - virus-quarantine, mbx=/var/spool/amavis/ quarantine/virus/virus-20060106-093112-20255-06.gz Jan 6 09:31:13 www.mindblow.ch /usr/sbin/amavisd[20255]: (20255-06) Blocked INFECTED (Worm.SomeFool.Gen-2), [62.2.174.133] - [EMAIL PROTECTED], quarantine: virus/ virus-20060106-093112-20255-06.gz, Message-ID: [EMAIL PROTECTED], mail_id: YmNLvRSucV83, Hits: -, 524 ms As far as I could find out, the p.path BANNED is just an intermediate information which should NOT be taken into account by the script.. ! p.path BANNED is not counted as far as I can see and as I stated earlier, I've been able to
Re: [AMaViS-user] amavis-stats 0.1.20 problem - Is this the right place to ask?
On Jan 18, 2006, at 14:38 , Stephen Carter wrote: I didn't find a current amavis-stats list or openly available e- mail address for dale walsh (at least not in the amavis-stats download) so I thought I'd ask here. I'm running SuSE 9.3 and have amavis-stats v 0.1.20 pretty much running fine, apart from the following php errors when I access the stats web page: Warning: Cannot modify header information - headers already sent by (output started at /usr/local/share/amavis-stats/amavis-stats.php: 277) in /usr/local/share/amavis-stats/includes/page_header.php on line 41 Warning: Cannot modify header information - headers already sent by (output started at /usr/local/share/amavis-stats/amavis-stats.php: 277) in /usr/local/share/amavis-stats/includes/page_header.php on line 47 Warning: Cannot modify header information - headers already sent by (output started at /usr/local/share/amavis-stats/amavis-stats.php: 277) in /usr/local/share/amavis-stats/includes/page_header.php on line 48 This occurs when some output is generated in the web interface before it outputs the html page. I did a little hunting around and found some general coding errors that can cause this and couldn't find anything similar in amavis- stats although I'm not a programmer so I wasn't counting on finding the problem there. Can someome tell me how to fix this problem? The alternative I have is to comment out the offending lines, but this disables what appears to be some no-cache directives in the original php code which I'd like to keep if possible. I see a small error in my logic where I don't test for the existence of virus data before calling rrd_graph, I'll correct this and have a solution soon but seeing as how you now have virus data you wont be experiencing this problem anymore. Thanks, Stephen Carter -- Dale --- This SF.net email is sponsored by: Splunk Inc. Do you grep through log files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! http://sel.as-us.falkag.net/sel?cmd=lnkkid=103432bid=230486dat=121642 ___ AMaViS-user mailing list AMaViS-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/amavis-user AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3 AMaViS-HowTos:http://www.amavis.org/howto/
