Re: 3.1.2 issue with UnixNFSSafe.pm ?
wrote on Wed, 31 May 2006 19:37:54 -0400 (EDT): some under the 5.6.1 site_perl, some under 5.005. Did you notice that you didn't say anything about this earlier? ;-) It seems this is the cause of your problem. Kai -- Kai Schätzl, Berlin, Germany Get your web at Conactive Internet Services: http://www.conactive.com
RE: 3.1.2 issue with UnixNFSSafe.pm ?
[EMAIL PROTECTED] wrote: A couple of days after an upgrade from 3.0.4 to 3.1.2, I'm noticing that it seems alot slower. I turned off most network tests, including DCC, Pyzor and Razor and it still looks like there's an issue. I raised max children from 15 to 25, yet it still seems to be spending most of it's time at 25, and smtp connections are stacking up behind it and occasionally spamd is so overwhelmed a spam gets through with no checks. CPU also spikes up to over 20.0 at times, on a dual Xeon server with maybe a thousand mailboxes. In the logs, the only thing I see that's showing an issue with SA is this: May 31 07:53:52 mail spamd[59117]: Use of uninitialized value in subtraction (-) at /usr/local/lib/perl5/site_perl/5.6.1/Mail/SpamAssassin/Locker/UnixNFSSafe.pm line 102, GEN108 line 46. Could this be causing children to hang? They seem to take forever to exit... Sounds like it could be a memory issue. Check your memory usage and see if you are going into swap. If so, lower max children until it stops using swap. Once SA starts using swap memory, performance goes way down. The network tests are unlikely to be the culprit. Usually, excessive memory use is caused by having one or two really large add-on rule sets. -- Bowie
RE: 3.1.2 issue with UnixNFSSafe.pm ?
On Wed, 31 May 2006, Bowie Bailey wrote: [EMAIL PROTECTED] wrote: A couple of days after an upgrade from 3.0.4 to 3.1.2, I'm noticing that it seems alot slower. I turned off most network tests, including DCC, Pyzor and Razor and it still looks like there's an issue. I raised max children from 15 to 25, yet it still seems to be spending most of it's time at 25, and smtp connections are stacking up behind it and occasionally spamd is so overwhelmed a spam gets through with no checks. CPU also spikes up to over 20.0 at times, on a dual Xeon server with maybe a thousand mailboxes. In the logs, the only thing I see that's showing an issue with SA is this: May 31 07:53:52 mail spamd[59117]: Use of uninitialized value in subtraction (-) at /usr/local/lib/perl5/site_perl/5.6.1/Mail/SpamAssassin/Locker/UnixNFSSafe.pm line 102, GEN108 line 46. Could this be causing children to hang? They seem to take forever to exit... Sounds like it could be a memory issue. Check your memory usage and see if you are going into swap. If so, lower max children until it stops using swap. Once SA starts using swap memory, performance goes way down. The network tests are unlikely to be the culprit. Usually, excessive memory use is caused by having one or two really large add-on rule sets. According to top, it most definitely does not look like a memory issue...more CPU than anything: last pid: 21486; load averages: 30.71, 27.04, 21.48up 1+18:19:37 11:10:07750 processes: 22 running, 727 sleeping, 1 zombie CPU states: 86.0% user, 0.0% nice, 13.6% system, 0.4% interrupt, 0.0% idle Mem: 662M Active, 948M Inact, 297M Wired, 56M Cache, 199M Buf, 48M Free Swap: 4000M Total, 4000M Free James Smallacombe PlantageNet, Inc. CEO and Janitor [EMAIL PROTECTED] http://3.am =
RE: 3.1.2 issue with UnixNFSSafe.pm ?
[EMAIL PROTECTED] wrote: On Wed, 31 May 2006, Bowie Bailey wrote: [EMAIL PROTECTED] wrote: A couple of days after an upgrade from 3.0.4 to 3.1.2, I'm noticing that it seems alot slower. I turned off most network tests, including DCC, Pyzor and Razor and it still looks like there's an issue. I raised max children from 15 to 25, yet it still seems to be spending most of it's time at 25, and smtp connections are stacking up behind it and occasionally spamd is so overwhelmed a spam gets through with no checks. CPU also spikes up to over 20.0 at times, on a dual Xeon server with maybe a thousand mailboxes. In the logs, the only thing I see that's showing an issue with SA is this: May 31 07:53:52 mail spamd[59117]: Use of uninitialized value in subtraction (-) at /usr/local/lib/perl5/site_perl/5.6.1/Mail/SpamAssassin/Locker/UnixNFSSafe.pm line 102, GEN108 line 46. Could this be causing children to hang? They seem to take forever to exit... Sounds like it could be a memory issue. Check your memory usage and see if you are going into swap. If so, lower max children until it stops using swap. Once SA starts using swap memory, performance goes way down. The network tests are unlikely to be the culprit. Usually, excessive memory use is caused by having one or two really large add-on rule sets. According to top, it most definitely does not look like a memory issue...more CPU than anything: last pid: 21486; load averages: 30.71, 27.04, 21.48up 1+18:19:37 11:10:07750 processes: 22 running, 727 sleeping, 1 zombie CPU states: 86.0% user, 0.0% nice, 13.6% system, 0.4% interrupt, 0.0% idle Mem: 662M Active, 948M Inact, 297M Wired, 56M Cache, 199M Buf, 48M Free Swap: 4000M Total, 4000M Free Interesting. Most of the slowdown problems I have seen have been related to memory. I don't know about that error message, I'll leave that question to someone else. What rulesets are you running? Have you created any custom rules? Maybe one of those is causing problems. -- Bowie
RE: 3.1.2 issue with UnixNFSSafe.pm ?
[EMAIL PROTECTED] wrote: A couple of days after an upgrade from 3.0.4 to 3.1.2, I'm noticing that it seems alot slower. I turned off most network tests, including DCC, Pyzor and Razor and it still looks like there's an issue. Is it possible that you're locking on a Bayes write? Try disabling Bayes temporarily. If that fixes it, move to a SQL-based Bayes instead of DBM. -- Matthew.van.Eerde (at) hbinc.com 805.964.4554 x902 Hispanic Business Inc./HireDiversity.com Software Engineer
Re: 3.1.2 issue with UnixNFSSafe.pm ?
On Wed, May 31, 2006 at 11:12:03AM -0400, [EMAIL PROTECTED] wrote: May 31 07:53:52 mail spamd[59117]: Use of uninitialized value in subtraction (-) at /usr/local/lib/perl5/site_perl/5.6.1/Mail/SpamAssassin/Locker/UnixNFSSafe.pm line 102, GEN108 line 46. Hrm. That's not good. Seems like there's a race condition in there. :( 1) If you can open a bz ticket about that, we ought to prod the code and try to eliminate the race. 2) If the files are on local disk, switch to using the flock method instead. It's much better, imo. -- Randomly Generated Tagline: CPU-angle has to be adjusted because of vibrations coming from the nearby road. - Today's BOFH Excuse pgpRq7t7xJjZa.pgp Description: PGP signature
RE: 3.1.2 issue with UnixNFSSafe.pm ?
On Wed, 31 May 2006 [EMAIL PROTECTED] wrote: [EMAIL PROTECTED] wrote: A couple of days after an upgrade from 3.0.4 to 3.1.2, I'm noticing that it seems alot slower. I turned off most network tests, including DCC, Pyzor and Razor and it still looks like there's an issue. Is it possible that you're locking on a Bayes write? Try disabling Bayes temporarily. If that fixes it, move to a SQL-based Bayes instead of DBM. Ugh...I have seen some locking errors while I wasn't before. WOuld simply turning off auto-learning accomplish the same thing? If it needs to be turned off altogether, will just commenting out bayes_path in local.cf do it? I don't see any other reference in the docs... James Smallacombe PlantageNet, Inc. CEO and Janitor [EMAIL PROTECTED] http://3.am =
Re: 3.1.2 issue with UnixNFSSafe.pm ?
On Wed, 31 May 2006, Theo Van Dinter wrote: On Wed, May 31, 2006 at 11:12:03AM -0400, [EMAIL PROTECTED] wrote: May 31 07:53:52 mail spamd[59117]: Use of uninitialized value in subtraction (-) at /usr/local/lib/perl5/site_perl/5.6.1/Mail/SpamAssassin/Locker/UnixNFSSafe.pm line 102, GEN108 line 46. Hrm. That's not good. Seems like there's a race condition in there. :( 1) If you can open a bz ticket about that, we ought to prod the code and try to eliminate the race. Will do. 2) If the files are on local disk, switch to using the flock method instead. It's much better, imo. Wow, I didn't even know that was a config option (didn't see it in the docs). I just switched it and so far, it appears to have done the trick, although the condition was coming and going, so I'll keep an eye on it. Thanks again! James Smallacombe PlantageNet, Inc. CEO and Janitor [EMAIL PROTECTED] http://3.am =
RE: 3.1.2 issue with UnixNFSSafe.pm ?
[EMAIL PROTECTED] wrote: Ugh...I have seen some locking errors while I wasn't before. WOuld simply turning off auto-learning accomplish the same thing? Possibly. If it needs to be turned off altogether, will just commenting out bayes_path in local.cf do it? I don't see any other reference in the docs... See perldoc Mail::SpamAssassin::Conf, LEARNING OPTIONS use_bayes 0 (default is 1) -- Matthew.van.Eerde (at) hbinc.com 805.964.4554 x902 Hispanic Business Inc./HireDiversity.com Software Engineer
Re: 3.1.2 issue with UnixNFSSafe.pm ?
On Wed, 31 May 2006 [EMAIL PROTECTED] wrote: On Wed, 31 May 2006, Theo Van Dinter wrote: On Wed, May 31, 2006 at 11:12:03AM -0400, [EMAIL PROTECTED] wrote: May 31 07:53:52 mail spamd[59117]: Use of uninitialized value in subtraction (-) at /usr/local/lib/perl5/site_perl/5.6.1/Mail/SpamAssassin/Locker/UnixNFSSafe.pm line 102, GEN108 line 46. Hrm. That's not good. Seems like there's a race condition in there. :( 1) If you can open a bz ticket about that, we ought to prod the code and try to eliminate the race. Will do. 2) If the files are on local disk, switch to using the flock method instead. It's much better, imo. Wow, I didn't even know that was a config option (didn't see it in the docs). I just switched it and so far, it appears to have done the trick, although the condition was coming and going, so I'll keep an eye on it. It appeared to have fixed the race condition from what I could see...cpu was down, number of children, smtp connections, etc. However, about 12 minutes ago, the spamd parent died for the second time since the upgrade. Here is all I could find in the maillog: May 31 12:59:44 mail spamd[35583]: spamd: result: Y 12 - BAYES_80,FORGED_RCVD_HE LO,HTML_40_50,HTML_MESSAGE,RCVD_IN_DSBL,URIBL_JP_SURBL,URIBL_SBL scantime=0.9,si ze=2585,user=simscan,uid=0,required_score=6.0,rhost=localhost.pil.net,raddr=127. 0.0.1,rport=4599,mid=[EMAIL PROTECTED],bayes=0.930576978197 612,autolearn=no May 31 12:59:49 mail qmail: 1149094789.960550 new msg 289788 May 31 12:59:49 mail qmail: 1149094789.960823 info msg 289788: bytes 14074 from [EMAIL PROTECTED] qp 35721 uid 1003 May 31 12:59:49 mail spamc[35739]: connect(AF_INET) to spamd at 127.0.0.1 failed, retrying (#1 of 3): Connection refused May 31 12:59:49 mail spamc[35744]: connect(AF_INET) to spamd at 127.0.0.1 failed, retrying (#1 of 3): Connection refused May 31 12:59:50 mail qmail: 1149094790.217124 starting delivery 70430: ms (deliveries continued, without filtering) James Smallacombe PlantageNet, Inc. CEO and Janitor [EMAIL PROTECTED] http://3.am =
Re: 3.1.2 issue with UnixNFSSafe.pm ?
Can you get output of strace -f -o trace on the affected processes? (easy way: just stop spamd, then strace the spamd startup script.) --j. [EMAIL PROTECTED] writes: On Wed, 31 May 2006 [EMAIL PROTECTED] wrote: On Wed, 31 May 2006, Theo Van Dinter wrote: On Wed, May 31, 2006 at 11:12:03AM -0400, [EMAIL PROTECTED] wrote: May 31 07:53:52 mail spamd[59117]: Use of uninitialized value in subtraction (-) at /usr/local/lib/perl5/site_perl/5.6.1/Mail/SpamAssassin/Locker/UnixNFSSafe.pm line 102, GEN108 line 46. Hrm. That's not good. Seems like there's a race condition in there. :( 1) If you can open a bz ticket about that, we ought to prod the code and try to eliminate the race. Will do. 2) If the files are on local disk, switch to using the flock method instead. It's much better, imo. Wow, I didn't even know that was a config option (didn't see it in the docs). I just switched it and so far, it appears to have done the trick, although the condition was coming and going, so I'll keep an eye on it. It appeared to have fixed the race condition from what I could see...cpu was down, number of children, smtp connections, etc. However, about 12 minutes ago, the spamd parent died for the second time since the upgrade. Here is all I could find in the maillog: May 31 12:59:44 mail spamd[35583]: spamd: result: Y 12 - BAYES_80,FORGED_RCVD_HE LO,HTML_40_50,HTML_MESSAGE,RCVD_IN_DSBL,URIBL_JP_SURBL,URIBL_SBL scantime=0.9,si ze=2585,user=simscan,uid=0,required_score=6.0,rhost=localhost.pil.net,raddr=127. 0.0.1,rport=4599,mid=[EMAIL PROTECTED],bayes=0.930576978197 612,autolearn=no May 31 12:59:49 mail qmail: 1149094789.960550 new msg 289788 May 31 12:59:49 mail qmail: 1149094789.960823 info msg 289788: bytes 14074 from [EMAIL PROTECTED] qp 35721 uid 1003 May 31 12:59:49 mail spamc[35739]: connect(AF_INET) to spamd at 127.0.0.1 failed, retrying (#1 of 3): Connection refused May 31 12:59:49 mail spamc[35744]: connect(AF_INET) to spamd at 127.0.0.1 failed, retrying (#1 of 3): Connection refused May 31 12:59:50 mail qmail: 1149094790.217124 starting delivery 70430: ms (deliveries continued, without filtering) James Smallacombe PlantageNet, Inc. CEO and Janitor [EMAIL PROTECTED] http://3.am =
Re: 3.1.2 --lint -D warn: config: could not find userprefs file
Hi All, Problem resolved. The creation of a dir /root/.spamassassin with a file user_prefs in it has resolved the issue. After adding the above it now lint's clean. I'm not sure why it didn't get created at install - but at least now it's fixed. Many thanks to James Rallo for his correct suggestions. Kind regards Nigel On Wed, 31 May 2006 18:39:10 +0100, Nigel Frankcom [EMAIL PROTECTED] wrote: Hi, I've run in 3.1.2 on a test server before letting it loose live. When running --lint -D I get the following output: info: config: can not determine userstate dir warn: config: could not find userprefs file My SPAMDOPTIONS are set as: SPAMDOPTIONS=-d -c -m5 -H -i0.0.0.0 -A192.168.2. -q -x -s /var/log/spamd.log My Local.cf has: user_scores_dsn DBI:mysql:spamassassin:192.168.2.42 user_scores_sql_usernamemyuser user_scores_sql_passwordmypass allow_user_rules1 Does anyone have an idea what I might have misconfig'd? Apart from those two 'errors' everything else lint's clean. I appreciate the info: is not to be worried about, but the warn: is worrying me. Kind regards Nigel
Re: Re: 3.1.2 --lint -D warn: config: could not find userprefs file
On the plus side the manual fix is pretty simple and now it's in the lists it should make it a little easier for others to find a solution. I wouldn't beat yourself up too much, if it was perfect every time where would the entertainment value be? ;-) KR Nigel On Wed, 31 May 2006 17:16:56 -0400, Theo Van Dinter [EMAIL PROTECTED] wrote: On Wed, May 31, 2006 at 03:09:55PM -0600, Gary V wrote: looks like there is a patch, and will be fixed in 3.1.3 http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4932 Actually that patch doesn't fix the problem, assuming the user doesn't have a ~/.spamassassin directory already and they're just running sa-learn. I updated bug 4923 with some info. Ironically, all of this is being caused by a fix for bug 4827 which was spurred into existence by problems debugging problems because incorrect paths were being chosen for things. Unfortunately, my patch wasn't comprehensive enough it seems. :( Stupid software. G
Re: 3.1.2 spamd dies without warning
On Tue, May 30, 2006 at 10:31:27AM -0400, [EMAIL PROTECTED] wrote: BTW, can anybody tell me whether or not spamd re-reads the .pre files if you just give it a SIG HUP, or do you have to kill and restart it (I did that this time anyway)? A HUP should cause spamd to re-exec (basically killing itself off and restarting), reading in all the config files again. -- Randomly Generated Tagline: You have to stay in shape. My grandmother, she started walking 5 miles a day when she was 60. She's 97 today and we don't know where the hell she is. - Ellen DeGeneres pgpQfs7H8PzLN.pgp Description: PGP signature
RE: 3.1.2-Windows, exit codes broken?
I mean this: - bug 3754: if there's a problem opening a file via sa-learn or spamassassin, return an error exit value. It indicates there might have been changes to that area of the code which concerns your problem. Kai I understand, thanks for the clarification. I had some time over the weekend to install 3.1.2 on a second Windows box and had the same result, exit code always zero with -e option and messages determined by spamassassin to be spam. Oh well, back to 3.1.1 for me. The ability to script spamassassin as an MTA pickup event and process the message according to the result code is too good to give up :) Thanks, Craig
Re: 3.1.2-Windows, exit codes broken?
A reminder: if you don't bug that it won't get fixed. Kai -- Kai Schätzl, Berlin, Germany Get your web at Conactive Internet Services: http://www.conactive.com
Re: 3.1.2-Windows, exit codes broken?
Craig Emmitt wrote on Fri, 26 May 2006 11:01:53 -0500: 3.1.2 is not setting a non-zero exit code when a message is classified as spam (spamassassin.bat -e mailfilein mailfileout) Known issue/bug? There is something about exit codes in the changelog. Kai -- Kai Schätzl, Berlin, Germany Get your web at Conactive Internet Services: http://www.conactive.com
RE: 3.1.2-Windows, exit codes broken?
3.1.2 is not setting a non-zero exit code when a message is classified as spam (spamassassin.bat -e mailfilein mailfileout) Known issue/bug? There is something about exit codes in the changelog. I had checked the changelog and didn't see anything obviously relevant. Craig
Re: 3.1.2-Windows, exit codes broken?
Craig Emmitt wrote on Fri, 26 May 2006 12:02:57 -0500: I had checked the changelog and didn't see anything obviously relevant. I mean this: - bug 3754: if there's a problem opening a file via sa-learn or spamassassin, return an error exit value. It indicates there might have been changes to that area of the code which concerns your problem. Kai -- Kai Schätzl, Berlin, Germany Get your web at Conactive Internet Services: http://www.conactive.com
Re: 3.1.2?
On Tue, May 23, 2006 at 12:50:41PM -0400, [EMAIL PROTECTED] wrote: I was hoping to get it out this month, but I think it'll probably be next early month before it's all ready to go. ie: hopefully a week or two, depending on how much time people have to create/review patches, etc. Any word on this? Same motivations here... :-/ Crap, did I say early next month (May) ? Oops. Time flies and all of that. :( There are still 3 tickets in the 3.1.2 queue. 2 have patches and are waiting for some more reviews. 1 may be punted to 3.1.3, or else needs a little more work for a patch. I'll prod a little on the dev@ list and see if we can't get this done by next week. -- Randomly Generated Tagline: I don't get it. Who was this Ted Danson, and why would you pay $10,000 for his skeleton? -Leela pgpezJTWwYHvo.pgp Description: PGP signature
Re: 3.1.2?
On Thu, 27 Apr 2006, Theo Van Dinter wrote: On Wed, Apr 26, 2006 at 05:32:45PM -0400, Joe Flowers wrote: Any educated guesses on when 3.1.2 will be released? From a selfish point of view, I'm trying to kill several upgrades with one stone. I was hoping to get it out this month, but I think it'll probably be next early month before it's all ready to go. ie: hopefully a week or two, depending on how much time people have to create/review patches, etc. Any word on this? Same motivations here... :-/ James Smallacombe PlantageNet, Inc. CEO and Janitor [EMAIL PROTECTED] http://3.am =
Re: 3.1.2?
On Wed, Apr 26, 2006 at 05:32:45PM -0400, Joe Flowers wrote: Any educated guesses on when 3.1.2 will be released? I was hoping to get it out this month, but I think it'll probably be next early month before it's all ready to go. Any word on whether or not it includes a fix for bug #4590? http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4590 Thanks for your efforts - you guys rock. St-
Re: Re: 3.1.2?
Thanks Theo. That'll be good then. I saw your plea on the Dev list to get another version out this month, so I've been kinda holding off on the upgrades. I'll try to fend off the users until mid-May then. Thank you! Yes, as Steve said, you guys rock!!! Joe -Original Message- From: Theo Van Dinter [EMAIL PROTECTED] To: users@spamassassin.apache.org Date: Thu, 27 Apr 2006 14:05:25 -0400 Subject: Re: 3.1.2? On Wed, Apr 26, 2006 at 05:32:45PM -0400, Joe Flowers wrote: Any educated guesses on when 3.1.2 will be released? From a selfish point of view, I'm trying to kill several upgrades with one stone. I was hoping to get it out this month, but I think it'll probably be next early month before it's all ready to go. ie: hopefully a week or two, depending on how much time people have to create/review patches, etc. -- Randomly Generated Tagline: Where are all the great pot head writers? There aren't any. Because no one wants to read a book about the most delicious twinkie. - Dave Attell, Insomniac, New York City, 2001
