Re: SA 3.0 is eating up all my memory!!!
On Wed, 6 Oct 2004, Luis Hernán Otegui wrote: In my setup this is not an option, because I run SA as a milter, via spamass-milter. If every process has to die after the scan, it cannot pass the results of the scan to Sendmail (at least, this is what happened after I tried this option, so I kept on receiving messages like this: Oct 4 09:27:55 nahuel spamass-milter[14646]: Could not extract score from So, after all, I had to discard my precious Bayes databases, and got back to good old 2.64... How odd... I too use a spamass-milter, and this works fine for me. The spamd runs on a somewhat more beefy host. On Tue, 5 Oct 2004 15:09:50 -0500, Doug Block [EMAIL PROTECTED] wrote: I had this problem till I set the max per child option to = 1 This caused spamd to kill the process used to scan every msg once it's done. Not the best answer I know but it keeps it in check -- Jon Trulsonmailto:[EMAIL PROTECTED] ID: 1A9A2B09, FP: C23F328A721264E7 B6188192EC733962 PGP keys at http://radscan.com/~jon/PGPKeys.txt #include std/disclaimer.h I am Nomad. -Nomad
Re: SA 3.0 is eating up all my memory!!!
In my setup this is not an option, because I run SA as a milter, via spamass-milter. If every process has to die after the scan, it cannot pass the results of the scan to Sendmail (at least, this is what happened after I tried this option, so I kept on receiving messages like this: Oct 4 09:27:55 nahuel spamass-milter[14646]: Could not extract score from So, after all, I had to discard my precious Bayes databases, and got back to good old 2.64... On Tue, 5 Oct 2004 15:09:50 -0500, Doug Block [EMAIL PROTECTED] wrote: I had this problem till I set the max per child option to = 1 This caused spamd to kill the process used to scan every msg once it's done. Not the best answer I know but it keeps it in check -- - GNU-GPL: May The Source Be With You... -
RE: SA 3.0 is eating up all my memory!!!
On Fri, 1 Oct 2004, Morris Jones wrote: I found 3.0 pushing my machine into swapping as well this afternoon -- a first for me. I stopped and restarted my smtp server and spamd, and it's back to normal for now. I'm beginning to think I might be better off running spamassassin in unique processes instead of as a daemon. The load time was never terribly bad, and they certainly can't leak. See my response in a previous thread on this problem. For kicks, try --max-conn-per-child=1 to spamd see and see if your machine will last longer :) Mine did... -- Jon Trulsonmailto:[EMAIL PROTECTED] ID: 1A9A2B09, FP: C23F328A721264E7 B6188192EC733962 PGP keys at http://radscan.com/~jon/PGPKeys.txt #include std/disclaimer.h I am Nomad. -Nomad
Re: SA 3.0 is eating up all my memory!!!
On Tue, 5 Oct 2004, Jon Trulson wrote:
On Mon, 4 Oct 2004, Luis Hernan Otegui wrote:
Well, a weekend update:
Nothing has changed here. I removed EVERYTHING (except for local.cf)
from /etc/mail/spamassassin, and still it chews as much memory as it
could get. I limited the number of childs to five (removed the -m
switch in the startup script), and nothing changed. The only
improvement was that instead of 20 processes claiming all the
memory, there were only five trying to freeze my box... But the oldest
one still is a big memory grabber: It reached up to 133 MB, and NEVER
got any lower, it just keeps grabbing and grabbing memory... Seems
pretty much strange to me...
Same thing I saw, except in my case, it was 320MB. Once a child
had it, it never let it go until terminated (or hit the default 200
connection limit).
Is there a Perl equivalent to the Unix 'setrlimit' or 'ulimit'
function? (IE something to set the max data size that a process is
allowed to use).
Just set it to limit the child processes to something reasonable,
(say 50~100MB) and have them die if it is exceeded.
--
Dave Funk University of Iowa
dbfunk (at) engineering.uiowa.eduCollege of Engineering
319/335-5751 FAX: 319/384-0549 1256 Seamans Center
Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527
#include std_disclaimer.h
Better is not better, 'standard' is better. B{
Re: SA 3.0 is eating up all my memory!!!
On Tue, 5 Oct 2004, Jon Trulson wrote: On Mon, 4 Oct 2004, Luis Hernan Otegui wrote: Well, a weekend update: Nothing has changed here. I removed EVERYTHING (except for local.cf) from /etc/mail/spamassassin, and still it chews as much memory as it could get. I limited the number of childs to five (removed the -m switch in the startup script), and nothing changed. The only improvement was that instead of 20 processes claiming all the memory, there were only five trying to freeze my box... But the oldest one still is a big memory grabber: It reached up to 133 MB, and NEVER got any lower, it just keeps grabbing and grabbing memory... Seems pretty much strange to me... Same thing I saw, except in my case, it was 320MB. Once a child had it, it never let it go until terminated (or hit the default 200 connection limit). Is there a Perl equivalent to the Unix 'setrlimit' or 'ulimit' function? (IE something to set the max data size that a process is allowed to use). Just set it to limit the child processes to something reasonable, (say 50~100MB) and have them die if it is exceeded. Is there any reason why you couldn't just use the unix ulimit command in the script that launches the spamd daemon ? Are the spamd children threads or processes ? If they're just forked processes, shouldn't they inherit the ulimit values of the parent ? If one of them went over it's ulimit it would be killed. Whether the message being processed would then pass through unscanned or not, and whether the parent spamd would notice and respawn a replacement is another matter though... :) Regards, Simon
RE: SA 3.0 is eating up all my memory!!!
Is there a Perl equivalent to the Unix 'setrlimit' or 'ulimit' function? (IE something to set the max data size that a process is allowed to use). I use djb's softlimit and supervise my spamd process with daemontools. I softlimit spamd at 100MB just to prevent childs from running away with all the memory. It works well. d -- Dallas Engelken NMGI
RE: SA 3.0 is eating up all my memory!!!
I had this problem till I set the max per child option to = 1 This caused spamd to kill the process used to scan every msg once it's done. Not the best answer I know but it keeps it in check
Re: SA 3.0 is eating up all my memory!!!
Loren Wilton wrote: 80M doesn't strike me as unusual for spamd if you have any of the addon rulesets. [EMAIL PROTECTED]@#sputter...! Yes, that is too unusual unless you're using ALL the addon rulesets, including BigEvil, which, I hear, eats pets and small children when nobody's looking, and should be avoided. And also probably several non-SARE rulesets too.
RE: SA 3.0 is eating up all my memory!!!
Hence my comments on the OT thread earlier today about the BigEvil author going mad one day... :) -Original Message- From: snowjack [mailto:[EMAIL PROTECTED] Sent: Friday, October 01, 2004 4:20 PM To: users@spamassassin.apache.org Subject: Re: SA 3.0 is eating up all my memory!!! Loren Wilton wrote: 80M doesn't strike me as unusual for spamd if you have any of the addon rulesets. [EMAIL PROTECTED]@#sputter...! Yes, that is too unusual unless you're using ALL the addon rulesets, including BigEvil, which, I hear, eats pets and small children when nobody's looking, and should be avoided. And also probably several non-SARE rulesets too.
RE: SA 3.0 is eating up all my memory!!!
I found 3.0 pushing my machine into swapping as well this afternoon -- a first for me. I stopped and restarted my smtp server and spamd, and it's back to normal for now. I'm beginning to think I might be better off running spamassassin in unique processes instead of as a daemon. The load time was never terribly bad, and they certainly can't leak. Mojo -- Morris Jones * Monrovia, CA [EMAIL PROTECTED] http://www.whiteoaks.com
Re: SA 3.0 is eating up all my memory!!!
I have a similar situation, and I have removed all my rule sets. Here is the output from top top - 00:35:45 up 1 day, 14:45, 2 users, load average: 2.45, 2.40, 2.48 Tasks: 158 total, 3 running, 154 sleeping, 0 stopped, 1 zombie Cpu(s): 1.0% us, 1.6% sy, 97.4% ni, 0.0% id, 0.0% wa, 0.0% hi, 0.0% si Mem: 1036832k total, 800524k used, 236308k free, 141016k buffers Swap: 265060k total,66896k used, 198164k free, 110796k cached PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 7070 spam 23 0 26188 21m 5960 S 0.0 2.2 0:03.94 spamd 7073 spam 20 0 26188 21m 5960 S 0.0 2.2 0:00.02 spamd 7074 spam 20 0 26188 21m 5960 S 0.0 2.2 0:00.01 spamd 7075 spam 22 0 26188 21m 5960 S 0.0 2.2 0:00.01 spamd 7076 spam 23 0 26188 21m 5960 S 0.0 2.2 0:00.01 spamd 7077 spam 23 0 26188 21m 5960 S 0.0 2.2 0:00.02 spamd top - 08:37:17 up 1 day, 22:47, 2 users, load average: 2.45, 2.24, 2.15 Tasks: 146 total, 4 running, 141 sleeping, 0 stopped, 1 zombie Cpu(s): 19.3% us, 2.6% sy, 77.3% ni, 0.0% id, 0.0% wa, 0.0% hi, 0.8% si Mem: 1036832k total, 971724k used,65108k free, 148300k buffers Swap: 265060k total,72188k used, 192872k free, 102508k cached PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 7076 spam 18 0 35468 30m 5960 R 37.6 3.0 1:36.58 spamd 7070 spam 23 0 26188 21m 5960 S 0.0 2.2 0:03.94 spamd 7073 spam 18 0 31612 27m 5960 S 0.0 2.7 5:03.49 spamd 7074 spam 19 0 39732 35m 5960 S 0.0 3.5 1:40.51 spamd 7075 spam 19 0 35548 30m 5960 S 0.0 3.0 1:29.94 spamd 7077 spam 19 0 30204 25m 5960 S 0.0 2.6 1:15.38 spamd top - 10:25:33 up 2 days, 35 min, 2 users, load average: 2.89, 2.61, 2.55 Tasks: 162 total, 4 running, 157 sleeping, 0 stopped, 1 zombie Cpu(s): 20.1% us, 6.2% sy, 73.7% ni, 0.0% id, 0.0% wa, 0.0% hi, 0.0% si Mem: 1036832k total, 981864k used,54968k free, 144744k buffers Swap: 265060k total,81236k used, 183824k free, 114088k cached PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 7070 spam 23 0 26188 21m 5960 S 0.0 2.1 0:03.94 spamd 7073 spam 19 0 31612 27m 5960 S 0.0 2.7 5:47.55 spamd 7074 spam 19 0 39732 33m 5960 S 0.0 3.4 2:19.69 spamd 7075 spam 19 0 35548 30m 5960 S 0.0 3.0 2:09.12 spamd 7076 spam 18 0 38380 33m 5960 S 0.0 3.3 2:36.16 spamd 7077 spam 19 0 32556 28m 5960 S 0.0 2.8 1:49.98 spamd top - 13:39:35 up 2 days, 3:49, 1 user, load average: 2.01, 2.15, 2.18 Tasks: 155 total, 3 running, 151 sleeping, 0 stopped, 1 zombie Cpu(s): 1.0% us, 0.7% sy, 98.2% ni, 0.0% id, 0.0% wa, 0.0% hi, 0.2% si Mem: 1036832k total, 987092k used,49740k free, 142316k buffers Swap: 265060k total,81512k used, 183548k free,97608k cached PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 7070 spam 23 0 26188 21m 5960 S 0.0 2.1 0:03.94 spamd 7073 spam 19 0 32424 27m 5960 S 0.0 2.8 6:34.85 spamd 7074 spam 23 0 39732 34m 5960 S 0.0 3.4 3:11.99 spamd 7075 spam 22 0 35548 30m 5960 S 0.0 3.1 3:00.87 spamd 7076 spam 21 0 38380 33m 5960 S 0.0 3.3 3:16.20 spamd 7077 spam 18 0 32556 28m 5960 S 0.0 2.8 2:44.93 spamd -- Jim Gifford [EMAIL PROTECTED]
Re: SA 3.0 is eating up all my memory!!!
Well, I can't solve *all* of your problems, but I can help with one of them: Luis Hernán Otegui wrote: I'm having 51 issues when I run spamassassin --lint. I've clean the obsolete options from local.cf, but the rest of the issues come (apparently) from badly defined rules in the local.cf file, but I'm still getting this: warning: description for VIRUS_WARNING251 is over 50 chars etc. You've got a third-party ruleset installed that isn't compatible with SA 3.0. Look in /etc/mail/spamassassin (or wherever your config files are) for the file and either delete it or find the site you downloaded it from and see if there is a new one. I have an antivirus running as a milter, how do I disable the virus tests in SA, and also, why do I have this warnings? Actually, I suspect those are tests for notices *about* viruses. Many people have their antivirus programs set to send a notice to the sender of a virus, but they forget that the sender is almost always fake. So in addition to actual copies of a virus, some people get lots of these misdirected notices during an outbreak. I know there's at least one ruleset designed to filter them out, and I suspect that's what you have installed. -- Kelson Vibber SpeedGate Communications www.speed.net
Re: SA 3.0 is eating up all my memory!!!
At 07:29 PM 9/30/2004, Luis Hernán Otegui wrote: warning: rule 'VIRUS_WARNING_MYDOOM_BNCE' is over 22 chars lint: 51 issues detected. please rerun with debug enabled for more information. I have an antivirus running as a milter, how do I disable the virus tests in SA, and also, why do I have this warnings? Those aren't virus tests, those are tests for Someone sent a virus using your address and my virus scanner is stupidly telling you about it type emails. They are also not a part of spamassassin's default rules, they are an add on called bogus-virus-warnings.cf . If you don't want the ruleset, remove it. Otherwise update it from it's source: http://www.timj.co.uk/linux/bogus-virus-warnings.cf
Re: SA 3.0 is eating up all my memory!!!
ok, the virus warning issues have been solved, but NOT the fact that I have 22 copies of spamd running at the same time, even when I´ve limited the number of max children of Sendmail to 20, and each copy of spamd weights 21 MB! How can I limit the amount of memory spamd is chewing? On Thu, 30 Sep 2004 21:05:29 -0400, Matt Kettler [EMAIL PROTECTED] wrote: At 07:29 PM 9/30/2004, Luis Hernán Otegui wrote: warning: rule 'VIRUS_WARNING_MYDOOM_BNCE' is over 22 chars lint: 51 issues detected. please rerun with debug enabled for more information. I have an antivirus running as a milter, how do I disable the virus tests in SA, and also, why do I have this warnings? Those aren't virus tests, those are tests for Someone sent a virus using your address and my virus scanner is stupidly telling you about it type emails. They are also not a part of spamassassin's default rules, they are an add on called bogus-virus-warnings.cf . If you don't want the ruleset, remove it. Otherwise update it from it's source: http://www.timj.co.uk/linux/bogus-virus-warnings.cf -- - GNU-GPL: May The Source Be With You... -
Re: SA 3.0 is eating up all my memory!!!
Just a little update, this is what's going on over the server: 11:59am up 16:12, 1 user, load average: 10,51, 5,30, 2,47 151 processes: 144 sleeping, 6 running, 1 zombie, 0 stopped CPU states: 6,5% user, 2,1% system, 0,0% nice, 91,3% idle Mem: 449484K av, 76K used,5008K free, 0K shrd, 12K buff Swap: 265032K av, 263440K used,1592K free 24896K cached PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND 9295 spamd 15 0 90604 79M 4788 S 0,0 18,1 0:09 spamd 9293 spamd 15 0 90228 78M 4908 S 0,0 17,9 0:09 spamd 9289 spamd 18 0 90248 66M 3924 S 0,0 15,2 0:09 spamd 9298 spamd 15 0 22808 14M 4764 S 0,0 3,2 0:00 spamd 9296 spamd 15 0 22904 14M 4744 S 0,0 3,2 0:00 spamd 9291 spamd 15 0 22896 14M 4732 S 0,0 3,2 0:00 spamd 9297 spamd 15 0 23008 14M 4740 S 0,0 3,2 0:00 spamd 9292 spamd 16 0 22480 13M 4856 S 0,0 3,1 0:00 spamd 9294 spamd 15 0 22460 13M 4800 S 0,0 3,0 0:00 spamd 9299 spamd 15 0 22440 13M 4908 S 0,0 3,0 0:00 spamd 9302 spamd 15 0 22808 12M 4636 D 0,0 2,9 0:00 spamd 9300 spamd 15 0 22524 12M 4720 D 0,7 2,8 0:00 spamd 9301 spamd 15 0 22460 12M 4652 D 0,1 2,7 0:00 spamd 9290 spamd 15 0 22704 9628 3760 S 0,0 2,1 0:00 spamd 7844 apache15 0 7708 7204 6336 S 0,0 1,6 0:04 httpd 7437 apache15 0 7488 7008 4916 S 0,1 1,5 0:04 httpd 7564 apache15 0 7520 7008 4840 S 0,0 1,5 0:04 httpd I mean, 80 MB for the master spamd porcess? How do I limit this? On Fri, 1 Oct 2004 10:37:05 -0300, Luis Hernán Otegui [EMAIL PROTECTED] wrote: ok, the virus warning issues have been solved, but NOT the fact that I have 22 copies of spamd running at the same time, even when I´ve limited the number of max children of Sendmail to 20, and each copy of spamd weights 21 MB! How can I limit the amount of memory spamd is chewing? On Thu, 30 Sep 2004 21:05:29 -0400, Matt Kettler [EMAIL PROTECTED] wrote: At 07:29 PM 9/30/2004, Luis Hernán Otegui wrote: warning: rule 'VIRUS_WARNING_MYDOOM_BNCE' is over 22 chars lint: 51 issues detected. please rerun with debug enabled for more information. I have an antivirus running as a milter, how do I disable the virus tests in SA, and also, why do I have this warnings? Those aren't virus tests, those are tests for Someone sent a virus using your address and my virus scanner is stupidly telling you about it type emails. They are also not a part of spamassassin's default rules, they are an add on called bogus-virus-warnings.cf . If you don't want the ruleset, remove it. Otherwise update it from it's source: http://www.timj.co.uk/linux/bogus-virus-warnings.cf -- - GNU-GPL: May The Source Be With You... - -- - GNU-GPL: May The Source Be With You... -
RE: SA 3.0 is eating up all my memory!!!
Something is seriously wrong with your setup! Move all cf files except local.cf out of the /etc/mail/spamassassin dir and restart spamd. What does it read then? There is no way spamd should be that large!! --Chris -Original Message- From: Luis Hernán Otegui [mailto:[EMAIL PROTECTED] Sent: Friday, October 01, 2004 11:00 AM To: Matt Kettler Cc: users@spamassassin.apache.org Subject: Re: SA 3.0 is eating up all my memory!!! Just a little update, this is what's going on over the server: 11:59am up 16:12, 1 user, load average: 10,51, 5,30, 2,47 151 processes: 144 sleeping, 6 running, 1 zombie, 0 stopped CPU states: 6,5% user, 2,1% system, 0,0% nice, 91,3% idle Mem: 449484K av, 76K used,5008K free, 0K shrd, 12K buff Swap: 265032K av, 263440K used,1592K free 24896K cached PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND 9295 spamd 15 0 90604 79M 4788 S 0,0 18,1 0:09 spamd 9293 spamd 15 0 90228 78M 4908 S 0,0 17,9 0:09 spamd 9289 spamd 18 0 90248 66M 3924 S 0,0 15,2 0:09 spamd 9298 spamd 15 0 22808 14M 4764 S 0,0 3,2 0:00 spamd 9296 spamd 15 0 22904 14M 4744 S 0,0 3,2 0:00 spamd 9291 spamd 15 0 22896 14M 4732 S 0,0 3,2 0:00 spamd 9297 spamd 15 0 23008 14M 4740 S 0,0 3,2 0:00 spamd 9292 spamd 16 0 22480 13M 4856 S 0,0 3,1 0:00 spamd 9294 spamd 15 0 22460 13M 4800 S 0,0 3,0 0:00 spamd 9299 spamd 15 0 22440 13M 4908 S 0,0 3,0 0:00 spamd 9302 spamd 15 0 22808 12M 4636 D 0,0 2,9 0:00 spamd 9300 spamd 15 0 22524 12M 4720 D 0,7 2,8 0:00 spamd 9301 spamd 15 0 22460 12M 4652 D 0,1 2,7 0:00 spamd 9290 spamd 15 0 22704 9628 3760 S 0,0 2,1 0:00 spamd 7844 apache15 0 7708 7204 6336 S 0,0 1,6 0:04 httpd 7437 apache15 0 7488 7008 4916 S 0,1 1,5 0:04 httpd 7564 apache15 0 7520 7008 4840 S 0,0 1,5 0:04 httpd I mean, 80 MB for the master spamd porcess? How do I limit this? On Fri, 1 Oct 2004 10:37:05 -0300, Luis Hernán Otegui [EMAIL PROTECTED] wrote: ok, the virus warning issues have been solved, but NOT the fact that I have 22 copies of spamd running at the same time, even when I´ve limited the number of max children of Sendmail to 20, and each copy of spamd weights 21 MB! How can I limit the amount of memory spamd is chewing? On Thu, 30 Sep 2004 21:05:29 -0400, Matt Kettler [EMAIL PROTECTED] wrote: At 07:29 PM 9/30/2004, Luis Hernán Otegui wrote: warning: rule 'VIRUS_WARNING_MYDOOM_BNCE' is over 22 chars lint: 51 issues detected. please rerun with debug enabled for more information. I have an antivirus running as a milter, how do I disable the virus tests in SA, and also, why do I have this warnings? Those aren't virus tests, those are tests for Someone sent a virus using your address and my virus scanner is stupidly telling you about it type emails. They are also not a part of spamassassin's default rules, they are an add on called bogus-virus-warnings.cf . If you don't want the ruleset, remove it. Otherwise update it from it's source: http://www.timj.co.uk/linux/bogus-virus-warnings.cf -- - GNU-GPL: May The Source Be With You... - -- - GNU-GPL: May The Source Be With You... -
Re: SA 3.0 is eating up all my memory!!!
ok, all I had in the /etc/mail/spamassassin dir was my local.cf and the init.pre. I've cleaned the local.cf according to the list of obsolete rules and directives, and still it is eating as much memory as it can... No idea why, the setup is the same as for another 4 production servers, it started all when I upgraded to 3.0... On Fri, 1 Oct 2004 11:05:40 -0400, Chris Santerre [EMAIL PROTECTED] wrote: Something is seriously wrong with your setup! Move all cf files except local.cf out of the /etc/mail/spamassassin dir and restart spamd. What does it read then? There is no way spamd should be that large!! --Chris -Original Message- From: Luis Hernán Otegui [mailto:[EMAIL PROTECTED] Sent: Friday, October 01, 2004 11:00 AM To: Matt Kettler Cc: users@spamassassin.apache.org Subject: Re: SA 3.0 is eating up all my memory!!! Just a little update, this is what's going on over the server: 11:59am up 16:12, 1 user, load average: 10,51, 5,30, 2,47 151 processes: 144 sleeping, 6 running, 1 zombie, 0 stopped CPU states: 6,5% user, 2,1% system, 0,0% nice, 91,3% idle Mem: 449484K av, 76K used,5008K free, 0K shrd, 12K buff Swap: 265032K av, 263440K used,1592K free 24896K cached PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND 9295 spamd 15 0 90604 79M 4788 S 0,0 18,1 0:09 spamd 9293 spamd 15 0 90228 78M 4908 S 0,0 17,9 0:09 spamd 9289 spamd 18 0 90248 66M 3924 S 0,0 15,2 0:09 spamd 9298 spamd 15 0 22808 14M 4764 S 0,0 3,2 0:00 spamd 9296 spamd 15 0 22904 14M 4744 S 0,0 3,2 0:00 spamd 9291 spamd 15 0 22896 14M 4732 S 0,0 3,2 0:00 spamd 9297 spamd 15 0 23008 14M 4740 S 0,0 3,2 0:00 spamd 9292 spamd 16 0 22480 13M 4856 S 0,0 3,1 0:00 spamd 9294 spamd 15 0 22460 13M 4800 S 0,0 3,0 0:00 spamd 9299 spamd 15 0 22440 13M 4908 S 0,0 3,0 0:00 spamd 9302 spamd 15 0 22808 12M 4636 D 0,0 2,9 0:00 spamd 9300 spamd 15 0 22524 12M 4720 D 0,7 2,8 0:00 spamd 9301 spamd 15 0 22460 12M 4652 D 0,1 2,7 0:00 spamd 9290 spamd 15 0 22704 9628 3760 S 0,0 2,1 0:00 spamd 7844 apache15 0 7708 7204 6336 S 0,0 1,6 0:04 httpd 7437 apache15 0 7488 7008 4916 S 0,1 1,5 0:04 httpd 7564 apache15 0 7520 7008 4840 S 0,0 1,5 0:04 httpd I mean, 80 MB for the master spamd porcess? How do I limit this? On Fri, 1 Oct 2004 10:37:05 -0300, Luis Hernán Otegui [EMAIL PROTECTED] wrote: ok, the virus warning issues have been solved, but NOT the fact that I have 22 copies of spamd running at the same time, even when I´ve limited the number of max children of Sendmail to 20, and each copy of spamd weights 21 MB! How can I limit the amount of memory spamd is chewing? On Thu, 30 Sep 2004 21:05:29 -0400, Matt Kettler [EMAIL PROTECTED] wrote: At 07:29 PM 9/30/2004, Luis Hernán Otegui wrote: warning: rule 'VIRUS_WARNING_MYDOOM_BNCE' is over 22 chars lint: 51 issues detected. please rerun with debug enabled for more information. I have an antivirus running as a milter, how do I disable the virus tests in SA, and also, why do I have this warnings? Those aren't virus tests, those are tests for Someone sent a virus using your address and my virus scanner is stupidly telling you about it type emails. They are also not a part of spamassassin's default rules, they are an add on called bogus-virus-warnings.cf . If you don't want the ruleset, remove it. Otherwise update it from it's source: http://www.timj.co.uk/linux/bogus-virus-warnings.cf -- - GNU-GPL: May The Source Be With You... - -- - GNU-GPL: May The Source Be With You... - -- - GNU-GPL: May The Source Be With You... -
RE: SA 3.0 is eating up all my memory!!!
Can you post your local.cf file? Maybe one of the rules has a typo that is sending it to parts unknown. We've seen something like that before. Are you running bayes and awl? --Chris -Original Message- From: Luis Hernán Otegui [mailto:[EMAIL PROTECTED] Sent: Friday, October 01, 2004 11:52 AM To: Chris Santerre Cc: Matt Kettler; users@spamassassin.apache.org Subject: Re: SA 3.0 is eating up all my memory!!! ok, all I had in the /etc/mail/spamassassin dir was my local.cf and the init.pre. I've cleaned the local.cf according to the list of obsolete rules and directives, and still it is eating as much memory as it can... No idea why, the setup is the same as for another 4 production servers, it started all when I upgraded to 3.0... On Fri, 1 Oct 2004 11:05:40 -0400, Chris Santerre [EMAIL PROTECTED] wrote: Something is seriously wrong with your setup! Move all cf files except local.cf out of the /etc/mail/spamassassin dir and restart spamd. What does it read then? There is no way spamd should be that large!! --Chris -Original Message- From: Luis Hernán Otegui [mailto:[EMAIL PROTECTED] Sent: Friday, October 01, 2004 11:00 AM To: Matt Kettler Cc: users@spamassassin.apache.org Subject: Re: SA 3.0 is eating up all my memory!!! Just a little update, this is what's going on over the server: 11:59am up 16:12, 1 user, load average: 10,51, 5,30, 2,47 151 processes: 144 sleeping, 6 running, 1 zombie, 0 stopped CPU states: 6,5% user, 2,1% system, 0,0% nice, 91,3% idle Mem: 449484K av, 76K used,5008K free, 0K shrd, 12K buff Swap: 265032K av, 263440K used,1592K free 24896K cached PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND 9295 spamd 15 0 90604 79M 4788 S 0,0 18,1 0:09 spamd 9293 spamd 15 0 90228 78M 4908 S 0,0 17,9 0:09 spamd 9289 spamd 18 0 90248 66M 3924 S 0,0 15,2 0:09 spamd 9298 spamd 15 0 22808 14M 4764 S 0,0 3,2 0:00 spamd 9296 spamd 15 0 22904 14M 4744 S 0,0 3,2 0:00 spamd 9291 spamd 15 0 22896 14M 4732 S 0,0 3,2 0:00 spamd 9297 spamd 15 0 23008 14M 4740 S 0,0 3,2 0:00 spamd 9292 spamd 16 0 22480 13M 4856 S 0,0 3,1 0:00 spamd 9294 spamd 15 0 22460 13M 4800 S 0,0 3,0 0:00 spamd 9299 spamd 15 0 22440 13M 4908 S 0,0 3,0 0:00 spamd 9302 spamd 15 0 22808 12M 4636 D 0,0 2,9 0:00 spamd 9300 spamd 15 0 22524 12M 4720 D 0,7 2,8 0:00 spamd 9301 spamd 15 0 22460 12M 4652 D 0,1 2,7 0:00 spamd 9290 spamd 15 0 22704 9628 3760 S 0,0 2,1 0:00 spamd 7844 apache15 0 7708 7204 6336 S 0,0 1,6 0:04 httpd 7437 apache15 0 7488 7008 4916 S 0,1 1,5 0:04 httpd 7564 apache15 0 7520 7008 4840 S 0,0 1,5 0:04 httpd I mean, 80 MB for the master spamd porcess? How do I limit this? On Fri, 1 Oct 2004 10:37:05 -0300, Luis Hernán Otegui [EMAIL PROTECTED] wrote: ok, the virus warning issues have been solved, but NOT the fact that I have 22 copies of spamd running at the same time, even when I´ve limited the number of max children of Sendmail to 20, and each copy of spamd weights 21 MB! How can I limit the amount of memory spamd is chewing? On Thu, 30 Sep 2004 21:05:29 -0400, Matt Kettler [EMAIL PROTECTED] wrote: At 07:29 PM 9/30/2004, Luis Hernán Otegui wrote: warning: rule 'VIRUS_WARNING_MYDOOM_BNCE' is over 22 chars lint: 51 issues detected. please rerun with debug enabled for more information. I have an antivirus running as a milter, how do I disable the virus tests in SA, and also, why do I have this warnings? Those aren't virus tests, those are tests for Someone sent a virus using your address and my virus scanner is stupidly telling you about it type emails. They are also not a part of spamassassin's default rules, they are an add on called bogus-virus-warnings.cf . If you don't want the ruleset, remove it. Otherwise update it from it's source: http://www.timj.co.uk/linux/bogus-virus-warnings.cf -- - GNU-GPL: May The Source Be With You... - -- - GNU-GPL: May The Source Be With You... - -- - GNU-GPL: May The Source Be With You... -
Re: SA 3.0 is eating up all my memory!!!
Yes, I'm running bayes and AWL, but it has been running all the way from the very first releases which supported Bayes. Anyway, here's my local.cf (trimmed) it also has a large number of whitelist entries, since many many mails from yahoo get tagged as spam because they send the email ususlly three hours in the future, and because many suppliers of the Faculty send spammy-like mails, but I've cut them out, and I'm only sending the (I think) relevant part. OK, so here it is: #Users that bypass filtering use_auto_whitelist 1 lock_method flock skip_rbl_checks 1 ok_locales en es gr allow_user_rules 0 dns_available no trusted_networks 163.10.18.0/32 trusted_networks 163.10.7.0/32 #timelog (hace mucha basura, no dejar!!) #timelog_path/var/spool/spamassassin #razor use_razor2 0 #razor_config /etc/razor/razor-agent.conf #razor_timeout 5 #dcc use_dcc 0 #dcc_path /usr/local/bin/dccproc #dcc_add_header 1 #dcc_timeout5 auto_whitelist_path /var/spool/spamassassin/auto-whitelist bayes_path /var/spool/spamassassin/bayes auto_whitelist_file_mode 0777 required_hits 5 use_pyzor 0 # Custom Rules header NOTEBOOKS Subject =~ /[Nn]otebooks/i describeNOTEBOOKS Spam acerca de venta de notebooks score NOTEBOOKS 10.0 header MANUALES Subject =~ /[Mm]anuales/i describeMANUALES Spam acerca de venta de manuales score MANUALES 6.0 body SOLTERO/desuscribiremail2/i describeSOLTERO Spam acerca de dating sites score SOLTERO 6.0 bodyATANASOF/Atanasof/i describeATANASOFSpam de quien no debería hacerlo score ATANASOF17.0 #prueba de una regla para cachar el tipo nuevo de spam: header DATING_TRUCHO Subject =~ /re\:\[\d\]/i describeDATING_TRUCHO Sitios de citas en ingles score DATING_TRUCHO 7.0 Additionally, I have to add that I still have one warning when I run spamassassin --lint, and it seems to come from the standard ruleset: warning: description for EXCUSE_ES_03 is over 50 chars On Fri, 1 Oct 2004 12:14:59 -0400, Chris Santerre [EMAIL PROTECTED] wrote: Can you post your local.cf file? Maybe one of the rules has a typo that is sending it to parts unknown. We've seen something like that before. Are you running bayes and awl? --Chris -Original Message- From: Luis Hernán Otegui [mailto:[EMAIL PROTECTED] Sent: Friday, October 01, 2004 11:52 AM To: Chris Santerre Cc: Matt Kettler; users@spamassassin.apache.org Subject: Re: SA 3.0 is eating up all my memory!!! ok, all I had in the /etc/mail/spamassassin dir was my local.cf and the init.pre. I've cleaned the local.cf according to the list of obsolete rules and directives, and still it is eating as much memory as it can... No idea why, the setup is the same as for another 4 production servers, it started all when I upgraded to 3.0... On Fri, 1 Oct 2004 11:05:40 -0400, Chris Santerre [EMAIL PROTECTED] wrote: Something is seriously wrong with your setup! Move all cf files except local.cf out of the /etc/mail/spamassassin dir and restart spamd. What does it read then? There is no way spamd should be that large!! --Chris -Original Message- From: Luis Hernán Otegui [mailto:[EMAIL PROTECTED] Sent: Friday, October 01, 2004 11:00 AM To: Matt Kettler Cc: users@spamassassin.apache.org Subject: Re: SA 3.0 is eating up all my memory!!! Just a little update, this is what's going on over the server: 11:59am up 16:12, 1 user, load average: 10,51, 5,30, 2,47 151 processes: 144 sleeping, 6 running, 1 zombie, 0 stopped CPU states: 6,5% user, 2,1% system, 0,0% nice, 91,3% idle Mem: 449484K av, 76K used,5008K free, 0K shrd, 12K buff Swap: 265032K av, 263440K used,1592K free 24896K cached PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND 9295 spamd 15 0 90604 79M 4788 S 0,0 18,1 0:09 spamd 9293 spamd 15 0 90228 78M 4908 S 0,0 17,9 0:09 spamd 9289 spamd 18 0 90248 66M 3924 S 0,0 15,2 0:09 spamd 9298 spamd 15 0 22808 14M 4764 S 0,0 3,2 0:00 spamd 9296 spamd 15 0 22904 14M 4744 S 0,0 3,2 0:00 spamd 9291 spamd 15 0 22896 14M 4732 S 0,0 3,2 0:00 spamd 9297 spamd 15 0 23008 14M 4740 S 0,0 3,2 0:00 spamd 9292 spamd 16 0 22480 13M 4856 S 0,0 3,1 0:00 spamd 9294 spamd 15 0 22460 13M 4800 S 0,0 3,0 0:00 spamd 9299 spamd 15 0 22440 13M 4908 S 0,0 3,0 0:00 spamd 9302 spamd 15 0 22808 12M 4636 D 0,0 2,9 0:00 spamd 9300 spamd 15 0 22524 12M 4720 D 0,7 2,8 0:00 spamd 9301 spamd 15 0 22460 12M 4652 D 0,1 2,7 0:00 spamd 9290 spamd 15 0 22704 9628 3760 S 0,0 2,1 0:00 spamd 7844 apache15 0 7708 7204 6336
Re: SA 3.0 is eating up all my memory!!!
Luis Hernán Otegui said: ok, the virus warning issues have been solved, but NOT the fact that I have 22 copies of spamd running at the same time, even when I´ve limited the number of max children of Sendmail to 20, and each copy of spamd weights 21 MB! How can I limit the amount of memory spamd is chewing? Teh reason they limit hte spamd processes by default to 5 in spamassassin 3.0 is because, running more won't explicitly speed things up. All the spamd processes will be competing for processor time, so running more does not make things run N times faster. This is a silly answer, but why not just run less sendmail and spamd instances? -- Luke Computer Science System Administrator Security Administrator,College of Engineering Montana State University-Bozeman,Montana
Re: SA 3.0 is eating up all my memory!!!
At 12:23 PM 10/1/2004, Luis Hernán Otegui wrote: Additionally, I have to add that I still have one warning when I run spamassassin --lint, and it seems to come from the standard ruleset: warning: description for EXCUSE_ES_03 is over 50 chars Ick. Bad form for a final release.. 25_body_tests_es.cf:lang es describe EXCUSE_ES_03 Someone requested an spammer to spam you in Spanish (that's 51 characters of description, one over the limit) And on top of that, it's not grammatically correct. :) Suggested short-term fix: drop the 'n' so it makes grammatical sense and becomes short enough to not cause a warning. I'd also suggest filing a bugzilla on it.
Re: SA 3.0 is eating up all my memory!!!
BigEvil what?
On Fri, 1 Oct 2004 13:21:47 -0700, jdow [EMAIL PROTECTED] wrote:
BigEvil.
{^_-}
- Original Message -
From: Chris Santerre [EMAIL PROTECTED]
Something is seriously wrong with your setup! Move all cf files except
local.cf out of the
/etc/mail/spamassassin dir and restart spamd. What does it read then?
There is no way spamd should be that large!!
--Chris
-Original Message-
From: Luis Hernán Otegui [mailto:[EMAIL PROTECTED]
Just a little update, this is what's going on over the server:
11:59am up 16:12, 1 user, load average: 10,51, 5,30, 2,47
151 processes: 144 sleeping, 6 running, 1 zombie, 0 stopped
CPU states: 6,5% user, 2,1% system, 0,0% nice, 91,3% idle
Mem: 449484K av, 76K used,5008K free, 0K
shrd, 12K buff
Swap: 265032K av, 263440K used,1592K free
24896K cached
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
9295 spamd 15 0 90604 79M 4788 S 0,0 18,1 0:09 spamd
9293 spamd 15 0 90228 78M 4908 S 0,0 17,9 0:09 spamd
9289 spamd 18 0 90248 66M 3924 S 0,0 15,2 0:09 spamd
9298 spamd 15 0 22808 14M 4764 S 0,0 3,2 0:00 spamd
9296 spamd 15 0 22904 14M 4744 S 0,0 3,2 0:00 spamd
9291 spamd 15 0 22896 14M 4732 S 0,0 3,2 0:00 spamd
9297 spamd 15 0 23008 14M 4740 S 0,0 3,2 0:00 spamd
9292 spamd 16 0 22480 13M 4856 S 0,0 3,1 0:00 spamd
9294 spamd 15 0 22460 13M 4800 S 0,0 3,0 0:00 spamd
9299 spamd 15 0 22440 13M 4908 S 0,0 3,0 0:00 spamd
9302 spamd 15 0 22808 12M 4636 D 0,0 2,9 0:00 spamd
9300 spamd 15 0 22524 12M 4720 D 0,7 2,8 0:00 spamd
9301 spamd 15 0 22460 12M 4652 D 0,1 2,7 0:00 spamd
9290 spamd 15 0 22704 9628 3760 S 0,0 2,1 0:00 spamd
7844 apache15 0 7708 7204 6336 S 0,0 1,6 0:04 httpd
7437 apache15 0 7488 7008 4916 S 0,1 1,5 0:04 httpd
7564 apache15 0 7520 7008 4840 S 0,0 1,5 0:04 httpd
I mean, 80 MB for the master spamd porcess? How do I limit this?
On Fri, 1 Oct 2004 10:37:05 -0300, Luis Hernán Otegui
[EMAIL PROTECTED] wrote:
ok, the virus warning issues have been solved, but NOT the
fact that I
have 22 copies of spamd running at the same time, even when I´ve
limited the number of max children of Sendmail to 20, and
each copy of
spamd weights 21 MB! How can I limit the amount of memory spamd is
chewing?
On Thu, 30 Sep 2004 21:05:29 -0400, Matt Kettler
[EMAIL PROTECTED] wrote:
At 07:29 PM 9/30/2004, Luis Hernán Otegui wrote:
warning: rule 'VIRUS_WARNING_MYDOOM_BNCE' is over 22 chars
lint: 51 issues detected. please rerun with debug
enabled for more
information.
I have an antivirus running as a milter, how do I disable
the virus
tests in SA, and also, why do I have this warnings?
Those aren't virus tests, those are tests for Someone
sent a virus using
your address and my virus scanner is stupidly telling you
about it type
emails.
They are also not a part of spamassassin's default rules,
they are an add
on called bogus-virus-warnings.cf .
If you don't want the ruleset, remove it. Otherwise update
it from it's source:
http://www.timj.co.uk/linux/bogus-virus-warnings.cf
--
-
GNU-GPL: May The Source Be With You...
-
Re: SA 3.0 is eating up all my memory!!!
Just a little update, this is what's going on over the server: 11:59am up 16:12, 1 user, load average: 10,51, 5,30, 2,47 151 processes: 144 sleeping, 6 running, 1 zombie, 0 stopped CPU states: 6,5% user, 2,1% system, 0,0% nice, 91,3% idle Mem: 449484K av, 76K used,5008K free, 0K shrd, 12K buff Swap: 265032K av, 263440K used,1592K free 24896K cached PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND 9295 spamd 15 0 90604 79M 4788 S 0,0 18,1 0:09 spamd 9293 spamd 15 0 90228 78M 4908 S 0,0 17,9 0:09 spamd 9289 spamd 18 0 90248 66M 3924 S 0,0 15,2 0:09 spamd 9298 spamd 15 0 22808 14M 4764 S 0,0 3,2 0:00 spamd 9296 spamd 15 0 22904 14M 4744 S 0,0 3,2 0:00 spamd 9291 spamd 15 0 22896 14M 4732 S 0,0 3,2 0:00 spamd 9297 spamd 15 0 23008 14M 4740 S 0,0 3,2 0:00 spamd 9292 spamd 16 0 22480 13M 4856 S 0,0 3,1 0:00 spamd 9294 spamd 15 0 22460 13M 4800 S 0,0 3,0 0:00 spamd 9299 spamd 15 0 22440 13M 4908 S 0,0 3,0 0:00 spamd 9302 spamd 15 0 22808 12M 4636 D 0,0 2,9 0:00 spamd 9300 spamd 15 0 22524 12M 4720 D 0,7 2,8 0:00 spamd 9301 spamd 15 0 22460 12M 4652 D 0,1 2,7 0:00 spamd 9290 spamd 15 0 22704 9628 3760 S 0,0 2,1 0:00 spamd 7844 apache15 0 7708 7204 6336 S 0,0 1,6 0:04 httpd 7437 apache15 0 7488 7008 4916 S 0,1 1,5 0:04 httpd 7564 apache15 0 7520 7008 4840 S 0,0 1,5 0:04 httpd I mean, 80 MB for the master spamd porcess? How do I limit this? Well, the oldest (I don't know about 'master') spamd there is 9289, which is only using 66M. And to all intents, you only have 3 spamd processes doing anything, the rest are just sitting there. I'm not sure what is creating 14 of the beasts, but it looks like you could probably get away with 6 or so just fine at that workload. That would save you 80M or so. 80M doesn't strike me as unusual for spamd if you have any of the addon rulesets. So I'd suggest you don't limit the 80M if you want to catch spam. Instead you work with it and perhaps limit the number of 80M instances. One thing I notice is that you seem to be near the swap limit as well as memory limit. I think you would be a lot better off if you weren't completely out of swap space. Try taking swap up to some larger size, like maybe at least the size of memory, rather than half the size of memory. Loren
