Re: spamd process using to much cpu (again)
Michael Parker wrote: On Thu, Dec 02, 2004 at 04:25:46PM -0300, Matias Lopez Bergero wrote: From time to time, some spamd process sticks on top of the top listing with an ~90% CPU utilization, like this: 27639 mselig39 19 30104 29M 2472 R N 105.2 1.9 60:04 0 spamd There is a way to prevent this? Possibly. Depending on your setup you could turn off bayes_auto_expire and do the expiration manually (ie sa-learn --force-expire), at a time that you control. Unfortunately, unless you are running a sitewide bayes config, this may not work well with a lot of individual bayes dbs. I have disabled the bayes_auto_expire from spamd conf, and run the sa-learn --force-expire but I still get some spamd process that uses many CPU resources for a very long time(until I kill them). The only thing that I found curious, is that I always see the same user running this spamd processes with high CPU usage. Normally the was only a single process of spamd killing my CPU, but now there are many. 3090 mselig25 0 29112 28M 2564 R49.6 1.8 3:46 1 spamd 28737 mselig25 0 30604 29M 2564 R48.1 1.9 42:35 0 spamd 28103 mselig25 0 32168 31M 2564 R46.5 2.0 13:24 1 spamd 31333 mselig25 0 31732 30M 2564 R45.0 2.0 22:43 1 spamd This is happening since I disable the bayes_auto_expire. I'm doing something wrong here? On the spamd configuration file I have only defined the languages, the mount of score that a mail must to have to be flagged as spam, and the bayes_auto_expire to 0. Do I need to configure anything else?? BR, Matías.
Re: spamd process using to much cpu (again)
On Mon, Dec 06, 2004 at 09:03:20AM -0600, Smart,Dan wrote: > PMJI, but how do you know you exceeded the token threshold? What command do > you run, and what statistic do you look for? "sa-learn --dump magic" Check to see when ntokens goes over your bayes_expiry_max_db_size setting. -- Randomly Generated Tagline: Oh, the guys are work are going to have a field day with this. -- Homer Simpson The Call of the Simpsons pgpi8cGpLcslx.pgp Description: PGP signature
RE: spamd process using to much cpu (again)
PMJI, but how do you know you exceeded the token threshold? What command do you run, and what statistic do you look for? <> > -Original Message- > From: Michael Parker [mailto:[EMAIL PROTECTED] > Sent: Friday, December 03, 2004 5:43 PM > To: users@spamassassin.apache.org > Subject: Re: spamd process using to much cpu (again) > > On Fri, Dec 03, 2004 at 01:05:38PM -0300, Matias Lopez Bergero wrote: > > > > How often do I need to run sa-lern --force-expire? > > > > That highly depends on your email traffic. Turn off auto > expiration and watch your database for a day or so and > determine how long it takes to learn enough to go over the > token threshold, then setup your cron job accordingly. > > Michael >
Re: spamd process using to much cpu (again)
On Fri, Dec 03, 2004 at 01:05:38PM -0300, Matias Lopez Bergero wrote: > > How often do I need to run sa-lern --force-expire? > That highly depends on your email traffic. Turn off auto expiration and watch your database for a day or so and determine how long it takes to learn enough to go over the token threshold, then setup your cron job accordingly. Michael pgpJdkclB1Uft.pgp Description: PGP signature
Re: spamd process using to much cpu (again)
Michael Parker wrote: On Thu, Dec 02, 2004 at 05:07:48PM -0300, Matias Lopez Bergero wrote: Depending on your setup you could turn off bayes_auto_expire and do the expiration manually (ie sa-learn --force-expire), at a time that you control. Unfortunately, unless you are running a sitewide bayes config, this may not work well with a lot of individual bayes dbs. I am running sitewide installation. But now I got another question. I need to run sa-learn by hand? There is no way to configure spamd to do that?? You can script it, or run it via cron. spamd doesn't have this ability. How often do I need to run sa-lern --force-expire? BR, Matías.
Re: spamd process using to much cpu (again)
Michael Parker wrote: On Thu, Dec 02, 2004 at 05:07:48PM -0300, Matias Lopez Bergero wrote: If you're running 3.0, you could move your bayes databases to SQL which has a much faster expiration time (roughly 7 times faster). This would also allow you to offload some of the CPU and IO consumption to a separate machine. That would be nice :) Unfortunately I cannot put the bayes db on another machine, but I have a local mysql service, so if that improves the performance I could move the bayes db there. What would be your suggestion? I'm a little biased, but I suggest the mysql route. I'm going to do that. Thanks a lot Michael! BR, Matías.
Re: spamd process using to much cpu (again)
On Thu, Dec 02, 2004 at 05:07:48PM -0300, Matias Lopez Bergero wrote: > > >Depending on your setup you could turn off bayes_auto_expire and do > >the expiration manually (ie sa-learn --force-expire), at a time that > >you control. Unfortunately, unless you are running a sitewide bayes > >config, this may not work well with a lot of individual bayes dbs. > > I am running sitewide installation. > But now I got another question. > I need to run sa-learn by hand? There is no way to configure spamd to do > that?? You can script it, or run it via cron. spamd doesn't have this ability. > >If you're running 3.0, you could move your bayes databases to SQL > >which has a much faster expiration time (roughly 7 times faster). > >This would also allow you to offload some of the CPU and IO > >consumption to a separate machine. > > That would be nice :) > Unfortunately I cannot put the bayes db on another machine, but I have a > local mysql service, so if that improves the performance I could move > the bayes db there. > What would be your suggestion? I'm a little biased, but I suggest the mysql route. http://www.apache.org/~parker/presentations/ Michael pgpHiNd2VSt8N.pgp Description: PGP signature
Re: spamd process using to much cpu (again)
Michael Parker wrote: On Thu, Dec 02, 2004 at 04:25:46PM -0300, Matias Lopez Bergero wrote: From time to time, some spamd process sticks on top of the top listing with an ~90% CPU utilization There is a way to prevent this? > Possibly. What can be causing this hi CPU usage? I can't be 100% sure, but I'd put money on bayes expiration, which can be very CPU and IO intensive while it runs. Yes, I forgot to say that I was also having an increasing iowait usage. Some times I see about 150 to 190% of the CPU resources waisted iowait. Depending on your setup you could turn off bayes_auto_expire and do the expiration manually (ie sa-learn --force-expire), at a time that you control. Unfortunately, unless you are running a sitewide bayes config, this may not work well with a lot of individual bayes dbs. I am running sitewide installation. But now I got another question. I need to run sa-learn by hand? There is no way to configure spamd to do that?? If you're running 3.0, you could move your bayes databases to SQL which has a much faster expiration time (roughly 7 times faster). This would also allow you to offload some of the CPU and IO consumption to a separate machine. That would be nice :) Unfortunately I cannot put the bayes db on another machine, but I have a local mysql service, so if that improves the performance I could move the bayes db there. What would be your suggestion? Hope that helps. It really helps, Thank you! Matías.
Re: spamd process using to much cpu (again)
On Thu, Dec 02, 2004 at 04:25:46PM -0300, Matias Lopez Bergero wrote: > > Well, I fix the speed problem that I was having with SA, but I still > have the CPU consumption problem. > > From time to time, some spamd process sticks on top of the top listing > with an ~90% CPU utilization, like this: > > 27639 mselig39 19 30104 29M 2472 R N 105.2 1.9 60:04 0 spamd > > Here the spamd process is using 105.2% of my CPU resources, and 29MB of > memory. > Normally the spamd process uses between 10 and 30% of CPU resources, the > processes that use around 90% stay there until I kill them. > > There is a way to prevent this? Possibly. > What can be causing this hi CPU usage? I can't be 100% sure, but I'd put money on bayes expiration, which can be very CPU and IO intensive while it runs. Depending on your setup you could turn off bayes_auto_expire and do the expiration manually (ie sa-learn --force-expire), at a time that you control. Unfortunately, unless you are running a sitewide bayes config, this may not work well with a lot of individual bayes dbs. If you're running 3.0, you could move your bayes databases to SQL which has a much faster expiration time (roughly 7 times faster). This would also allow you to offload some of the CPU and IO consumption to a separate machine. Hope that helps. Michael pgppPgA9kvTWO.pgp Description: PGP signature
spamd process using to much cpu (again)
Hello, A couple of days ago, I post a msg asking for help with SA because it was to slow and the spamd processes was using to much resources. Well, I fix the speed problem that I was having with SA, but I still have the CPU consumption problem. From time to time, some spamd process sticks on top of the top listing with an ~90% CPU utilization, like this: 27639 mselig39 19 30104 29M 2472 R N 105.2 1.9 60:04 0 spamd Here the spamd process is using 105.2% of my CPU resources, and 29MB of memory. Normally the spamd process uses between 10 and 30% of CPU resources, the processes that use around 90% stay there until I kill them. There is a way to prevent this? What can be causing this hi CPU usage? Any help will be most welcome. BR, Matías.
Re: spamd process using to much cpu
problem solved. the thing was the LANG variable, it as set to en_US.UTF-8, I set to en_US and that was the end of the problem. :-X RB, Matías ps. it is still geting to much spam trought, where should i read to solve this?? Matías López Bergero wrote: Hello Matt, Thank you for ur answer, Matt Kettler dijo: At 04:13 PM 11/18/2004, Matias Lopez Bergero wrote: I'm seeing a heavy cpu usage in some process of spamd for a long time and sometimes they just hang there until I kill them(usage goes from 80% to 97%). Also my system is reporting a high iowait load and a high disk usage that stops if a shutdown spammassassin processes. This is normal? Anyone with the same problem?? Define "for a long time"... Minutes? Hours? Less than a minute, but wen it hangs, it hangs there until i kill it. I haved noticed that this spamd process hanging is ocurring with the same user almost all the times. That means anything to you? From the sounds of it, it looks like SA is doing an opportunistic expiry on your bayes DB.. But that should only take a few minutes unless things are really haywire or your box is really slow. Try running a sa-learn -D --force-expire on the command line and see if that runs smoothly. Also, look around for bayes_toks.expire files laying around next to your bayes DB.. that's a very clear sign SA is being killed while running expiry. I'm going to try that. Thanks again! BR, Matías.
Re: spamd process using to much cpu
Hello Matt, Thank you for ur answer, Matt Kettler dijo: > At 04:13 PM 11/18/2004, Matias Lopez Bergero wrote: >>I'm seeing a heavy cpu usage in some process of spamd for a long time and >>sometimes they just hang there until I kill them(usage goes from 80% to >> 97%). >>Also my system is reporting a high iowait load and a high disk usage that >>stops if a shutdown spammassassin processes. >> >>This is normal? >>Anyone with the same problem?? > > Define "for a long time"... Minutes? Hours? Less than a minute, but wen it hangs, it hangs there until i kill it. I haved noticed that this spamd process hanging is ocurring with the same user almost all the times. That means anything to you? > From the sounds of it, it looks like SA is doing an opportunistic expiry > on your bayes DB.. But that should only take a few minutes unless things > are really haywire or your box is really slow. > > Try running a sa-learn -D --force-expire on the command line and see if > that runs smoothly. > > Also, look around for bayes_toks.expire files laying around next to > your bayes DB.. that's a very clear sign SA is being killed while running > expiry. > I'm going to try that. Thanks again! BR, Matías.
Re: spamd process using to much cpu
At 04:13 PM 11/18/2004, Matias Lopez Bergero wrote: I'm seeing a heavy cpu usage in some process of spamd for a long time and sometimes they just hang there until I kill them(usage goes from 80% to 97%). Also my system is reporting a high iowait load and a high disk usage that stops if a shutdown spammassassin processes. This is normal? Anyone with the same problem?? Define "for a long time"... Minutes? Hours? From the sounds of it, it looks like SA is doing an opportunistic expiry on your bayes DB.. But that should only take a few minutes unless things are really haywire or your box is really slow. Try running a sa-learn -D --force-expire on the command line and see if that runs smoothly. Also, look around for bayes_toks.expire files laying around next to your bayes DB.. that's a very clear sign SA is being killed while running expiry.
spamd process using to much cpu
Hello, I'm running spamassassin 3.0.1 on linux 2.4, using milter-spamc to talk with sendmail milter. I'm seeing a heavy cpu usage in some process of spamd for a long time and sometimes they just hang there until I kill them(usage goes from 80% to 97%). Also my system is reporting a high iowait load and a high disk usage that stops if a shutdown spammassassin processes. This is normal? Anyone with the same problem?? I'm running spamassassin with this flags: -d -c -m5 -H -D Any comments/ideas will be most welcome. BR, Matías.
