Re: One thing about bug 6558

2011-04-02 Thread Henrik K
On Fri, Apr 01, 2011 at 04:08:02PM -0400, Kris Deugau wrote: We switched from MyISAM on physical disk to MyISAM on RAMdisk. Since we're running a global Bayes setup, ~2G of disk is plenty. A little tweaking of the MySQL init script makes sure the tmpfs is mounted, and then loads a backup

Re: One thing about bug 6558

2011-04-01 Thread Marcin Mirosław
W dniu 30.03.2011 16:27, Adam Moffett pisze: Your watchdog idea is a valid one, Should i fill bug with this idea (or bug descibed there: http://osdir.com/ml/users-spamassassin/2011-03/msg00481.html or there: http://www.gossamer-threads.com/lists/spamassassin/users/161931#161931 ) ? Or it's

Re: One thing about bug 6558

2011-04-01 Thread Mark Martinec
Marcin, This option doesn't work as i wish ;) spamd child isn't killing after time set in --timeout-child, it is working, and working and utilizes 100% CPU until i send kill signal. It seems for me this option works as described below: (for this example --timeout-child=10 sec) Should i

Re: One thing about bug 6558

2011-04-01 Thread Mark Martinec
David F. Skoll wrote: Using Pg for Bayes data will be really slow. We don't use the SpamAssassin Bayes implementation and we went through three iterations of storage back-ends before finding one we liked. 1) PostgreSQL: Convenient but slow. 2) Berkeley DB: Faster than PostgreSQL, but

Re: One thing about bug 6558

2011-04-01 Thread David F. Skoll
On Fri, 1 Apr 2011 19:52:54 +0200 Mark Martinec mark.martinec...@ijs.si wrote: I can very much believe and agree that for a read-only bayes database the CDB provides the best performance - as long as you can afford (or have no other choice in large scale environments) to update it

Re: One thing about bug 6558

2011-04-01 Thread Mark Martinec
Btw, I forgot to add that the: bayes_auto_learn_on_error 1 feature in SpamAssassin trunk is a big help in keeping an auto-learning Bayes database growth in check. $ man Mail::SpamAssassin::Plugin::AutoLearnThreshold bayes_auto_learn_on_error (0 | 1)(default: 0) With

Re: One thing about bug 6558

2011-04-01 Thread Michael Scheidell
On 4/1/11 1:52 PM, Mark Martinec wrote: The MyISAM screwups were finally resolved by switching our database to a InnoDB -type engine and using Mail::SpamAssassin::BayesStore::MySQL back-end (instead of the Mail::SpamAssassin::BayesStore::SQL). This finally solver the reliability/stability

Re: One thing about bug 6558

2011-04-01 Thread Kris Deugau
Mark Martinec wrote: When we started using SpamAssassin years back our bayes and awl databases were on a Berkeley DB. This worked reasonably well (sharing your opinion on being 'occasionally flaky'), but the auto-expiration long times started to grow from minutes to hours. Initially this was

Re: One thing about bug 6558

2011-04-01 Thread Mark Martinec
Kris Deugau wrote: Mark Martinec wrote: When we started using SpamAssassin years back our bayes and awl databases were on a Berkeley DB. This worked reasonably well (sharing your opinion on being 'occasionally flaky'), but the auto-expiration long times started to grow from minutes to

One thing about bug 6558

2011-03-30 Thread Marcin Mirosław
Hi! Sometimes (for example bug 6558) happen than spamd child(s) utilizes 100% cpu for a long time (until i kill them myself). My suggestion is to add new functionality to SA, something like watchdog for spamd child processes. If child hangs to long with email, it should be killed by parrent

Re: One thing about bug 6558

2011-03-30 Thread Daniel Lemke
://old.nabble.com/One-thing-about-bug-6558-tp31275747p31275828.html Sent from the SpamAssassin - Users mailing list archive at Nabble.com.

Re: One thing about bug 6558

2011-03-30 Thread Marcin Mirosław
W dniu 30.03.2011 12:33, Daniel Lemke pisze: You mean something like --timeout-child=secs as a spamd starting option? ;) http://spamassassin.apache.org/full/3.3.x/doc/spamd.html Thanks for quick reply! This option doesn't work as i wish ;) spamd child isn't killing after time set in

Re: One thing about bug 6558

2011-03-30 Thread Marcin Mirosław
W dniu 30.03.2011 14:06, Per Jessen pisze: Have you looked at what spamd is doing when it so busy? Did You mean spamd child? At this moment bayes engine do very hard work with email.

Re: One thing about bug 6558

2011-03-30 Thread Per Jessen
Marcin Mirosław wrote: W dniu 30.03.2011 14:06, Per Jessen pisze: Have you looked at what spamd is doing when it so busy? Did You mean spamd child? At this moment bayes engine do very hard work with email. Yes, I meant the child - obviously, it sounds as if it's a problem in the bayes

Re: One thing about bug 6558

2011-03-30 Thread Marcin Mirosław
W dniu 30.03.2011 15:47, Per Jessen pisze: Yes, I meant the child - obviously, it sounds as if it's a problem in the bayes processing. I don't use SA bayes, but that problem ought to be investigated first before we look at work-arounds. IMHO. I'm expecting that bayes can do its work for long

Re: One thing about bug 6558

2011-03-30 Thread Per Jessen
Marcin Mirosław wrote: W dniu 30.03.2011 15:47, Per Jessen pisze: Yes, I meant the child - obviously, it sounds as if it's a problem in the bayes processing. I don't use SA bayes, but that problem ought to be investigated first before we look at work-arounds. IMHO. I'm expecting that

Re: One thing about bug 6558

2011-03-30 Thread Kris Deugau
Marcin Mirosław wrote: W dniu 30.03.2011 15:47, Per Jessen pisze: Yes, I meant the child - obviously, it sounds as if it's a problem in the bayes processing. I don't use SA bayes, but that problem ought to be investigated first before we look at work-arounds. IMHO. I'm expecting that bayes

Re: One thing about bug 6558

2011-03-30 Thread Marcin Mirosław
W dniu 30.03.2011 16:21, Per Jessen pisze: Well, isn't the behaviour you're seeing working-as-expected then? If it was an indefinite loop, setting up a time-out would be a possible work-around. If the bayes code is doing what it is supposed to do, but just taking long to do it, no

Re: One thing about bug 6558

2011-03-30 Thread Marcin Mirosław
W dniu 30.03.2011 16:21, Per Jessen pisze: Well, isn't the behaviour you're seeing working-as-expected then? If it was an indefinite loop, setting up a time-out would be a possible work-around. If the bayes code is doing what it is supposed to do, but just taking long to do it, no

Re: One thing about bug 6558

2011-03-30 Thread Marcin Mirosław
W dniu 30.03.2011 16:27, Adam Moffett pisze: Your watchdog idea is a valid one, but if you can improve bayes performance it may become a non-issue. Have you tried moving bayes to a MySQL database? It consumes more disk space and memory, but it's dramatically faster than using the default

Re: One thing about bug 6558

2011-03-30 Thread Per Jessen
Marcin Mirosław wrote: W dniu 30.03.2011 16:21, Per Jessen pisze: Well, isn't the behaviour you're seeing working-as-expected then? If it was an indefinite loop, setting up a time-out would be a possible work-around. If the bayes code is doing what it is supposed to do, but just taking

Re: One thing about bug 6558

2011-03-30 Thread David F. Skoll
On Wed, 30 Mar 2011 16:51:57 +0200 Marcin Mirosław mar...@mejor.pl wrote: I'm using postgresql, but machine isn't quick... Any db is slowly there. Using Pg for Bayes data will be really slow. We don't use the SpamAssassin Bayes implementation and we went through three iterations of storage

Re: One thing about bug 6558

2011-03-30 Thread Marcin Mirosław
W dniu 30.03.2011 16:34, Kris Deugau pisze: My experience here has been that if a spamd child is pegging a CPU core for an extended period, there's simply a *lot* of body text to run (raw)body rules against (eg, ~ 200K). We've found that a fairly effective defense against this is to set up a

Re: One thing about bug 6558

2011-03-30 Thread RW
On Wed, 30 Mar 2011 13:17:55 +0200 Marcin Mirosław mar...@mejor.pl wrote: W dniu 30.03.2011 12:33, Daniel Lemke pisze: You mean something like --timeout-child=secs as a spamd starting option? ;) http://spamassassin.apache.org/full/3.3.x/doc/spamd.html Thanks for quick reply! This option

Re: One thing about bug 6558

2011-03-30 Thread Marcin Mirosław
W dniu 30.03.2011 17:12, RW pisze: Are you sure that there were idle spamd children? The above is consistent with ~14 seconds queuing for a child, followed by a timeout after 10 seconds. I was watching cpu time of spamd child (i took delta, not last cpu time ;) ). 14+10 is coincidence.