RE: SARE rules timing out?
-Original Message- From: jdow [mailto:[EMAIL PROTECTED] Sent: Thursday, January 06, 2005 4:17 PM To: users@spamassassin.apache.org Subject: Re: SARE rules timing out? From: Chris Santerre [EMAIL PROTECTED] Are you saying that using spamd/c gives you problems for users who have their own local rules? Just curious as to what problem? --Chris I have not migrated Loren over to the new machine because there is a problem HERE with spamd. I set it down to only allow one client to make the problem more obvious. I have local.cf allowing user rules as well as user scores. I have my own set of local rules and Loren has his, which he uses while testing rules before they become SARE rules. So we each need our own cache of personal rules. The first time I use spamc to scan a message all is just fine. My rules and scores trigger properly. The SECOND time I use spamc (actually the second trip through the same spamd when multiple spamd's are running) still sees the message triggering on my rules. But every one of my rule's scores were arbitrarily set to one rather than the score in my .spamassassin/user_prefs file. This drove me nuts finding it. No, it drove me over the edge. I was using language that would make a sailor blush. He**, it would even have made a hockey player blush and get sent to the showers with an order to wash his mouth out with soap. I am running through postfix and procmail without the postfix being in its little jail. (If in its jail how would the procmail be able to fire off with MY rules? Maybe that's not a worry. But that is another problem. I tested this with spamc directly spamc test_message and saw the effect that way. So it's not in the postfix or procmail part of the chain.) OK, that is very interesting. I'm assuming your custom scores are stored in the same file as your custom rules? Only spamc is doing this, correct? spamassassin scores correct? Is it every run after the first, or only the second? --Chris (Silly questions I know, but getting my head around it.)
Re: SARE rules timing out?
From: Chris Santerre [EMAIL PROTECTED]
-Original Message-
From: jdow [mailto:[EMAIL PROTECTED]
From: Chris Santerre [EMAIL PROTECTED]
Are you saying that using spamd/c gives you problems for
users who have
their own local rules? Just curious as to what problem?
--Chris
I have not migrated Loren over to the new machine because there is a
problem HERE with spamd. I set it down to only allow one client to
make the problem more obvious.
I have local.cf allowing user rules as well as user scores. I have
my own set of local rules and Loren has his, which he uses while
testing rules before they become SARE rules. So we each need our
own cache of personal rules.
The first time I use spamc to scan a message all is just fine. My
rules and scores trigger properly.
The SECOND time I use spamc (actually the second trip through
the same spamd when multiple spamd's are running) still sees
the message triggering on my rules. But every one of my rule's
scores were arbitrarily set to one rather than the score in my
.spamassassin/user_prefs file. This drove me nuts finding it.
No, it drove me over the edge. I was using language that would
make a sailor blush. He**, it would even have made a hockey
player blush and get sent to the showers with an order to wash
his mouth out with soap.
I am running through postfix and procmail without the postfix
being in its little jail. (If in its jail how would the procmail
be able to fire off with MY rules? Maybe that's not a worry. But
that is another problem. I tested this with spamc directly
spamc test_message and saw the effect that way. So it's not
in the postfix or procmail part of the chain.)
OK, that is very interesting. I'm assuming your custom scores are stored
in
the same file as your custom rules?
Only spamc is doing this, correct? spamassassin scores correct?
Is it every run after the first, or only the second?
--Chris (Silly questions I know, but getting my head around it.)
That is entirely true. There is only one file, ~/.spamassassin/user_prefs,
that contains both the custom rules and custom scores typically as a
threesome of rule, description, and score in that order. I've fiddled
with test messages to trigger on multiple rules in that file and they
all show a score of 1 rather than the designated score on the second
and subsequent runs of spamc when spamd is configured -m1. (It appears
sporadically with -m5 until all five spamd's have serviced at least
one message. Then it's 100% skipping my scores.)
If you can duplicate this we can toss a BK bug report into the system.
I suspect it may have a relationship to the apparent memory leaks that
are only sort of fixed with 3.0.2.
{^_^}
SARE rules timing out?
Does anyone know if any of the SARE rules are causing timeouts? My server was bogging down really bad. I removed the custom rules and that seems to have fixed it. Only thing is, I don't know which one is causing the problem. Here is what I have: SARE_ADULT SARE BIZ/Marketing/Learning Ruleset SARE Fraud Detection EvilNumber SARE_HTML_ENG SARE_HEADER_ENG SARE Specific SARE Spoof SARE Random SARE OEM SARE_GENLSUBJ SARE Abused Redirect Subject SARE_GENLSUB_ENGJ SARE UNSUB SARE uri TripWire Mangled Words Any way to trim this list down and still see good effects? Thanks.
Re: SARE rules timing out?
The usual solution is more memory.
You need to tell us what version of SpamAssassin you are running, how
much memory you have in the machine, and how you are using SpamAssassin
including the options. Your mail load will also make a difference.
The SARE rules do consume a lot of memory. But they do not cause the
machine to bog down terribly. A typical spamassassin -t run here is
about 2.7 seconds on a 2G Athlon with a gig of memory. That is one of
the more brutal ways to run spamassassin, though. spamc-spamd was
under half that time. (I have a peculiar problem with per user rules
with that configuration so I do not use it.)
{^_^}
- Original Message -
From: MIKE YRABEDRA [EMAIL PROTECTED]
Does anyone know if any of the SARE rules are causing timeouts? My server
was bogging down really bad. I removed the custom rules and that seems to
have fixed it. Only thing is, I don't know which one is causing the
problem.
Re: SARE rules timing out?
I am using SA 3.0.2, CGPSA 1.4, CommunigatePro, Mac OS X
When the custom rules are present, process times are ranging from 12-45
seconds per message. This is causing a bottleneck.
It looks like it may be a DNS lookup issue, but I can't be sure.
on 1/6/05 2:06 PM, Chris Santerre at [EMAIL PROTECTED] wrote:
Are you saying that using spamd/c gives you problems for users who have
their own local rules? Just curious as to what problem?
--Chris
-Original Message-
From: jdow [mailto:[EMAIL PROTECTED]
Sent: Thursday, January 06, 2005 1:19 PM
To: users@spamassassin.apache.org
Subject: Re: SARE rules timing out?
The usual solution is more memory.
You need to tell us what version of SpamAssassin you are running, how
much memory you have in the machine, and how you are using SpamAssassin
including the options. Your mail load will also make a difference.
The SARE rules do consume a lot of memory. But they do not cause the
machine to bog down terribly. A typical spamassassin -t run here is
about 2.7 seconds on a 2G Athlon with a gig of memory. That is one of
the more brutal ways to run spamassassin, though. spamc-spamd was
under half that time. (I have a peculiar problem with per user rules
with that configuration so I do not use it.)
{^_^}
- Original Message -
From: MIKE YRABEDRA [EMAIL PROTECTED]
Does anyone know if any of the SARE rules are causing
timeouts? My server
was bogging down really bad. I removed the custom rules and
that seems to
have fixed it. Only thing is, I don't know which one is causing the
problem.
++
Mike Yrabedra (President)
323 Incorporated
Our Sites:
MacDock.com
MacAgent.com
iTuneAgent.com
MacSurfShop.com
++
W: http://www.323inc.com/
P: 770.382.1195
F: 734.448.5164
E: [EMAIL PROTECTED]
I: ichatmacdock
++
Whatever you do, work at it with all your heart,
as working for the Lord, not for men.
~Colossians 3:23 {{{
++
Re: SARE rules timing out?
Does anyone know if any of the SARE rules are causing timeouts? My server was bogging down really bad. I removed the custom rules and that seems to have fixed it. Only thing is, I don't know which one is causing the problem. In general timeouts tend to get caused by net tests. I can't recall any of our tests that are currently net tests, so the likelyhood of a SARE rule timeout should be small. However - a regexp can be a dangerous thing, and a pathelogical spam input to one might end up finding a logic hole that can take a VERY long time. You may have found such a thing. However, it should happen on very specific spams, and not in general. We did have that happen once before with one re, long since fixed. In general .* is dangerous, and it likely occurs in several of our rules. Of the rulesets you listed it is a little hard to guess which (if any) might be the culprit. About all I can suggest is divide and conquer. I would start by trying to find a particular message that causes the problem, so you can run it through SA by hand. Then I would start lopping rulesets out until the problem went away. Then it would be possible to start chopping that single ruleset down until the problem became fairly obvious. Loren
Re: SARE rules timing out?
When the custom rules are present, process times are ranging from 12-45 seconds per message. This is causing a bottleneck. It looks like it may be a DNS lookup issue, but I can't be sure. Ah. If this is a general thing where all messages slow down, then I have to agree - you probably need more memory. See what your memory usage is. Loren
Re: SARE rules timing out?
From: Chris Santerre [EMAIL PROTECTED]
Are you saying that using spamd/c gives you problems for users who have
their own local rules? Just curious as to what problem?
--Chris
I have not migrated Loren over to the new machine because there is a
problem HERE with spamd. I set it down to only allow one client to
make the problem more obvious.
I have local.cf allowing user rules as well as user scores. I have
my own set of local rules and Loren has his, which he uses while
testing rules before they become SARE rules. So we each need our
own cache of personal rules.
The first time I use spamc to scan a message all is just fine. My
rules and scores trigger properly.
The SECOND time I use spamc (actually the second trip through
the same spamd when multiple spamd's are running) still sees
the message triggering on my rules. But every one of my rule's
scores were arbitrarily set to one rather than the score in my
.spamassassin/user_prefs file. This drove me nuts finding it.
No, it drove me over the edge. I was using language that would
make a sailor blush. He**, it would even have made a hockey
player blush and get sent to the showers with an order to wash
his mouth out with soap.
I am running through postfix and procmail without the postfix
being in its little jail. (If in its jail how would the procmail
be able to fire off with MY rules? Maybe that's not a worry. But
that is another problem. I tested this with spamc directly
spamc test_message and saw the effect that way. So it's not
in the postfix or procmail part of the chain.)
{O.O}
Re: SARE rules timing out?
That is very likely if you have not told it a valid DNS server for
use with the DNS tests.
Can you fire up a local caching only name server?
{^_^}
- Original Message -
From: MIKE YRABEDRA [EMAIL PROTECTED]
I am using SA 3.0.2, CGPSA 1.4, CommunigatePro, Mac OS X
When the custom rules are present, process times are ranging from 12-45
seconds per message. This is causing a bottleneck.
It looks like it may be a DNS lookup issue, but I can't be sure.
