Re: [SPAM] (6.70/5.00) Re: [FW: spam control

2005-07-28 Thread Matt Kettler
The Doctor wrote:

> Question:  How can ever user use Spam Assassin without having to specify a
> user?  It would be nice for every user to govern their own account.
> 

SA will scan mail as the userid of the process calling SA, unless you pass -u to
either spamc or spamd.

SA has no way of reliably determining who the recipient is based on the message
content. It cannot "automatically figure it out" for you.

Typically people using procmail just do a quick trick where the envelope
recipient gets passed to -u. I don't use procmail so I don't know the exact
syntax, but there's a variable (%1?) that gets used for this.

I'm not sure if any of the milters can do the same trick, but there's nothing
about the nature of a milter to prevent this. The only problem is that at the
milter level you'll see outbound mail, which you won't have a local user for.

Procmail on the other hand only gets called for mail that's locally delivered,
so you can know for sure the recipient is local.



OT: Dr's. was [FW: spam control

2005-07-28 Thread Andy Jezierski

"jdow" <[EMAIL PROTECTED]> wrote
on 07/27/2005 06:35:46 PM:

[snip]
> 
> (One thing I have found is that people who use the term "Dr."
in front
> of their monikers when "out in public" are incapable of
learning because
> all the public is too dumb to listen to. It earns them incredible
amounts
> of heartburn.)
>
 
One of the guys I work with has earned the title Dr. Death.  Back
in the Windows 3.1 days he managed to cause BSOD's daily and call us for
help. When asked what had changed, oh nothing, I just installed this beta
software, or this new device driver We used to have a field day with
him, GREAT guy, doesn't use the Dr. title at all, we still tease him about
it to this day.

One of his best comments:  Yes I know that Phd
stands for Push Here Dummy.

I still crack up at that one.
Andy 

Re: [SPAM] (6.70/5.00) Re: [FW: spam control

2005-07-27 Thread The Doctor
On Wed, Jul 27, 2005 at 06:00:58PM -0600, The Doctor wrote:
> On Wed, Jul 27, 2005 at 04:45:36PM -0400, Matt Kettler wrote:
> > The Doctor wrote:
> > >
> > > 
> > > The whitelist in question:
> > > 
> > > /.spamassassin/user_prefs:
> > > 
> > > 
> > 
> > 
> > 
> > > 
> > > And the spamassassin is called as follows:
> > > 
> > >   echo -n ' Spam Assassin';   /usr/contrib/bin/spamd -d -i -D -u 
> > > defang --user-config --siteconfigpath=/etc/mail/spamassassin 
> > > --syslog=/var/log/spamd.log --pidfile=/var/run/spamd.pid;
> > >   /usr/contrib/bin/smf-spamd;
> > 
> > 
> > is "" in the user_prefs path the home directory for the user 
> > "defang"...
> > if not, then that whole file will NOT under ANY condition be read.
> > 
> > Since you're passing -u defang to spamd, it will ONLY run as defang, and it 
> > will
> > ONLY check defang's home directory for a user_prefs file.
> > 
> 
> 
> Question:  How can ever user use Spam Assassin without having to specify a
> user?  It would be nice for every user to govern their own account.
> 

Also, IS it possible for Spam Assassin to skip over a realm?

-- 
Member - Liberal International  
This is [EMAIL PROTECTED]   Ici [EMAIL PROTECTED]
God Queen and country! Beware Anti-Christ rising!
Better to serve in Heaven that to Rule in Hell.


Re: [SPAM] (6.70/5.00) Re: [FW: spam control

2005-07-27 Thread The Doctor
On Wed, Jul 27, 2005 at 04:45:36PM -0400, Matt Kettler wrote:
> The Doctor wrote:
> >
> > 
> > The whitelist in question:
> > 
> > /.spamassassin/user_prefs:
> > 
> > 
> 
> 
> 
> > 
> > And the spamassassin is called as follows:
> > 
> > echo -n ' Spam Assassin';   /usr/contrib/bin/spamd -d -i -D -u 
> > defang --user-config --siteconfigpath=/etc/mail/spamassassin 
> > --syslog=/var/log/spamd.log --pidfile=/var/run/spamd.pid;
> > /usr/contrib/bin/smf-spamd;
> 
> 
> is "" in the user_prefs path the home directory for the user "defang"...
> if not, then that whole file will NOT under ANY condition be read.
> 
> Since you're passing -u defang to spamd, it will ONLY run as defang, and it 
> will
> ONLY check defang's home directory for a user_prefs file.
> 


Question:  How can ever user use Spam Assassin without having to specify a
user?  It would be nice for every user to govern their own account.

-- 
Member - Liberal International  
This is [EMAIL PROTECTED]   Ici [EMAIL PROTECTED]
God Queen and country! Beware Anti-Christ rising!
Better to serve in Heaven that to Rule in Hell.


Re: [FW: spam control

2005-07-27 Thread jdow
From: "Kai Schaetzl" <[EMAIL PROTECTED]>

> The Doctor wrote on Wed, 27 Jul 2005 13:34:42 -0600:
> 
> > This is [EMAIL PROTECTED] Ici [EMAIL PROTECTED]
> 
> Ah, it's he again. Setting to ignore mode.
> 
> Kai

He rather is an example of why we tended not to allow our doctorates
into the lab when I was doing RF engineering for Rockwell International.
They broke everything they touched. Heck, one fellow only had to step
inside the room and half the equipment quit working.

If he has a doctorate in "Computer Science" he knows too much to get
SpamAssassin running. He knows how things "should be done" and knows
that what he knows is "absolutely the only way things should be done."
So why sit down and figure out how it really works carefully and
methodically. He religiously seems to hide major pieces of his
configuration from us and then demand solutions. I've quit even bothering
to reply to him. I do read him. He's so silly he's amusing.

(One thing I have found is that people who use the term "Dr." in front
of their monikers when "out in public" are incapable of learning because
all the public is too dumb to listen to. It earns them incredible amounts
of heartburn.)

{^_^}



Re: [FW: spam control

2005-07-27 Thread Matt Kettler
The Doctor wrote:
> 
> Next?
> 

My honest suggestion?

Stop everything, and take a step back. Read, think about the options, then act.

First, Fix your setup as Andy Jezerski suggested. Have ONE and only ONE call to
spamassassin. You've got 3 right now. Two milters and a procmail call. That's
VERY bad news, and will greatly complicate configuration, testing and debugging.

Pick ONE of the following:
smf-spamd
milter-spamc
procmailrc call to spamc

And ditch the other two. With all three of them in place, that's 3 tools you
have to configure, and if any one of them isn't set up right you'll have
problems. Reducing it to one tool, one call, will make your life easier.

Second, I would personally just get rid of your local.cf and start over. At the
very minimum get rid of every "score" statement you've added in there.

You've been raising rule scores all over the place, which wound up causing FP
problems. Then you raised your threshold to counteract the FP problems your
modified scores caused. Bad news. You're getting into an arms race with 
yourself.

Third, once you've picked one of the methods of calling SpamAssassin (instead of
three) configure that tool to bypass SA calls. If you decide to keep
milter-spamc, I'd suggest using Andy's suggestion of a /etc/mail/access 
statement.

Milter-Spamc-From:[EMAIL PROTECTED]OK




Re: [FW: spam control

2005-07-27 Thread Kai Schaetzl
The Doctor wrote on Wed, 27 Jul 2005 13:34:42 -0600:

> This is [EMAIL PROTECTED] Ici [EMAIL PROTECTED]

Ah, it's he again. Setting to ignore mode.

Kai

-- 
Kai Schätzl, Berlin, Germany
Get your web at Conactive Internet Services: http://www.conactive.com
IE-Center: http://ie5.de & http://msie.winware.org





Re: [FW: spam control

2005-07-27 Thread The Doctor
On Wed, Jul 27, 2005 at 05:25:42PM -0400, Matt Kettler wrote:
> Chris Santerre wrote:
> >>score   gtube   4.0
> 
> > 
> > 
> > *snip*
> > 
> > Holy carp!!! Why did you rescore just about every rule higher? 
> 
> An even better question.. why did he try to rescore GTUBE down to 4.0?
> 
> Although that was slightly screwed up by not puting the rule name in all-caps,
> GTUBE should always cause a message to be high-scoring spam.
> 
> That's the whole point of GTUBE. GTUBE detects a really odd-ball test-string
> which should never be present in normal email, and it's kind of like the EICAR
> virus-test string, but for spam.
> 
> 
> 


I did try to go back to default and raise the level to 7.5 and
did try to restart spamd amd spamc, but it seems that Spam Assassin
still has the old high features.

The local.cf looks like:
-

# Add your own customisations to this file.  See 'man Mail::SpamAssassin::Conf'
# SpamAssassin user preferences file.
#
# Format:
#
#   required_hits n
#   (how many hits are required to tag a mail as spam.)
#
#   score SYMBOLIC_TEST_NAME n
#   (if this is omitted, 1 is used as a default score.
#   Set the score to 0 to ignore the test.)
#
# # starts a comment, whitespace is not significant.
#
# NOTE!  In conjunction with MIMEDefang, SpamAssassin can *NOT* make any
# changes to the message header or body.  Any SpamAssassin settings that
# relate to changing the message will have *NO EFFECT* when used from
# MIMEDefang.  Instead, use the various MIMEDefang Perl functions if you
# need to alter the message.
###

###
# First of all, the generally useful stuff; thresholds and the whitelist
# of addresses which, for some reason or another, often trigger false
# positives.

required_hits   7.5

# Whitelist and blacklist addresses are *not* patterns; they're just normal
# strings.  one exception is that "[EMAIL PROTECTED]" is allowed.  They should 
be in
# lower-case.  You can either add multiple addrs on one line,
# whitespace-separated, or you can use multiple lines.
#
# Monty Solomon: he posts from an ISP that has often been the source of spam
# (no fault of his own ;), and sometimes uses Bcc: when mailing.
#
# whitelist_from[EMAIL PROTECTED]

# Add your blacklist entries in the same format...
#
# blacklist_from[EMAIL PROTECTED]

# Mail using languages used in these country codes will not be marked
# as being possibly spam in a foreign language.
#
##ok_localesen

# By default, the subject lines of suspected spam will be tagged.
# This can be disabled here.
#
##rewrite_subject 0
# By default, spamassassin will include its report in the body
# of suspected spam. Enabling this causes the report to go in the
# headers instead. Using 'use_terse_report' for this is recommended.
#
# report_header 1

# By default, SpamAssassin uses a fairly long report format.
# Enabling this uses a shorter format which includes all the
# information in the normal one, but without the superfluous
# explanations.
#
# use_terse_report 0

# By default, spamassassin will change the Content-type: header of
# suspected spam to "text/plain". This is a safety feature. If you
# prefer to leave the Content-type header alone, set this to 0.
#
defang_mime 0

# By default, SpamAssassin will run RBL checks.  If your ISP already
# does this, set this to 1.

#skip_rbl_checks 1

###
# Add your own customised scores for some tests below.  The default scores are
# read from the installed "spamassassin.cf" file, but you can override them
# here.  To see the list of tests and their default scores, go to
# http://spamassassin.taint.org/tests.html .

# for details of what can be tweaked.
#

# SpamAssassin config file for version 2.5x
# generated by http://www.yrex.com/spam/spamconfig.php (version 1.01)

# How many hits before a message is considered spam.
required_hits   7.5

# Whether to change the subject of suspected spam
##rewrite_subject 1

# Text to prepend to subject if rewrite_subject is used
##subject_tag *SPAM*
rewrite_header Subject SPAM(_SCORE_)   

# Encapsulate spam in an attachment
report_safe 1

# Use terse version of the spam report
use_terse_report0

# Enable the Bayes system
use_bayes   1

# Enable Bayes auto-learning
auto_learn  1

# Enable or disable network checks
skip_rbl_checks 0
use_razor2  1
use_dcc 1
use_pyzor   1

# Mail using languages used in these country codes will not be marked
# as being possibly spam in a foreign language.
ok_languagesall

# Mail using locales used in these country codes will not be marked
# as being possibly spam in a foreign lang

Re: [FW: spam control

2005-07-27 Thread Andy Jezierski

The Doctor <[EMAIL PROTECTED]> wrote
on 07/27/2005 03:42:41 PM:

[snip]
> 
> In my /etc/rc for spam assassin I have,
> 
> 
>    echo -n ' Spam Assassin';   /usr/contrib/bin/spamd
-d -i -D -u 
> defang --user-config --siteconfigpath=/etc/mail/spamassassin --
> syslog=/var/log/spamd.log --pidfile=/var/run/spamd.pid;

You do realize you have debugging turned on right?
 -D  Makes for a HUGE log file each day.


>    /usr/contrib/bin/smf-spamd;

First call to SpamAssassin  (Is this spam?)

[snip]

>     /usr/contrib/bin/daemon /usr/contrib/bin/milter-spamc
-r 50 -S -
> v all unix:/var/lib/milter-spamc/socket;

Second call to SpamAssassin   (Is this REALLY
spam?)

[snip]

> 
> /etc/procmailrc in my system reads:
> 
> 
> :0fw:spamassassin.lock
> * < 1000
> |/usr/contrib/bin/spamc
> 
> :0 w
> ! -oi -f "$@"
> 

Third call to SpamAssassin   (Are you REALY REALY
sure this is spam?)

Pick ONE method to call SA

Andy

Re: [FW: spam control

2005-07-27 Thread Matt Kettler
Chris Santerre wrote:
>>score gtube   4.0

> 
> 
> *snip*
> 
> Holy carp!!! Why did you rescore just about every rule higher? 

An even better question.. why did he try to rescore GTUBE down to 4.0?

Although that was slightly screwed up by not puting the rule name in all-caps,
GTUBE should always cause a message to be high-scoring spam.

That's the whole point of GTUBE. GTUBE detects a really odd-ball test-string
which should never be present in normal email, and it's kind of like the EICAR
virus-test string, but for spam.





Re: [FW: spam control

2005-07-27 Thread Andy Jezierski

The Doctor <[EMAIL PROTECTED]> wrote
on 07/27/2005 03:51:13 PM:

[snip]
> > X-Spam-Status: NO, hits=2.20 required=5.00
> > 
> > X-Spam-Level: xx
> > 
> > X-Mark-SPAM: YES, score=5.40/5.00, processed for 2.536s on doctor.nl2k.ab.ca
> > 
> > [...]
> > 
> > What looks odd to me is that X-Spam-Status says NO (I'm assuming
that 
> > this comes from sa), level is only 2, but X-Mark-Spam: is yes..
with a 
> > score of 5.40.. where is this coming from?
> >
> 
> I am using milter-spamc and smf-spamd .
> 
>  

Pick one milter and get rid of the other. 

Above, Milter-spamc said the message wasn't spam,
and I'm assuming that the X-Mark-SPAM is from smf-spamd said the message
is spam. Looks like your two milters might be looking at different configs,
since they are coming up with different scores.  Also as Chris said,
get rid of ALL of your score overrides. That's probably your biggest problem.

Andy 

Re: [FW: spam control

2005-07-27 Thread The Doctor
On Wed, Jul 27, 2005 at 04:40:40PM -0400, JamesDR wrote:
> The Doctor wrote:
> >On Wed, Jul 27, 2005 at 03:48:22PM -0400, Jim Maul wrote:
> >
> >>The Doctor wrote:
> >>
> >>
> >>
> >>>
> >>>All right, the short and simple is that Spam-Assassin may not be doing
> >>>the correct job.  This user has a whitelist in place and
> >>>some e-mail are getting the label of spam.
> >>>
> >>>Even some of my cron jobs are getting  a [SPAM] label when they should 
> >>>nt.
> >>>
> >>>Why?
> >>>
> >>
> >>
> >>Perhaps if you posted the headers of the messages that were marked as 
> >>spam we can look to see what rules hit which would answer your "why?" 
> >>question.  Until then, no one knows that the problem is, and as such, 
> >>wont be able to fix it.
> >>
> >>-Jim
> >
> >
> 
> 
> 
> Looks like your users send/receive a lot of HTML mail. I had to adjust 
> the rules for those down slightly to help reduce the possibility of FP's.
> 
> Here, I don't care if 'chain mail' is marked as spam -- that is not 
> legitimate mail for our users, tho, my system doesn't delete up to a 
> certain threshold.
> Your second example had this (watch for line wraps):
> 
> [...]
> 
> X-Spam-Flag: NO
> 
> X-Scanned-By: milter-7bit/0.7.101 (doctor.nl2k.ab.ca [204.209.81.1]); 
> Wed, 27 Jul 2005 11:24:55 -0600
> 
> X-Scanned-By: milter-date/0.12.160 (doctor.nl2k.ab.ca [204.209.81.1]); 
> Wed, 27 Jul 2005 11:24:55 -0600
> 
> X-Scanned-By: milter-spamc/0.25.321 (doctor.nl2k.ab.ca [204.209.81.1]); 
> Wed, 27 Jul 2005 11:24:33 -0600
> 
> X-Spam-Status: NO, hits=2.20 required=5.00
> 
> X-Spam-Level: xx
> 
> X-Mark-SPAM: YES, score=5.40/5.00, processed for 2.536s on doctor.nl2k.ab.ca
> 
> [...]
> 
> What looks odd to me is that X-Spam-Status says NO (I'm assuming that 
> this comes from sa), level is only 2, but X-Mark-Spam: is yes.. with a 
> score of 5.40.. where is this coming from?
>

I am using milter-spamc and smf-spamd .

 
> 
> -- 
> Thanks,
> James
> 

-- 
Member - Liberal International  
This is [EMAIL PROTECTED]   Ici [EMAIL PROTECTED]
God Queen and country! Beware Anti-Christ rising!
Better to serve in Heaven that to Rule in Hell.


RE: [FW: spam control

2005-07-27 Thread Chris Santerre

> score gtube   4.0
> score razor2_check4
> score RAZOR2_CF_RANGE_11_50   4
> score RAZOR2_CF_RANGE_51_100  4
> score DCC_CHECK   5
> score PYZOR_CHECK 5
> score REMOVE_IN_QUOTES4
> score CLICK_TO_REMOVE_2   4
> score ASCII_FORM_ENTRY4
> score TRACKER_ID  4

*snip*

Holy carp!!! Why did you rescore just about every rule higher? Those
rules are bound to cause FPs. Scored waaay too high.

Doc, right now I would remove all traces of SA, and start over. There seems
to be issues just about everywhere. 

Setup SA fresh, and callit ONLY for a test account. (Like your own.)

--Chris 


Re: [SPAM: score=6.0/5.0] Re: [FW: spam control

2005-07-27 Thread The Doctor
On Wed, Jul 27, 2005 at 03:13:41PM -0500, Andy Jezierski wrote:
> The Doctor <[EMAIL PROTECTED]> wrote on 07/27/2005 02:34:42 PM:
> 
> > - Forwarded message from Angry and Concerned Customer -
> > 
> > X-Scanned-By: milter-spamc/0.25.321 (localhost.nl2k.ab.ca [0.0.0.
> > 0]); Wed, 27 Jul 2005 13:11:47 -0600
> >   
> 
> [snip]
> 
> > 
> > 
> > All right, the short and simple is that Spam-Assassin may not be doing
> > the correct job.  This user has a whitelist in place and
> > some e-mail are getting the label of spam.
> > 
> > Even some of my cron jobs are getting  a [SPAM] label when they should 
> nt.
> > 
> > Why?
> > 
> 
> As everyone has said, we need to see the message headers at a minimum in 
> order to try and help.  Also, judging from the X-Scanned-By: line above I 
> assume you're using milter-spamc to call SA.  If you'd like you can add a 
> few lines to your sendmail access file to bypass SA for individual 
> senders/recipents.
> 
> Milter-Spamc-From:[EMAIL PROTECTED]OK
> Milter-Spamc-To:[EMAIL PROTECTED]   OK
>  
>

Will try.
 
> Andy
-- 
Member - Liberal International  
This is [EMAIL PROTECTED]   Ici [EMAIL PROTECTED]
God Queen and country! Beware Anti-Christ rising!
Better to serve in Heaven that to Rule in Hell.


RE: [FW: spam control

2005-07-27 Thread Chris Santerre

> > Also, post the whitelist entry you're using... And what 
> file it's in, and how
> > you're calling SA.
> 
> Whitelist from /.spamassassin/user_prefs:
> 
> 

NEVER post other peoples' email addresses to a public and archived list!!! 

Deep breaths Doc!

--Chris


Re: [FW: spam control

2005-07-27 Thread The Doctor
On Wed, Jul 27, 2005 at 04:02:32PM -0400, Ron Johnson wrote:
> The Doctor writes:
> > 
> > - Forwarded message from Angry and Concerned Customer -
> > 
> > 
> > 
> > All right, the short and simple is that Spam-Assassin may not be doing
> > the correct job.  This user has a whitelist in place and
> > some e-mail are getting the label of spam.
> > 
> > Even some of my cron jobs are getting  a [SPAM] label when they should nt.
> > 
> > Why?
> 
> What version are you running? Are you running any additional rulesets?
> Have you written any custom rules yourself? Do you have bayes enabled?
> If so, are you running with autolearn? Do you have AWL enabled? (If so,
> you may want to start over)
> 
> You need to find out what rules your false positives are tripping over.
> 
> I personally find it convenient to run the false positives manually
> (though that's really not required)
> 
> 


I am running 3.0.4 on BSD/OS 4.3.1 .

Here is my local.cf:


# Add your own customisations to this file.  See 'man Mail::SpamAssassin::Conf'
# SpamAssassin user preferences file.
#
# Format:
#
#   required_hits n
#   (how many hits are required to tag a mail as spam.)
#
#   score SYMBOLIC_TEST_NAME n
#   (if this is omitted, 1 is used as a default score.
#   Set the score to 0 to ignore the test.)
#
# # starts a comment, whitespace is not significant.
#
# NOTE!  In conjunction with MIMEDefang, SpamAssassin can *NOT* make any
# changes to the message header or body.  Any SpamAssassin settings that
# relate to changing the message will have *NO EFFECT* when used from
# MIMEDefang.  Instead, use the various MIMEDefang Perl functions if you
# need to alter the message.
###

###
# First of all, the generally useful stuff; thresholds and the whitelist
# of addresses which, for some reason or another, often trigger false
# positives.

required_hits   7.5

# Whitelist and blacklist addresses are *not* patterns; they're just normal
# strings.  one exception is that "[EMAIL PROTECTED]" is allowed.  They should 
be in
# lower-case.  You can either add multiple addrs on one line,
# whitespace-separated, or you can use multiple lines.
#
# Monty Solomon: he posts from an ISP that has often been the source of spam
# (no fault of his own ;), and sometimes uses Bcc: when mailing.
#
# whitelist_from[EMAIL PROTECTED]

# Add your blacklist entries in the same format...
#
# blacklist_from[EMAIL PROTECTED]

# Mail using languages used in these country codes will not be marked
# as being possibly spam in a foreign language.
#
##ok_localesen

# By default, the subject lines of suspected spam will be tagged.
# This can be disabled here.
#
##rewrite_subject 0
# By default, spamassassin will include its report in the body
# of suspected spam. Enabling this causes the report to go in the
# headers instead. Using 'use_terse_report' for this is recommended.
#
# report_header 1

# By default, SpamAssassin uses a fairly long report format.
# Enabling this uses a shorter format which includes all the
# information in the normal one, but without the superfluous
# explanations.
#
# use_terse_report 0

# By default, spamassassin will change the Content-type: header of
# suspected spam to "text/plain". This is a safety feature. If you
# prefer to leave the Content-type header alone, set this to 0.
#
defang_mime 0

# By default, SpamAssassin will run RBL checks.  If your ISP already
# does this, set this to 1.

#skip_rbl_checks 1

###
# Add your own customised scores for some tests below.  The default scores are
# read from the installed "spamassassin.cf" file, but you can override them
# here.  To see the list of tests and their default scores, go to
# http://spamassassin.taint.org/tests.html .

# for details of what can be tweaked.
#

# SpamAssassin config file for version 2.5x
# generated by http://www.yrex.com/spam/spamconfig.php (version 1.01)

# How many hits before a message is considered spam.
required_hits   7.5

# Whether to change the subject of suspected spam
##rewrite_subject 1

# Text to prepend to subject if rewrite_subject is used
##subject_tag *SPAM*
rewrite_header Subject SPAM(_SCORE_)   

# Encapsulate spam in an attachment
report_safe 1

# Use terse version of the spam report
use_terse_report0

# Enable the Bayes system
use_bayes   1

# Enable Bayes auto-learning
auto_learn  1

# Enable or disable network checks
skip_rbl_checks 0
use_razor2  1
use_dcc 1
use_pyzor   1

# Mail using languages used in these country codes will not be marked
# as being possibly spam in a foreign language.
ok_languagesall

# Mai

Re: [FW: spam control

2005-07-27 Thread JamesDR

The Doctor wrote:

On Wed, Jul 27, 2005 at 03:48:22PM -0400, Jim Maul wrote:


The Doctor wrote:





All right, the short and simple is that Spam-Assassin may not be doing
the correct job.  This user has a whitelist in place and
some e-mail are getting the label of spam.

Even some of my cron jobs are getting  a [SPAM] label when they should nt.

Why?




Perhaps if you posted the headers of the messages that were marked as 
spam we can look to see what rules hit which would answer your "why?" 
question.  Until then, no one knows that the problem is, and as such, 
wont be able to fix it.


-Jim







Looks like your users send/receive a lot of HTML mail. I had to adjust 
the rules for those down slightly to help reduce the possibility of FP's.


Here, I don't care if 'chain mail' is marked as spam -- that is not 
legitimate mail for our users, tho, my system doesn't delete up to a 
certain threshold.

Your second example had this (watch for line wraps):

[...]

X-Spam-Flag: NO

X-Scanned-By: milter-7bit/0.7.101 (doctor.nl2k.ab.ca [204.209.81.1]); 
Wed, 27 Jul 2005 11:24:55 -0600


X-Scanned-By: milter-date/0.12.160 (doctor.nl2k.ab.ca [204.209.81.1]); 
Wed, 27 Jul 2005 11:24:55 -0600


X-Scanned-By: milter-spamc/0.25.321 (doctor.nl2k.ab.ca [204.209.81.1]); 
Wed, 27 Jul 2005 11:24:33 -0600


X-Spam-Status: NO, hits=2.20 required=5.00

X-Spam-Level: xx

X-Mark-SPAM: YES, score=5.40/5.00, processed for 2.536s on doctor.nl2k.ab.ca

[...]

What looks odd to me is that X-Spam-Status says NO (I'm assuming that 
this comes from sa), level is only 2, but X-Mark-Spam: is yes.. with a 
score of 5.40.. where is this coming from?



--
Thanks,
James



RE: [FW: spam control

2005-07-27 Thread Chris Santerre
OK something is wrong with your setup!

> 
> Sample 1 from a cron job:
> 
> Subject: [SPAM] Cron <[EMAIL PROTECTED]> /usr/bin/nice -20 
> X-Spam-Flag: NO
> --

Marked as spam but not?


> 
> Sample 2
> 
> Headers:
> 
> Subject: [SPAM: score=5.4/5.0] spam control and assorted issues
> 
> X-Spam-Flag: NO
> X-Spam-Status: NO, hits=2.20 required=5.00
> X-Spam-Level: xx
> X-Mark-SPAM: YES, score=5.40/5.00, processed for 2.536s on 
> doctor.nl2k.ab.ca
> 
>   ---

Same! Is it being run thru twice? 

> 
> Sample 3
> 
> 
> X-Spam-Filter: [EMAIL PROTECTED] by digitalanswers.org

> Subject: [SPAM: score=10.0/5.0] [SPAM] (5.00/5.00) Great 
> Canadian website
> 
> X-Spam-Flag: YES
> 
> X-Spam-Status: YES, hits=5.00 required=5.00
> 
> X-Spam-Level: x
> 
> X-Spam-Report: Spam detection software, running on the system 
> "doctor.nl2k.ab.ca", has
> 

>   Content analysis details:   (5.0 points, 5.0 required)
>pts rule name  description
> 
>    -- 
> --
> 
>3.0 HTML_MESSAGE   BODY: HTML included in message
> 
>0.0 BAYES_50   BODY: Bayesian spam probability 
> is 40 to 60%
> 
>   [score: 0.4039]
> 
>2.0 HTML_10_20 BODY: Message is 10% to 20% HTML
> 
>   
> 
> X-Mark-SPAM: YES, score=10.00/5.00, processed for 2.167s on 
> doctor.nl2k.ab.ca

3.0 points for an HTML messege That can't be right!


> 
> 
> 
> Sample 4:
> 
> 

> 
> Subject: [SPAM: score=11.0/5.0] [SPAM] (13.80/5.00) Re: 
> [SPAM] (16.50/5.00) Fwd: Fw: 9 Things I Hate About Everyone

> 
> Subject: [SPAM] (16.50/5.00) Fwd: Fw: 9 Things I Hate 

> 
>   Content analysis details:   (13.8 points, 5.0 required)
> 
>   
> 
>pts rule name  description
> 
>    -- 
> --
> 
>4.0 MAILTO_TO_SPAM_ADDRURI: Includes a link to a 
> likely spammer email
> 
>3.0 HTML_MESSAGE   BODY: HTML included in message
> 
>0.0 BAYES_50   BODY: Bayesian spam probability 
> is 40 to 60%
> 
>   [score: 0.5585]
> 
>4.0 HTML_70_80 BODY: Message is 70% to 80% HTML
> 
>2.7 AWLAWL: From: address is in the 
> auto white-list
> 
>   
> 
> X-Mark-SPAM: YES, score=11.00/5.00, processed for 25.087s on 
> doctor.nl2k.ab.ca


Sample 4 has scores all over the place!!  11.00, 13.8, and 16.5!! It went
thru 3 times!!

Shut off AWL for now! Fix the 3 point score for HTML. Then figure out why
your getting multiple scans!

--Chris 


Re: [FW: spam control

2005-07-27 Thread The Doctor
On Wed, Jul 27, 2005 at 03:48:22PM -0400, Jim Maul wrote:
> The Doctor wrote:
> 
> 
> >
> >
> >All right, the short and simple is that Spam-Assassin may not be doing
> >the correct job.  This user has a whitelist in place and
> >some e-mail are getting the label of spam.
> >
> >Even some of my cron jobs are getting  a [SPAM] label when they should nt.
> >
> >Why?
> >
> 
> 
> Perhaps if you posted the headers of the messages that were marked as 
> spam we can look to see what rules hit which would answer your "why?" 
> question.  Until then, no one knows that the problem is, and as such, 
> wont be able to fix it.
> 
> -Jim

Sample 1 from a cron job:

---

From [EMAIL PROTECTED] Wed Jul 27 13:19:15 2005
Return-Path: <[EMAIL PROTECTED]>
Received: from doctor.nl2k.ab.ca ([EMAIL PROTECTED] [127.0.0.1])
by doctor.nl2k.ab.ca (8.13.4/8.13.4) with ESMTP id j6RJJ6BM011313
for <[EMAIL PROTECTED]>; Wed, 27 Jul 2005 13:19:06 -0600 (MDT)
Authentication-Results: doctor.nl2k.ab.ca [EMAIL PROTECTED]; sender-id=neutral; 
spf=neutral
X-SenderID: Sendmail Sender-ID Filter v0.2.8 doctor.nl2k.ab.ca j6RJJ6BM011313
X-Spam-Filter: [EMAIL PROTECTED] by digitalanswers.org
Received: (from [EMAIL PROTECTED])
by doctor.nl2k.ab.ca (8.13.4/8.13.4/Submit) id j6RJJ31O011310;
Wed, 27 Jul 2005 13:19:03 -0600 (MDT)
Date: Wed, 27 Jul 2005 13:19:03 -0600 (MDT)
Message-Id: <[EMAIL PROTECTED]>
From: [EMAIL PROTECTED] (Cron Daemon)
To: [EMAIL PROTECTED]
Subject: [SPAM] Cron <[EMAIL PROTECTED]> /usr/bin/nice -20 
/usr/home/cariwest/html/analog/analog
X-Cron-Env: 
X-Cron-Env: 
X-Cron-Env: 
X-Cron-Env: 
X-Cron-Env: 
X-Virus-Scanned: ClamAV version 0.86.2, clamav-milter version 0.86 on 
doctor.nl2k.ab.ca
X-Virus-Status: Clean
X-Spam-Flag: NO
X-Scanned-By: milter-7bit/0.7.101 (localhost.nl2k.ab.ca [0.0.0.0]); Wed, 27 Jul 
2005 13:19:12 -0600
X-Scanned-By: milter-date/0.12.160 (localhost.nl2k.ab.ca [0.0.0.0]); Wed, 27 
Jul 2005 13:19:12 -0600
X-Scanned-By: milter-spamc/0.25.321 (localhost.nl2k.ab.ca [0.0.0.0]); Wed, 27 
Jul 2005 13:19:12 -0600
X-Spam-Status: NO, hits=-105.70 required=5.00
X-Spam-Level: 
X-milter-date-PASS: YES
X-milter-7bit-Report: error=7bit octet=0x80 offset=74 line=2 position=11
X-milter-7bit-Pass: NO
Status: RO
Content-Length: 718
Lines: 13

/usr/home/cariwest/html/analog/analog: analog version 6.0/Unix
: Warning €: Turning off empty Virtual Host Report
  (For help on all errors and warnings, see docs/errors.html)
: Warning €: Turning off empty Virtual Host Redirection Report
: Warning €: Turning off empty Virtual Host Failure Report
: Warning €: Turning off empty User Report
: Warning €: Turning off empty User Redirection Report
: Warning €: Turning off empty User Failure Report
&meta=: Warning €: Turning off empty Internal Search Query Report
&meta=: Warning €: Turning off empty Internal Search Word Report
: Warning €: Turning off empty Processing Time Report
: Warning : In Redirected Referrer Report, turning off pie chart of only one
  wedge

--

Sample 2

Headers:

---

Subject: [SPAM: score=5.4/5.0] spam control and assorted issues

Date: Wed, 27 Jul 2005 11:28:31 -0600

Message-ID: <[EMAIL PROTECTED]>

MIME-Version: 1.0

Content-Type: text/plain;

charset="iso-8859-1"

Content-Transfer-Encoding: 7bit

X-Priority: 3 (Normal)

X-MSMail-Priority: Normal

X-Mailer: Microsoft Outlook CWS, Build 9.0.6604 (9.0.2911.0)

Importance: Normal

X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1409

X-Virus-Scanned: ClamAV version 0.86.2, clamav-milter version 0.86 on 
doctor.nl2k.ab.ca

X-Virus-Status: Clean

X-Spam-Flag: NO

X-Scanned-By: milter-7bit/0.7.101 (doctor.nl2k.ab.ca [204.209.81.1]); Wed, 27 
Jul 2005 11:24:55 -0600

X-Scanned-By: milter-date/0.12.160 (doctor.nl2k.ab.ca [204.209.81.1]); Wed, 27 
Jul 2005 11:24:55 -0600

X-Scanned-By: milter-spamc/0.25.321 (doctor.nl2k.ab.ca [204.209.81.1]); Wed, 27 
Jul 2005 11:24:33 -0600

X-Spam-Status: NO, hits=2.20 required=5.00

X-Spam-Level: xx

X-Mark-SPAM: YES, score=5.40/5.00, processed for 2.536s on doctor.nl2k.ab.ca

X-milter-date-PASS: YES

X-milter-7bit-Pass: YES

X-UIDL: efU!!CF,"!([EMAIL PROTECTED]"!

  ---


Sample 3

Return-Path: <[EMAIL PROTECTED]>

Received: from web31112.mail.mud.yahoo.com (web31112.mail.mud.yahoo.com 
[68.142.201.74])

by doctor.nl2k.ab.ca (8.13.4/8.13.4) with SMTP id j6RITs3q002842

for <[EMAIL PROTECTED]>; Wed, 27 Jul 2005 12:29:55 -0600 (MDT)

Authentication-Results: doctor.nl2k.ab.ca [EMAIL PROTECTED]; sender-id=neutral; 
spf=neutral

X-SenderID: Sendmail Sender-ID Filter v0.2.8 doctor.nl2k.ab.ca j6RITs3q002842

X-Spam-Filter: [EMAIL PROTECTED] by digitalanswers.org

Received: (qmail 2762 invoked by uid 60001); 27 Jul 2005 18:29:48 -

DomainKey-Signature:

Re: [FW: spam control

2005-07-27 Thread Andy Jezierski

The Doctor <[EMAIL PROTECTED]> wrote
on 07/27/2005 02:34:42 PM:

> - Forwarded message from Angry and Concerned Customer -
> 
> X-Scanned-By: milter-spamc/0.25.321 (localhost.nl2k.ab.ca [0.0.0.
> 0]); Wed, 27 Jul 2005 13:11:47 -0600
>   

[snip]

> 
> 
> All right, the short and simple is that Spam-Assassin may not be doing
> the correct job.  This user has a whitelist in place and
> some e-mail are getting the label of spam.
> 
> Even some of my cron jobs are getting  a [SPAM] label when they
should nt.
> 
> Why?
> 

As everyone has said, we need to see the message headers
at a minimum in order to try and help.  Also, judging from the X-Scanned-By:
line above I assume you're using milter-spamc to call SA.  If you'd
like you can add a few lines to your sendmail access file to bypass SA
for individual senders/recipents.

Milter-Spamc-From:[EMAIL PROTECTED]    OK
Milter-Spamc-To:[EMAIL PROTECTED]   OK
  

Andy

Re: [FW: spam control

2005-07-27 Thread Ron Johnson
The Doctor writes:
> 
> - Forwarded message from Angry and Concerned Customer -
> 
> 
> 
> All right, the short and simple is that Spam-Assassin may not be doing
> the correct job.  This user has a whitelist in place and
> some e-mail are getting the label of spam.
> 
> Even some of my cron jobs are getting  a [SPAM] label when they should nt.
> 
> Why?

What version are you running? Are you running any additional rulesets?
Have you written any custom rules yourself? Do you have bayes enabled?
If so, are you running with autolearn? Do you have AWL enabled? (If so,
you may want to start over)

You need to find out what rules your false positives are tripping over.

I personally find it convenient to run the false positives manually
(though that's really not required)




Re: [FW: spam control

2005-07-27 Thread Matt Kettler
Jim Maul wrote:
> The Doctor wrote:
> 
> 
> 
>>
>>
>> All right, the short and simple is that Spam-Assassin may not be doing
>> the correct job.  This user has a whitelist in place and
>> some e-mail are getting the label of spam.
>>
>> Even some of my cron jobs are getting  a [SPAM] label when they should
>> nt.
>>
>> Why?
>>
> 
> 
> Perhaps if you posted the headers of the messages that were marked as
> spam we can look to see what rules hit which would answer your "why?"
> question.  Until then, no one knows that the problem is, and as such,
> wont be able to fix it.
> 

Also, post the whitelist entry you're using... And what file it's in, and how
you're calling SA.

Some general notes about this kind of problem:

Bear in mind that "whitelist_to", "more_spam_to" and "all_spam_to" commands will
generally NOT match any messages which are BCC'ed to you, including mailing
lists. SpamAssassin will *try* to find out who the BCC'ed recipient is, but if
your MTA doesn't add any headers indicative of this, there's no way for
spamassassin to determine the message recipient.

If you have users that really do not want their mail marked as spam, its is a
much better solution to avoid calling SA in the first place for those users. You
save CPU time, and most tools that call SA have direct access to the message
envelope and can make decisions based on the real recipient. For example, if you
call spamc from procmail, you can write a procmail rule that bypasses spamc for
some users.

Also bear in mind that if you're using a user's user_prefs file to declare a
whitelist, you MUST be calling spamc as that user, or passing their username to
spamc -u. Most site-wide installs will only read the user_prefs for one account
(usually mail, root, or nobody), and will NOT read the user_prefs file in the
recipient's home directory.








Re: [FW: spam control

2005-07-27 Thread Jim Maul

The Doctor wrote:





All right, the short and simple is that Spam-Assassin may not be doing
the correct job.  This user has a whitelist in place and
some e-mail are getting the label of spam.

Even some of my cron jobs are getting  a [SPAM] label when they should nt.

Why?




Perhaps if you posted the headers of the messages that were marked as 
spam we can look to see what rules hit which would answer your "why?" 
question.  Until then, no one knows that the problem is, and as such, 
wont be able to fix it.


-Jim


[FW: spam control

2005-07-27 Thread The Doctor
- Forwarded message from Angry and Concerned Customer -

X-Scanned-By: milter-spamc/0.25.321 (localhost.nl2k.ab.ca [0.0.0.0]); Wed, 27 
Jul 2005 13:11:47 -0600
  

Hi Dave - we are still getting people labeled as sending us spam that should
be on that white list (this includes emails from employees).  The last two
were addressed to me from Rhonda and one from Jim Wooley - both were labeled
as spam!

This is nuts!  If it doesn't work - it doesn't work!

Also can we raise the threshold on the spam to 7.5 instead of 5.00 (7.5 >
and it is labeled spam)

Really for us - we would rather not have anything labeled as spam AT ALL.
this would fix most of this issue.  Then the only issue would be making sure
your Spam filters (Spam Assassin) pass all legitimate emails through to us
(even if some spam slipped through with it - we would rather not miss
anything).

Our issues are major to us - and it seems we have a number of them, so I am
going to go over them here again so we don't lose sight of them:


3) Email issues.  Spam.  We didn't ask for our emails to be labeled with
"spam" and it is creating problems for us.  This creates certain issues
within the organization when we "accidentally" reply to a member (not
noticing anymore the spam label - since every email seems to have it) and
they get an email from us with "spam" marked in it 

Also, a number of members at one time or another could not send email
through to us.  I haven't heard of any lately, but that was why we went to a
"whitelist" approach - to ensure that people on that whitelist were allowed
through - regardless of spam filtering and that their emails would not be
labeled spam.  (Note I am saying spam filtering - not the standard antivirus
checking).  Well, it's been a couple months now and the Whitelist doesn't
seem to be working as it should/intended and there is also been no way to
update that white list (replace the file of "acceptable" email addresses
with updated ones or add people to it).

- End forwarded message -


All right, the short and simple is that Spam-Assassin may not be doing
the correct job.  This user has a whitelist in place and
some e-mail are getting the label of spam.

Even some of my cron jobs are getting  a [SPAM] label when they should nt.

Why?

-- 
Member - Liberal International  
This is [EMAIL PROTECTED]   Ici [EMAIL PROTECTED]
God Queen and country! Beware Anti-Christ rising!
Better to serve in Heaven that to Rule in Hell.