Re: Webmail and IP rules

[List Mail User]

>>From [EMAIL PROTECTED] Wed Mar  2 15:01:17 2005
>Mailing-List: contact [EMAIL PROTECTED]; run by ezmlm
>...
>Delivered-To: mailing list users@spamassassin.apache.org
>...
>
>I think the problem is being caused by IMP being "too good" at
>generating a Received header that looks like a normal one added
>by an MTA.  Good enough to fool SpamAssassin into thinking it's
>an SMTP one, anyway. ;)
>
>Could someone open a bug about this?  we may indeed be able to
>look for the "with HTTP" and ignore that.

Of course, that would leave a vulnerability to "formmail.pl" exploits
on misconfigured web servers being used as first hop proxies.  I think the OP
should fix the IMP installation (ar add rules for the cases of both 127.0.0.1
and the RFC1918 leakage which I seem to remember in the original example - i.e.
192.168.x.x).  When properly configured, the "-notfirsthop" qualifier should do
all that is needed.

>
>- --j.
>
>Shane Williams writes:
>> ...

Paul Shupak
[EMAIL PROTECTED]

Re: Webmail and IP rules

[Justin Mason]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1


I think the problem is being caused by IMP being "too good" at
generating a Received header that looks like a normal one added
by an MTA.  Good enough to fool SpamAssassin into thinking it's
an SMTP one, anyway. ;)

Could someone open a bug about this?  we may indeed be able to
look for the "with HTTP" and ignore that.

- --j.

Shane Williams writes:
> I noticed the HELO_DYNAMIC_* thread and the conclusion that IMP adding
> a Received header may be a source of problems.  I pieced together the
> same conclusion just this morning based on several false positives
> that went through our campus' IMP-based webmail.  In addition to
> the several variations of HELO_DYNAMIC_*, I also saw one which hit an
> SPF rule (since it didn't get relayed through the "official" relay.
> 
> My first question, for anyone who knows the relavent RFCs better than
> I, is IMP's behavior of adding a Received header following specs?
> 
> Second, has anyone determined the best way to handle this?  The two
> options that immediately come to mind would be to turn off the
> HELO_DYNAMIC_* rules (but I suspect this would cause more false
> negatives), or create a score-lowering rule that fires when a
> webmail/IMP header is detected (also problematic since a webmail
> header isn't necessarily related to the spamminess of the email, only
> to the likely existence of false triggers on other rules).
> 
> Alternately, is this something that spammassassin should be taking
> into account in its analysis?  That is, when SA sees a "with HTTP"
> descriptor in a received header, it should just ignore that header
> altogether (or ignore it in relation to certain rules).
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Exmh CVS

iD8DBQFCJkV0MJF5cimLx9ARAh5pAJ9RTEcXz46ABrVa40PXEmzuVFIMHgCfSLiO
HADKznPdV4nuEeRy3pVcLB8=
=jNdY
-END PGP SIGNATURE-

Re: spamassasin global bayes database

[Zaine]

Hello,

Edit the $HOME/.spamassassin/userprefs file and add :
bayes_path $PATH to db

Then also add the same bayes_path into the local.cf file.

Regards
Zaine

On Wednesday 02 March 2005 22:16, Matt wrote:
> What do I have to do to get spamassassin to use a global bayes
> database for all users on the system, rather then per user?

Webmail and IP rules

[Shane Williams]

I noticed the HELO_DYNAMIC_* thread and the conclusion that IMP adding
a Received header may be a source of problems.  I pieced together the
same conclusion just this morning based on several false positives
that went through our campus' IMP-based webmail.  In addition to
the several variations of HELO_DYNAMIC_*, I also saw one which hit an
SPF rule (since it didn't get relayed through the "official" relay.
My first question, for anyone who knows the relavent RFCs better than
I, is IMP's behavior of adding a Received header following specs?
Second, has anyone determined the best way to handle this?  The two
options that immediately come to mind would be to turn off the
HELO_DYNAMIC_* rules (but I suspect this would cause more false
negatives), or create a score-lowering rule that fires when a
webmail/IMP header is detected (also problematic since a webmail
header isn't necessarily related to the spamminess of the email, only
to the likely existence of false triggers on other rules).
Alternately, is this something that spammassassin should be taking
into account in its analysis?  That is, when SA sees a "with HTTP"
descriptor in a received header, it should just ignore that header
altogether (or ignore it in relation to certain rules).
--
Public key #7BBC68D9 at| Shane Williams
http://pgp.mit.edu/|  System Admin - UT iSchool
=--+---
All syllogisms contain three lines |  [EMAIL PROTECTED]
Therefore this is not a syllogism  | www.ischool.utexas.edu/~shanew

Re: Suggestion: OCR

[JamesDR]

I like the idea, however, I can see this adding quite a bit of time to 
the scan on large images.  (I've never used gocr so as far as I can 
tell, i compare it to other ocr products I've used and they were all 
pretty slow.) I had the problem you described, mails getting just image 
spams, what I did was hand modify the rules that did hit, I set them 
just a bit higher till it hit the limit that marked it as spam, while 
making sure ham messages (why someone embeds images in their mails is 
beyond me.. but it does happen.)

My 2 cents
Thanks,
JamesDR
[EMAIL PROTECTED] wrote:
 --- On Wed 03/02, Matt Kettler < [EMAIL PROTECTED] > wrote:

That part is definitely NOT safe in the context of spamassassin... Nonsense 
looks a lot like bugs in spam mailers, and very little like legitimate email to 
SA.

If nothing else, consider the tripwire rules, which look for letter 

combinations that don't exist in normal English...
---

Thanks! If so, then it's a bit more work to implement. For example, a trivial 
idea is not to let the attachments, which stem from images, go through the 
rules that search for nonsense.

I meant 'safe' in the following sense: if the tool says some meaningful word (e.g. 
present in the english wordlist up to a small misspell), then this word is surely present 
in the image up to a small misspell. So, if some spam rule sees "viagra" or 
'click here to get removed' after OCRing, then it is 'safe' to give a hit for it, for 
example.

Another work-intensive method could be as follows (corrections are welcome)
1. OCR.
2. Throw out all the words which are not in the english (german, russian, 
etc...) dictionary up to a misspell. E.g. tolerate at most one error per word. 
Correct the misspelled words. (Fast dictionary search required, e.g. represent 
wordlists as binary balanced trees.)
3. run other text-based rules.

Actually, I posted because I get too much image spam (which goes ok through SA) 
and tried to determine the possibility of catching it with the present tools. 
Sometimes I get photos and image-smileys so I'm very reluctant to stop all 
mails containing images without inspecting images.

My strong belief is that such tools as gocr can really help. The other question 
is how to integrate it in SA and who does it. I'm afraid I cannot dig into the 
SA code myself; so it's a suggestion to the advanced users and developers.

Regards,
sasha.
___
Join Excite! - http://www.excite.com
The most personalized portal on the Web!


smime.p7s
Description: S/MIME Cryptographic Signature

Re: Typical spam not detected at all.. there is no rule for it :-\

[Stuart Johnston]

Duncan Hill wrote:
works like a charm, and RM should be adding a SARE rule that catches that 
email address at the bottom.  The numbers change, but the prefix is good, as 
is the suffix.
Here's mine:
body L_STOX /stox\d{4}\s{0,[EMAIL PROTECTED],4}yahoo.com/

RE: Typical spam not detected at all.. there is no rule for it :- \

[List Mail User]

Chris,

I know you don't like bayes, but it is the best single tool for stock
scams.  The trouble with counting '|' is the frequency of transcribed spead-
sheets would give too many FPs (typical is to use '|' to separate the columns).
Most scock scams use non-obfucated words to look legitimate and things like
"price", "soar", "undervalued", "goal", "target price", "due di|igence", and
"stock", etc. get well recognized by the bayes analyzer.

Paul Shupak
[EMAIL PROTECTED]

Re: Why was this message not tagged?

[Matt Kettler]

At 04:23 PM 3/2/2005, Jim Maul wrote:
First, why doubt Matt? :)
I could write at least a 1000 page novel of good reasons to doubt me :)
Secondly, what would the second parameter be?  If the first is the 
required hits, the second number would be?
That's a lot more sensible..

RE: Why was this message not tagged?

[Jon Dossey]

> At 01:08 PM 3/2/2005, you wrote:
> 
> >Can anyone confirm that it should only have one parameter?
> 
> Yup, the manual can.. :)
> 
>
http://spamassassin.apache.org/full/3.0.x/dist/doc/Mail_SpamAssassin_Con
f.
> html
> 
> >   I can't image why it would have two?
> 
> A mistake when someone (not to name names) was editing it perhaps? ;)

Well I haven't edited it since it was installed, so I guess I'll just
have to take this up with the other admins :)

Still very odd that some messages are tagged, and some aren't.  Once the
configuration is parsed by spamassassin, god only knows how a mistake
like that would end up assigned to some int somewhere.  

I'll take it out and see how it goes.  Thanks guys.

.jon

Re: Typical spam not detected at all.. there is no rule for it :-\

[List Mail User]

Marian,

For these stock scams, bayes is your friend; Parsing it locally I get

Content analysis details:   (3.2 points, 5.0 required)

 pts rule name  description
 -- --
 0.1 MISSING_HEADERSMissing To: header
 1.9 BAYES_99   BODY: Bayesian spam probability is 99 to 100%
[score: 1.]
 1.2 MISSING_SUBJECTMissing Subject: header

Which still isn't enough to trigger, but the message has the email
address of"stox0033@  yahoo.com",  which someone (I forget who) pointed out
earlier this week or last that they were seeing all stock scams with email
of the form "[EMAIL PROTECTED]".  Bayes training together with a rule for that
email pattern in the body, you should catch them -- They key for me is that
the stock scams seldom have enough header points to "autolearn" for bayes
and must be hand trained;  Still they all have so many common tokens, that
the BAYES_99 I see is quite different than the BAYES_OO you see.  Also,
quite a few people have mentioned that they have increased the score for
BAYES_99; I use the stock install myself with only a set of custom URI rules.

A good regexp for the email would probably be something like STOX_YAHOO \
[EMAIL PROTECTED] and a point count of only 1.8 or 1.9  would cause a trigger.
Note: This clearly would/should be a temporary rule as it is unlikely the
spammer would be so dumb as to continue the pattern.  Also, I think Rules du
Jour and SARE have some stock rules which I don't use that would catch these
even better. (I'm sure someone else on the this list knows better than me.)

Good luck,

Paul Shupak
[EMAIL PROTECTED]

P.S.  I didn't check the validity of the email as I usually do, but if valid,
mail to Yahoo! with a copy of the spam will cause the account to be revoked,

Re: Why was this message not tagged?

[Jim Maul]

Jon Dossey wrote:
At 11:25 AM 3/2/2005, Jon Dossey wrote:
I apologize, I was in a rush.  System is redhat fc2, sendmail 8.13.1,
spamassassin 3.0.1 and spamass-milter 0.2.0 (updated for SA 3.0,
haven't
switched to 0.3.0 yet).
Here's (most of) my /home/spamd/.spamassassin/user_prefs:

# How many points before a mail is considered spam.
required_hits   5.0 4.5
Erm.. that's not right... required_hits should only have one
parameter...
You're right, that does look odd.  Can anyone confirm that it should
only have one parameter?  I can't image why it would have two?

Of course it only takes 1 parameter.  First, why doubt Matt? :) 
Secondly, what would the second parameter be?  If the first is the 
required hits, the second number would be?

-Jim

RE: Why was this message not tagged?

[Evan Platt]

At 01:08 PM 3/2/2005, you wrote:
Can anyone confirm that it should only have one parameter?
Yup, the manual can.. :)
http://spamassassin.apache.org/full/3.0.x/dist/doc/Mail_SpamAssassin_Conf.html
  I can't image why it would have two?
A mistake when someone (not to name names) was editing it perhaps? ;)

RE: Why was this message not tagged?

[Jon Dossey]

> At 11:25 AM 3/2/2005, Jon Dossey wrote:
> >I apologize, I was in a rush.  System is redhat fc2, sendmail 8.13.1,
> >spamassassin 3.0.1 and spamass-milter 0.2.0 (updated for SA 3.0,
haven't
> >switched to 0.3.0 yet).
> >
> >Here's (most of) my /home/spamd/.spamassassin/user_prefs:
> 
> 
> 
> ># How many points before a mail is considered spam.
> >required_hits   5.0 4.5
> 
> Erm.. that's not right... required_hits should only have one
parameter...
> 

You're right, that does look odd.  Can anyone confirm that it should
only have one parameter?  I can't image why it would have two?

Thanks,
jon

Re: spamassasin global bayes database

[Matias Lopez Bergero]

Matt wrote:
What do I have to do to get spamassassin to use a global bayes
database for all users on the system, rather then per user?
read the wiki :)
http://wiki.apache.org/spamassassin/SiteWideBayesSetup

Re: spamassasin global bayes database

[Steven Dickenson]

Matt wrote:
What do I have to do to get spamassassin to use a global bayes
database for all users on the system, rather then per user?
http://wiki.apache.org/spamassassin/SiteWideBayesSetup
Steven
--
Steven Dickenson <[EMAIL PROTECTED]>
http://www.mrchuckles.net

what about Iron Mail

[Dana Holland]

I thought the discussion on Barracuda was interesting since we were 
considering it.  Has anyone ever heard of/used Iron Mail?

newbie ? on testing SA

[Tracey Gates]

I have a new server that I've brought up on SpamAssassin.  I have the
.cf files listed below that I've added to it to catch other spam
messages.  It's catching some spam but not the amount that my old server
had.
Here is the snippet of my SA's local.cf
rewrite_subject  0
report_header1
use_terse_report 1
spam_level_stars 0
defang_mime  0
required_hits3.5

My questions are:
1.  How do I test that these .cf files are working?
2.  What other .cf rules are good to add to catch most spam that
I'm missing?
3.  How do I test a message that got through to me but is spam?
4.  How do I tell SA that a message is spam?

evilnumbers.cf
70_sare_bayes_poison_nxm.cf
70_sare_html0.cf
70_sare_adult.cf
70_sare_random.cf
chickenpox.cf
weeds.cf
backhair.cf
Tripwire.cf

Thanks!

Tracey Gates
Lead Developer
[EMAIL PROTECTED]

This communication is intended only for the recipient(s) named above;
may be confidential and/or legally privileged; and, must be treated as
such in accordance with state and federal laws. If you are not the
intended recipient, you are hereby notified that any use of this
communication, or any of its contents, is prohibited. If you have
received this communication in error, please reply to the sender and
then delete the message from your computer system immediately.

Re: FPs on MSGID_FROM_MTA_ID

[Nick Leverton]

On Wed, Mar 02, 2005 at 10:59:20AM -0700, Bob Proulx wrote:
> David B Funk wrote:
> > I have a functionally equivalent rule that I created back in SA-2.5 days.
> 
> Me too.  I started out making that a hard test.  But I needed to back
> it out, darn it!  Why can't legitimate MTAs play by the rules?

Message-ID is a SHOULD rather than a MUST even in RFC 2822, and was not
a Required field in RFC 822.  Outlook 2003 (I think, some M$ MUA anyway) was
changed to not add a Message-Id, on the assumption that Exchange would.

Nick

spamassasin global bayes database

[Matt]

What do I have to do to get spamassassin to use a global bayes
database for all users on the system, rather then per user?

Re: Typical spam not detected at all.. there is no rule for it :- \

[Joe Kletch]

Would a similar rule but looking for @ work to flag messages with say 
10 addresses in the To: field?

Joe Kletch
On Mar 2, 2005, at 12:57 PM, Chris Santerre wrote:
How about an eval that counts the number of '|' in a message. Over 4 
and you
got yourself a spam :)

--Chris
-Original Message-
From: David Velásquez [mailto:[EMAIL PROTECTED]
Sent: Wednesday, March 02, 2005 1:46 PM
To: [EMAIL PROTECTED]
Subject: Typical spam not detected at all.. there is no rule for it 
:-\

Spam detection software, running on the system "co3.conexcol.com", has
identified this incoming email as possible spam.  The original message
has been attached to this so you can view it (if it isn't
spam) or label
similar future email.  If you have any questions, see
[EMAIL PROTECTED] for details.
Content preview:  US Oil and Gas Report Oi| C|imbs, Gains Soar We have
 the |eading track record for finding fast moving, Low-priced energy
 plays. Look at the moves made by our last 2 Hot Picks: SPRL
.14 to .36
 in 12 days, up 157% PRVB .10 to .265 in 8 days, up 165% [...]
Content analysis details:   (-2.6 points, 5.0 required)
pts rule name  description
 --
--
-2.6 BAYES_00   BODY: Bayesian spam probability is 0 to 1%
   [score: 0.]

Re: Typical spam not detected at all.. there is no rule for it :-\

[Duncan Hill]

On Wednesday 02 March 2005 18:45, David Velásquez wrote:
> Content preview:  US Oil and Gas Report Oi| C|imbs, Gains Soar We have
>   the |eading track record for finding fast moving, Low-priced energy
>   plays. Look at the moves made by our last 2 Hot Picks: SPRL .14 to .36
>   in 12 days, up 157% PRVB .10 to .265 in 8 days, up 165% [...]

Some days, custom rules are good things.  RM (I think, sorry, email is in 
other box) has some that pick these up.  The rule

bodySTOCK_SPAM2 /If you wish to stop future mai[l|\|]ings, or if you 
fee[l|\|] you have been wrong/

works like a charm, and RM should be adding a SARE rule that catches that 
email address at the bottom.  The numbers change, but the prefix is good, as 
is the suffix.

Re: Rules SA

[David Velásquez]

If the email you're trying to stop is really a SPAM, I suggest you sould 
report it.

To report a spam, just save it without modifying it into a file, then run 
command:

   spamassassin --report < thenameofthefile
...where, of course, "nameofthefile" is the name of the file you used to 
save the spam message.

Another usefull things I did to improve SA effectiveness was to install 
Phyzor, Razor2 and DCC to check on this databases and very important too is 
to install "rulesdujour script" wich will improve your SA by downloading a 
lot of mainteined rules wich SA doesn´t include originaly.

You can find "rulesdujour script" and rules set at: 
http://www.exit0.us/index.php/RulesDuJourRuleSets and 
http://sandgnat.com/rdj/

David A. Velásquez R.
Gerente Fundador
Conexiones Colombianas (CONEXCOL)
[EMAIL PROTECTED]
http://www.conexcol.com/ - http://www.sipo.cl
Tel/Fax. (57)(4) 3122600
Cel. (57)(300) 6533517
Cra. 34 No. 7 - 157
A.A. 12137 Medellín, Ant. CO.
- Original Message - 
From: "Matt Kettler" <[EMAIL PROTECTED]>
To: "Mario Sergio Candian" <[EMAIL PROTECTED]>
Cc: 
Sent: Wednesday, March 02, 2005 2:30 PM
Subject: Re: Rules SA


At 02:25 PM 3/2/2005, Mario Sergio Candian wrote:
Oks... I wrong.. sorry... but, I need too to block any emails with subject 
SERASA... :/ What I need to do to block it?

You could use a body rule, but that will match the body text as well as 
the subject line..

If you only want to match the subject line, use header rule that targets 
the subject line:

header SUBJECT_SERASA   Subject =~/SERASA/
score SUBJECT_SERASA5.0
Also note I left the i off the end of the regex, which makes it 
case-sensitive so it will only match all-caps. If you want to match 
lower-case letters, add the i back on the end.

RE: Typical spam not detected at all.. there is no rule for it :- \

[Kenneth Porter]

--On Wednesday, March 02, 2005 1:57 PM -0500 Chris Santerre 
<[EMAIL PROTECTED]> wrote:

How about an eval that counts the number of '|' in a message. Over 4 and
you got yourself a spam :)
Would FP on lots of source code examples, particularly those including 
regex's.

Re: Rules SA

[Matt Kettler]

At 02:25 PM 3/2/2005, Mario Sergio Candian wrote:
Oks... I wrong.. sorry... but, I need too to block any emails with subject 
SERASA... :/ What I need to do to block it?

You could use a body rule, but that will match the body text as well as the 
subject line..

If you only want to match the subject line, use header rule that targets 
the subject line:

header SUBJECT_SERASA   Subject =~/SERASA/
score SUBJECT_SERASA5.0
Also note I left the i off the end of the regex, which makes it 
case-sensitive so it will only match all-caps. If you want to match 
lower-case letters, add the i back on the end. 

Re: Rules SA

[Evan Platt]

At 11:25 AM 3/2/2005, you wrote:
Oks... I wrong.. sorry... but, I need too to block any emails with subject 
SERASA... :/ What I need to do to block it?
Depends on how you're calling SpamAssassin. SA will not block anything, it 
doesn't have the capabilities to do so.. 

Re: Rules SA

[Mario Sergio Candian]

Oks... I wrong.. sorry... but, I need too to block any emails with subject 
SERASA... :/ What I need to do to block it?

Mario Sergio Candian
-
"Dreams as if you'll live forever. Live as if you'll die today" -- James Dean
On Wed, 2 Mar 2005, Matt Kettler wrote:
At 01:46 PM 3/2/2005, Mario Sergio Candian wrote:
I need to block all emails that I receive with subject SERASA e MAMONAS 
ASSASSINAS. I try with this:

bodyPROVER_MAMONAS1/MAMONAS.*ASSASINAS/i
score   PROVER_MAMONAS15.0
Looks like you're missing the 4th S in ASSASSINAS

Re: Rules SA

[Matt Kettler]

At 01:46 PM 3/2/2005, Mario Sergio Candian wrote:
I need to block all emails that I receive with subject SERASA e MAMONAS 
ASSASSINAS. I try with this:

bodyPROVER_MAMONAS1/MAMONAS.*ASSASINAS/i
score   PROVER_MAMONAS15.0
Looks like you're missing the 4th S in ASSASSINAS

Re: Suggestion: OCR

 --- On Wed 03/02, Matt Kettler < [EMAIL PROTECTED] > wrote:

That part is definitely NOT safe in the context of spamassassin... Nonsense 
looks a lot like bugs in spam mailers, and very little like legitimate email to 
SA.

If nothing else, consider the tripwire rules, which look for letter 
combinations that don't exist in normal English...
---

Thanks! If so, then it's a bit more work to implement. For example, a trivial 
idea is not to let the attachments, which stem from images, go through the 
rules that search for nonsense.

I meant 'safe' in the following sense: if the tool says some meaningful word 
(e.g. present in the english wordlist up to a small misspell), then this word 
is surely present in the image up to a small misspell. So, if some spam rule 
sees "viagra" or 'click here to get removed' after OCRing, then it is 'safe' to 
give a hit for it, for example.

Another work-intensive method could be as follows (corrections are welcome)
1. OCR.
2. Throw out all the words which are not in the english (german, russian, 
etc...) dictionary up to a misspell. E.g. tolerate at most one error per word. 
Correct the misspelled words. (Fast dictionary search required, e.g. represent 
wordlists as binary balanced trees.)
3. run other text-based rules.

Actually, I posted because I get too much image spam (which goes ok through SA) 
and tried to determine the possibility of catching it with the present tools. 
Sometimes I get photos and image-smileys so I'm very reluctant to stop all 
mails containing images without inspecting images.

My strong belief is that such tools as gocr can really help. The other question 
is how to integrate it in SA and who does it. I'm afraid I cannot dig into the 
SA code myself; so it's a suggestion to the advanced users and developers.

Regards,
sasha.

___
Join Excite! - http://www.excite.com
The most personalized portal on the Web!

Re: Typical spam not detected at all.. there is no rule for it :-\

[David Velásquez]

Notice that kind of "misspelings" should be Tripwire ruleset work... right?
The good news is I reported it and now it's detected:
Content analysis details:   (7.3 points, 5.0 required)
pts rule name  description
 -- --
2.1 BAYES_80   BODY: Bayesian spam probability is 80 to 95%
   [score: 0.8973]
2.2 DCC_CHECK  Listed in DCC 
(http://rhyolite.com/anti-spam/dcc/)
3.1 RCVD_IN_XBLRBL: Received via a relay in Spamhaus XBL
   [83.132.46.217 listed in sbl-xbl.spamhaus.org]

- Original Message - 
From: "Chris Santerre" <[EMAIL PROTECTED]>
To: <[EMAIL PROTECTED]>
Sent: Wednesday, March 02, 2005 1:57 PM
Subject: RE: Typical spam not detected at all.. there is no rule for it :-\


How about an eval that counts the number of '|' in a message. Over 4 and 
you
got yourself a spam :)

--Chris
-Original Message-
From: David Velásquez [mailto:[EMAIL PROTECTED]
Sent: Wednesday, March 02, 2005 1:46 PM
To: [EMAIL PROTECTED]
Subject: Typical spam not detected at all.. there is no rule for it :-\
Spam detection software, running on the system "co3.conexcol.com", has
identified this incoming email as possible spam.  The original message
has been attached to this so you can view it (if it isn't
spam) or label
similar future email.  If you have any questions, see
[EMAIL PROTECTED] for details.
Content preview:  US Oil and Gas Report Oi| C|imbs, Gains Soar We have
 the |eading track record for finding fast moving, Low-priced energy
 plays. Look at the moves made by our last 2 Hot Picks: SPRL
.14 to .36
 in 12 days, up 157% PRVB .10 to .265 in 8 days, up 165% [...]
Content analysis details:   (-2.6 points, 5.0 required)
pts rule name  description
 -- 
--
-2.6 BAYES_00   BODY: Bayesian spam probability is 0 to 1%
   [score: 0.]

RE: Typical spam not detected at all.. there is no rule for it :- \

[Chris Santerre]

How about an eval that counts the number of '|' in a message. Over 4 and you
got yourself a spam :) 

--Chris 

>-Original Message-
>From: David Velásquez [mailto:[EMAIL PROTECTED]
>Sent: Wednesday, March 02, 2005 1:46 PM
>To: [EMAIL PROTECTED]
>Subject: Typical spam not detected at all.. there is no rule for it :-\
>
>
>Spam detection software, running on the system "co3.conexcol.com", has
>identified this incoming email as possible spam.  The original message
>has been attached to this so you can view it (if it isn't 
>spam) or label
>similar future email.  If you have any questions, see
>[EMAIL PROTECTED] for details.
>
>Content preview:  US Oil and Gas Report Oi| C|imbs, Gains Soar We have
>  the |eading track record for finding fast moving, Low-priced energy
>  plays. Look at the moves made by our last 2 Hot Picks: SPRL 
>.14 to .36
>  in 12 days, up 157% PRVB .10 to .265 in 8 days, up 165% [...]
>
>Content analysis details:   (-2.6 points, 5.0 required)
>
> pts rule name  description
> -- 
>--
>-2.6 BAYES_00   BODY: Bayesian spam probability is 0 to 1%
>[score: 0.] 
>

Typical spam not detected at all.. there is no rule for it :-\

[David Velásquez]

Spam detection software, running on the system "co3.conexcol.com", has
identified this incoming email as possible spam.  The original message
has been attached to this so you can view it (if it isn't spam) or label
similar future email.  If you have any questions, see
[EMAIL PROTECTED] for details.
Content preview:  US Oil and Gas Report Oi| C|imbs, Gains Soar We have
 the |eading track record for finding fast moving, Low-priced energy
 plays. Look at the moves made by our last 2 Hot Picks: SPRL .14 to .36
 in 12 days, up 157% PRVB .10 to .265 in 8 days, up 165% [...]
Content analysis details:   (-2.6 points, 5.0 required)
pts rule name  description
 -- --
-2.6 BAYES_00   BODY: Bayesian spam probability is 0 to 1%
   [score: 0.] 
--- Begin Message ---
US Oil and Gas Report
Oi| C|imbs, Gains Soar

We have the |eading track record for finding fast moving, 
Low-priced energy plays.

Look at the moves made by our last 2 Hot Picks:
SPRL .14 to .36 in 12 days, up 157%
PRVB .10 to .265 in 8 days, up 165%

Like a viper coi|ed to strike, this Hot Oi| P|ay is on the fast-track 
to Mi|lions in gains from expanding production in the Heart of the "Oil 
Be|t".

Emerson Oil and Gas, Inc. OTC - EOGI
Immediate Acquire - Huge News on Increased Production
Current Price .15
Target Price .4O in 1O days

Oi| prices continue to explode and ignite the shares of major producers 
which is followed by soaring underva|ued OTC Oi| Plays. Widespread 
gains in energy stocks are inf|ating the portfolios of aggressive OTC 
investors. Due to recent|y acquired high-potential oil and gas projects, 
EOGI is an undervalued gem experiencing a sharp in crease in production 
revenue and becoming a "cash-f|ow positive" |eader in major oil-producing 
states.

EOGI's high-yielding, multiple dri||ing programs have been solidified 
through valuable strategic a||iances. These exp|oration partnerships 
enable the Company to accumu|ate substantial working interests being 
developed into a continuous|y expanding revenue stream. EOGI is focused on 
joint-ventures invo|ving wel| re-entry programs in extremely prolific 
production areas using the highest leve| of technology currently 
available.

Recent developments for EOGI have been perfectly timed as all 6 Dow 
Jones Oi| and Gas Indexes just hit their highest |evels of the year, with 
Energy Stocks outperforming a|l other market sectors. The Company's Oil 
and Gas interests in Louisiana and Texas, in addition to their 
developing natural gas wel|s in Alberta, Canada, are continuing to in crease in 
value with energy prices up 6O% from one year ago. Leading economists 
and oil ana|ysts agree on predictions that a further 30-60% in crease is 
|ikely next year.

We are expecting EOGI to release major news announcements regarding 
production figures on multiple projects in high-gear. EOGI's expansion 
plans are on track to acquire additiona| prospects and properties that can 
on|y add to their growing bottom-line and spark further interest in 
this underva|ued oi| play. With a very sma|| f|oat estimated at |ess than 
5 million shares, EOGI should gap sharply to the upside on heavy vo|ume 
as the va|ue of their current oil deals are revea|ed.

US Oil and Gas Report cautions that sma|l and micro-cap stocks are 
high-risk investments and that some or a|| of the investment can be |ost. 
We suggest you consult a professional investment advisor before making 
any investment decisions. US Oi| and Gas Report recommends you use the 
information found here as an initia| starting point for conducting 
research and due di|igence on the featured company in order to determine an 
opinion of the company before investing. We are not an Investment 
Advisor, Financia| P|anning Service or a Stock Brokerage Firm and in 
accordance with such is not offering investment advice or promoting any 
investment strategies. This is not an offering of securities for sa|e or 
solicitation of any attempt to acquire or se|l securities. US Oil and Gas 
Report has been given nineteen thousand do|lars from a third party for 
the dissemination of this company profi|e. Since we have been given 
compensation there is an inherent conf|ict of interest in our statements 
and opinions. Readers of this pub|ication are cautioned not to p|ace 
undue re|iance on future-|ooking statements, which are based on certain 
assumptions and expectations invo|ving various risks and uncertainties 
that cou|d cause results to differ material|y from those set forth in 
those statements.

If you wish to stop future mai|ings, or if you fee| you have been 
wrongfully placed in our membership, p|ease go here or send a b|ank  
e mail with No Thanks in the subject to   (-stox0033@  yahoo.com-)






--- End Message ---

Rules SA

[Mario Sergio Candian]

Hi list,
I have the SA installed in my FreeBSD server. How I can create some rules 
like this:

bodyPROVER_SERASACOMBR1/serasa.com.br/
score   PROVER_SERASACOMBR15.0
bodyPROVER_SERASACOMBR2/SERASA/
score   PROVER_SERASACOMBR25.0
bodyPROVER_MAMONAS1/MAMONAS ASSASINAS/
score   PROVER_MAMONAS15.0
I need to block all emails that I receive with subject SERASA e MAMONAS 
ASSASSINAS. I try with this:

bodyPROVER_MAMONAS1/MAMONAS.*ASSASINAS/i
score   PROVER_MAMONAS15.0
but doesnt work. :/
How I can do it?
Thanks for all.
Mario Sergio Candian
-
"Dreams as if you'll live forever. Live as if you'll die today" -- James Dean

Re: Suggestion: OCR

[Matt Kettler]

At 01:35 PM 3/2/2005,  wrote:
I've kust made tests with gocr (a OCR command-line linux software) and it 
proves to be safe, i.e. if it fails to detect a text, you see some 
nonsense collection of symbols.
That part is definitely NOT safe in the context of spamassassin... Nonsense 
looks a lot like bugs in spam mailers, and very little like legitimate 
email to SA.

If nothing else, consider the tripwire rules, which look for letter 
combinations that don't exist in normal English...

RE: Suggestion: OCR

 And here is the attachment

___
Join Excite! - http://www.excite.com
The most personalized portal on the Web!
<>

Suggestion: OCR

I've kust made tests with gocr (a OCR command-line linux software) and it 
proves to be safe, i.e. if it fails to detect a text, you see some nonsense 
collection of symbols. It can handle pnm (and some other formats) directly and 
cannot handle gifs and jpegs directly. It supposses the text is darker than the 
background, so some preprocessing is needed (i don't know how to invert the 
colors with linux tools, but it's a matter of googling). What i managed to 
detect is

[quote]
click here to get removed

all other enquiries send to:
[EMAIL PROTECTED],com
[/quote]

The real picture is in the attachment. Two OCR errors were made. The command 
that used was
giftopnm tG0rzUDQO.gif | pnmtojpeg --quality=100 |djpeg -pnm -grayscale | gocr -

That gives you the quote on the standard output.
Sometimes you have to split animated gif's into a sequence of .png's. Then it 
looks like this

gif2png tG0rzUDQO.gif
[.png, .p01, .p02, ... are generated]
pngtopnm tG0rzUDQO.png | pnmtojpeg --quality=100 |djpeg  -pnm -grayscale | gocr 
- 

For jpegs, you have a simpler procedure for the last two steps. Time for 
working is neglectible (at least on my machine).

The question is whether it is implementable to convert all pictures via this 
tool gocr into text and then run the usual SA tests on the so combined email? 
If yes, could someone do that?
Thanks a lot.
Sasha.

___
Join Excite! - http://www.excite.com
The most personalized portal on the Web!

RE: Why was this message not tagged?

[Matt Kettler]

At 11:25 AM 3/2/2005, Jon Dossey wrote:
I apologize, I was in a rush.  System is redhat fc2, sendmail 8.13.1,
spamassassin 3.0.1 and spamass-milter 0.2.0 (updated for SA 3.0, haven't
switched to 0.3.0 yet).
Here's (most of) my /home/spamd/.spamassassin/user_prefs:

# How many points before a mail is considered spam.
required_hits   5.0 4.5
Erm.. that's not right... required_hits should only have one parameter...

Re: FPs on MSGID_FROM_MTA_ID

[Bob Proulx]

David B Funk wrote:
> I have a functionally equivalent rule that I created back in SA-2.5 days.

Me too.  I started out making that a hard test.  But I needed to back
it out, darn it!  Why can't legitimate MTAs play by the rules?

> I had given it a hefty score (1.5) as it seend a good spam-sign, but
> subsequently toned it down as I found some mail-list packages don't
> add Message-IDs to their output.
> I still have the rule, just with a low score (0.3).

I just checked my folders and there is some high profile mail that
that does not set the message id.  Such as all of the Red Cross mail I
have ever received.  Verisign messages (well that is a type of spam).
A large number of messages from various political parties (another
type of spam).  And various different automated order notification
systems which I would not want to lose.  I am not seeing any trouble
with false positives so the current default SA value seems to be a
good choice here.

Bob

Re: Confused about HELO_DYNAMIC_*

[David B Funk]

On Wed, 2 Mar 2005, Justin Mason wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
>
> It's OK to have a dyn IP as the source, alright, as long as it
> doesn't HELO using that hostname.   That's what HELO_DYNAMIC_*
> matches, as it's a very strong spam signature.
>
> > Received: from h00c04f2d101a.ne.client2.attbi.com 
> > (h00c04f2d101a.ne.client2.attbi.com [66.30.139.164])
> > by webmail.grad.brandeis.edu (IMP) with HTTP
>
> So IMP is doing this now?  argh, bad decision...
>
> - --j.

NO! NO!, Read the headers!
Note the "with HTTP" as the protocol.

That's somebody on a cable modem using a web browser, connecting to
a HORDE-IMP webmail server. So the webmail server is doing the
actual sending of the SMTP part, but it is adding that additonal
'Received:' header to expose the original source of the actual message,
namely the user operating the broswer.

It's a cute idea, extending the concept of the 'Received:' chain,
but causes confusion for things such as SA who don't note the 'with HTTP'
part. ;)

Dave (who also has a HORDE-IMP webmail server at our site.

-- 
Dave Funk  University of Iowa
College of Engineering
319/335-5751   FAX: 319/384-0549   1256 Seamans Center
Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527
#include 
Better is not better, 'standard' is better. B{

Re: Confused about HELO_DYNAMIC_*

[Matt Kettler]

At 11:20 AM 3/2/2005, Jay Levitt wrote:
> It looks like it might be a trust path issue.. are the brandeis.edu 
hosts trusted? If so, SA would be correct in deciding a dynamic node from 
attbi.com dropped mail off directly.

Nope, they're not - I had no trusted_networks or internal_networks defined.
Just because you have no trusted_networks defined does not mean a host 
isn't trusted... if you don't define a trusted_networks, SA tries to guess...

> What do the *.home.jay.fm hosts resolve as when the machine running SA 
does a DNS lookup? are they reserved IP's? If so, you'll have trust path 
issues and need to manualy define trusted_networks.

Yep, they're 192.168/16.  According to the man page for 
Mail::SpamAssassin::Conf, that should be automatically trusted due to the 
DNS checks... is that not correct?
Yes, but this is where the problem comes.. they'll be trusted... AND the 
first non-reserved IP will also be trusted.

Therefore, SA is going to trust blanca.unet.brandeis.edu by default.
SA's auto-detect assumption is that the external MX must have a real-world 
IP. This is true in some networks, false in others. However, this algorithm 
is pretty much the best-case guess SA can make. Under trust and over trust 
are one just as bad as the other, so to make it only trust the non-reserved 
IP's would break networks which have NATed internal servers and non-NATed 
external servers. Trusting reserved plus one (the way it is now) breaks 
networks which are entirely NATed.

I'll try setting it manually.
This is a good idea, as SA's autodetect is not going to work for your 
network.
Another symptom of the trust path bug is that ALL_TRUSTED winds up firing 
off for spam, which should never happen.

RE: Why was this message not tagged?

[Jon Dossey]

I apologize, I was in a rush.  System is redhat fc2, sendmail 8.13.1,
spamassassin 3.0.1 and spamass-milter 0.2.0 (updated for SA 3.0, haven't
switched to 0.3.0 yet).  

Here's (most of) my /home/spamd/.spamassassin/user_prefs:

# SpamAssassin user preferences file.  See 'perldoc
Mail::SpamAssassin::Conf'
# for details of what can be tweaked.

###

# How many points before a mail is considered spam.
required_hits   5.0 4.5
ok_locales  all
ok_languagesall
bayes_auto_learn_threshold_nonspam  0.1
bayes_auto_learn_threshold_spam 12.0
bayes_auto_learn1
report_safe 1
rewrite_header Subject [SPAM]
skip_rbl_checks 0

Other messages are being tagged just fine.  Its just this one message
that slipped through with > 5.0 score, but without being tagged.  The
only thing that seemed different about it was that it was flagged as
"Urgent".

Thanks,
.jon


> -Original Message-
> From: Evan Platt [mailto:[EMAIL PROTECTED]
> Sent: Wednesday, March 02, 2005 10:19 AM
> To: [EMAIL PROTECTED]
> Subject: Re: Why was this message not tagged?
> 
> At 08:13 AM 3/2/2005, you wrote:
> >It scored a 5.8 out of a required 5.0, but the message wasn't tagged.
> >Any ideas why?
> 
> My guess is people here would need to know the contents of your local
> config to know why. My guess is an incorrect setting.
> 
> 

Re: Confused about HELO_DYNAMIC_*

[Jay Levitt]

Matt Kettler wrote:
At 10:43 PM 3/1/2005, Jay Levitt wrote:
Why would the HELO_DYNAMIC_* rules trigger on these headers?  Surely 
it's ok to have a dynamic IP as the *source* of a message, just not 
in a relay..?

It looks like it might be a trust path issue.. are the brandeis.edu 
hosts trusted? If so, SA would be correct in deciding a dynamic node 
from attbi.com dropped mail off directly.
Nope, they're not - I had no trusted_networks or internal_networks defined.
What do the *.home.jay.fm hosts resolve as when the machine running SA 
does a DNS lookup? are they reserved IP's? If so, you'll have trust 
path issues and need to manualy define trusted_networks.
Yep, they're 192.168/16.  According to the man page for 
Mail::SpamAssassin::Conf, that should be automatically trusted due to 
the DNS checks... is that not correct?  I'll try setting it manually.

Jay

Re: Why was this message not tagged?

[Evan Platt]

At 08:13 AM 3/2/2005, you wrote:
It scored a 5.8 out of a required 5.0, but the message wasn't tagged.
Any ideas why?
My guess is people here would need to know the contents of your local 
config to know why. My guess is an incorrect setting.

Why was this message not tagged?

[Jon Dossey]

It scored a 5.8 out of a required 5.0, but the message wasn't tagged.
Any ideas why?

Message-ID: <[EMAIL PROTECTED]>
From: "VB LOTTERY" <[EMAIL PROTECTED]>
To: "WINNER" <[EMAIL PROTECTED]>
Subject: BANK GIRO LOTTERJ WINNING ANNOUNCEMENT
Date: Wed, 02 Mar 2005 10:28:01 +0400
MIME-Version: 1.0
Content-Type: text/plain;
charset="us-ascii"
Content-Transfer-Encoding: base64
X-Priority: 1
X-MSMail-Priority: High
X-Mailer: Microsoft Outlook Express 6.00.2800.1123
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1123
X-Virus-Scanned: ClamAV 0.80/735/Mon Feb 28 16:13:05 2005
clamav-milter version 0.80j
on dhgsrv17.deltahealthgroup.com
X-Virus-Status: Clean
X-Spam-Flag: YES
X-Spam-Status: Yes, score=5.8 required=5.0
tests=DATE_IN_PAST_06_12,DCC_CHECK,
MIME_BASE64_TEXT,NIGERIAN_BODY1,NIGERIAN_BODY2,SUBJ_ALL_CAPS,
X_MSMAIL_PRIORITY_HIGH,X_PRIORITY_HIGH autolearn=no
version=3.0.1
X-Spam-Level: *
X-Spam-Checker-Version: SpamAssassin 3.0.1 (2004-10-22) on 
dhgsrv17.deltahealthgroup.com
Return-Path: [EMAIL PROTECTED]
X-OriginalArrivalTime: 02 Mar 2005 13:54:46.0084 (UTC)
FILETIME=[648D6840:01C51F2F]



Thanks,
.jon

RE: Greylisting

[Chris Blaise]

If you use exim, check out:

 http://marc.merlins.org/linux/exim/sa.html

It allows SA scanning at the MTA level and includes a GreyListing
pluging for SA3.  You should be able to configure exim to only allow it for
certain recipient addreses but you'd have to do that research yourself.

 Chris 

-Original Message-
From: Matt [mailto:[EMAIL PROTECTED] 
Sent: Wednesday, March 02, 2005 6:20 AM
To: [EMAIL PROTECTED]
Subject: Greylisting

Hi,
Is there any kind of plugin or patch for spamassassin that will allow me to
selectively turn on GREYLISTing for certain user accounts?

When I say greylist I mean:  All e-mail coming into them is bounced with a
temporary error the first time, and then accepted the second
time.   If accepted (ie sent a second time) it is then put in a
database of valid e-mail accounts for the next 60 days (or until another
e-mail is received).

This has the potential to greatly reduce spam since spammers don't usually
try to send mail again, however I don't want to roll this out on a server
wide setup, but rather would like to do it on a per-user basis.

Any thoughts?

 ~ Matt

Re: Greylisting

[David B Funk]

On Wed, 2 Mar 2005, Matt wrote:

> Hi,
> Is there any kind of plugin or patch for spamassassin that will allow
> me to selectively turn on GREYLISTing for certain user accounts?
>
> When I say greylist I mean:  All e-mail coming into them is bounced
> with a temporary error the first time, and then accepted the second
> time.   If accepted (ie sent a second time) it is then put in a
> database of valid e-mail accounts for the next 60 days (or until
> another e-mail is received).

One thing that you need to be aware of, Greylisting MUST be done
at the MTA level, it cannot be done by SpamAssassin alone.
As a matter of fact Greylisting can be done without SA at all.

When doing Greylisting, you do -not- 'bounce' e-mail, you do a
SMTP REJECT (temp fail error code; 4xx) with your MTA.
It must be done at the topographical "edge" of your network
not at the "center". (IE if you have more than one system involved
with processing and storing e-mail, it must be the first system
that remote machines talk to).

Thus this is completely dependent upon what MTA you use, very
little to do with SA.

-- 
Dave Funk  University of Iowa
College of Engineering
319/335-5751   FAX: 319/384-0549   1256 Seamans Center
Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527
#include 
Better is not better, 'standard' is better. B{

RE: Potential new auto-learning strategy

[Gray, Richard]

For various reasons (some political, some technical) we 
don't use bayes here. It can be very frustrating, but I'm sure you guys know 
what its like to have your hands tied by corporate 
wrangling.
 
The reason I proposed a more complex logic than the one 
you suggest was to handle down-scoring rules that performed poorly as well as 
up-scoring effective rules. By using a fixed score, you run the risk either setting it too low and the system taking too long to learn, or too high (it has 
been demonstrated that this can cause chaotic behaviour in some systems). By 
using a function that calculates X based on the overall score of the message, 
the other rules hit, and diminished by the learn rate, the system can quickly 
cover the large gap, but when the distance between the two scores becomes small, 
the changes to the score values are appropriate small, tending the system towards stability (assume spammers don't change tactic)
 
Should 2 particular rules occur commonly together, this 
would also have the effect of balancing out score changes across them both, relative to their base values.
 
I'd like to get into doing this, but work is swamped (I 
don't get to play with spam all day :( ). If there are other people keen on doing this then maybe we can get a collaboration going. 
 
R


From: Chris Santerre 
[mailto:[EMAIL PROTECTED] Sent: 02 March 2005 
15:16To: Gray, Richard; 
users@spamassassin.apache.orgSubject: RE: Potential new auto-learning 
strategy

There has 
been a lot of talk about dynamic scoring. Most people argue that Bayes is a good 
substitute for it already. But not if you don't use Bayes ;) 

 
I think its a 
worthy idea for testing. Although the logic could be fairly simple. Like using 
the top hitting rules script in a cron job. pulling out the N'th top rules and 
adding X points to them based on the hits. Thats something I've wanted to play 
with, but had no time. 
 
--Chris 


  -Original Message-From: Gray, Richard 
  [mailto:[EMAIL PROTECTED]Sent: Wednesday, March 02, 2005 7:03 
  AMTo: users@spamassassin.apache.orgSubject: Potential 
  new auto-learning strategy
  I 
  saw an article a while back about some DJs who were using perl as a mixing 
  tool by writing perl code that edited itself while it ran in a loop. I thought 
  this was kind of cool. 
   
  I 
  studied AI at university, and remember a good bit of discussion regarding   feedback systems.
   
  So, to combine the two, I was thinking of how to use SA in a similar 
  structure, and propose a dynamic weighting system for SA rules. Consider the 
  scores that a base installation of SA gives to its rules, but when shown   messages to learn from, it modifies the score weighting of the rules rather 
  than the bayes system.
   
  I'll not throw out a discussion regarding learning rates and so, but I 
  can imagine the logic being loosely based on how much influence the rule had 
  on the total score, the distance of the final result from the spam/ham 
  boundary, and the learning rate chosen by the 
  administrator.
   
  Any feedback?
   
  R---This 
  email from dns has been validated by dnsMSS Managed Email Security and is free 
  from all known viruses.For further information contact 
  [EMAIL PROTECTED]

---
This email from dns has been validated by dnsMSS Managed Email Security and is free from all known viruses.

For further information contact [EMAIL PROTECTED]

milter-spamc question

[Matias Lopez Bergero]

Hello,
Sorry for this OT.
It is anybody using milter-spamc to connect to a unix spamd socket?
I have tried with the -H option but it doesn't works.
I was trying to test if using a unix socket instead of tcp connect would 
be a better in matters of performance.

Any comments/ideas are welcome :)
BR,
Matías.

Re: Greylisting

[Andy Jezierski]

Matt <[EMAIL PROTECTED]> wrote on 03/02/2005
07:19:42 AM:

> Hi,
> Is there any kind of plugin or patch for spamassassin that will allow
> me to selectively turn on GREYLISTing for certain user accounts?
> 

Yep, like Steven said, greylisting has to kick in
at the MTA level before SA even gets invoked.  So it depends on which
MTA you are using. For Sendmail, Milter-Greylist is a good one, I'm using
it. I'd safely say that greylisting has reduced our inbound spam by at
least 90%. Of the remaining 10% that makes it in, SA filters out about
97%-98% of it.  
Be careful about the length of time that you auto-whitelist
the senders, I believe that the table is stored in memory, someone chime
in if I'm wrong, so if you're going to keep the entries for 60 days, you
could potentially need a whole bunch of memory.

Also, if you're running sendmail 8.13 or above, look
into the pre-greeting pause to eliminate some more spam before it even
hits greylisting.

Andy

RE: Potential new auto-learning strategy

[Chris Santerre]

There has 
been a lot of talk about dynamic scoring. Most people argue that Bayes is a good 
substitute for it already. But not if you don't use Bayes ;) 

 
I think its a 
worthy idea for testing. Although the logic could be fairly simple. Like using 
the top hitting rules script in a cron job. pulling out the N'th top rules and 
adding X points to them based on the hits. Thats something I've wanted to play 
with, but had no time. 
 
--Chris 


  -Original Message-From: Gray, Richard 
  [mailto:[EMAIL PROTECTED]Sent: Wednesday, March 02, 2005 7:03 
  AMTo: users@spamassassin.apache.orgSubject: Potential 
  new auto-learning strategy
  I 
  saw an article a while back about some DJs who were using perl as a mixing 
  tool by writing perl code that edited itself while it ran in a loop. I thought 
  this was kind of cool. 
   
  I 
  studied AI at university, and remember a good bit of discussion regarding 
  feedback systems.
   
  So, to combine the two, I was thinking of how to use SA in a similar 
  structure, and propose a dynamic weighting system for SA rules. Consider the 
  scores that a base installation of SA gives to its rules, but when shown 
  messages to learn from, it modifies the score weighting of the rules rather 
  than the bayes system.
   
  I'll not throw out a discussion regarding learning rates and so, but I 
  can imagine the logic being loosely based on how much influence the rule had 
  on the total score, the distance of the final result from the spam/ham 
  boundary, and the learning rate chosen by the 
  administrator.
   
  Any feedback?
   
  R---This 
  email from dns has been validated by dnsMSS Managed Email Security and is free 
  from all known viruses.For further information contact 
  [EMAIL PROTECTED]

Re[2]: FPs on MSGID_FROM_MTA_ID

[Robert Menschel]

Hello Stuart,

Tuesday, March 1, 2005, 8:53:38 AM, you wrote:

>> It appears to be doing the right thing. The message originated off-net,
>> but the Message-ID was added locally, which is pretty good spam-sign.
>> Frankly I wish it worked here, because I've had to create my own rule to
>> hit the same thing.

SJ> Right, it is just that I get the impression that a lot of legitimate
SJ> mail servers may be sending mail without proper Message-ID's, causing
SJ> FPs.  So, I wondered if anyone else had seen this as well.

50_scores.cf:score MSGID_FROM_MTA_ID 1.440 1.704 1.756 1.723

IMO: any rule with a score under 1/2 your spam threshold cannot be an
FP.  It's a spam suggestion, nothing more.

Rules that score over 1 are /strong/ suggestions, but suggestions
still.

They don't count as "this is spam" FPs unless enough other rules agree
that this is spam, in which case the FP is the email itself, not
necessarily any single rule.

If you're getting ham which FP, and the FP includes this rule, then
what other rules are being hit? Why does that ham look so spammy?

Bob Menschel

RE: Greylisting

[Philipp Snizek]

HI

check postfix greylisting

Philipp 

> -Original Message-
> From: Matt [mailto:[EMAIL PROTECTED] 
> Sent: Mittwoch, 2. März 2005 14:20
> To: [EMAIL PROTECTED]
> Subject: Greylisting
> 
> Hi,
> Is there any kind of plugin or patch for spamassassin that 
> will allow me to selectively turn on GREYLISTing for certain 
> user accounts?
> 
> When I say greylist I mean:  All e-mail coming into them is 
> bounced with a temporary error the first time, and then 
> accepted the second
> time.   If accepted (ie sent a second time) it is then put in a
> database of valid e-mail accounts for the next 60 days (or 
> until another e-mail is received).
> 
> This has the potential to greatly reduce spam since spammers 
> don't usually try to send mail again, however I don't want to 
> roll this out on a server wide setup, but rather would like 
> to do it on a per-user basis.
> 
> Any thoughts?
> 
>  ~ Matt
> 

Re: Greylisting

[Steven Stern]

Matt wrote:
Hi,
Is there any kind of plugin or patch for spamassassin that will allow
me to selectively turn on GREYLISTing for certain user accounts?
When I say greylist I mean:  All e-mail coming into them is bounced
with a temporary error the first time, and then accepted the second
time.   If accepted (ie sent a second time) it is then put in a
database of valid e-mail accounts for the next 60 days (or until
another e-mail is received).
This has the potential to greatly reduce spam since spammers don't
usually try to send mail again, however I don't want to roll this out
on a server wide setup, but rather would like to do it on a per-user
basis.
Greylisting has to kick in earlier than SpamAssassin.  Have you looked 
at milter-greylist?  http://hcpnet.free.fr/milter-greylist/  It allows 
you to choose which users use grey listing.

I think you're suggesting, however, that it should be possible to set 
something up that lets users choose whether to be using grey-listing. 
You might be able to extend Maia Mailguard 
(http://www.maiamailguard.com/) to do that.

FWIW, I'm running milter-greylist on my account on a busy system.  I get 
almost zero spam coming through. The downside is that some servers may 
take several hours before retrying.  Most retry in about 30 minutes.  I 
generally think that's a fair trade off.

--
   Steve

Greylisting

[Matt]

Hi,
Is there any kind of plugin or patch for spamassassin that will allow
me to selectively turn on GREYLISTing for certain user accounts?

When I say greylist I mean:  All e-mail coming into them is bounced
with a temporary error the first time, and then accepted the second
time.   If accepted (ie sent a second time) it is then put in a
database of valid e-mail accounts for the next 60 days (or until
another e-mail is received).

This has the potential to greatly reduce spam since spammers don't
usually try to send mail again, however I don't want to roll this out
on a server wide setup, but rather would like to do it on a per-user
basis.

Any thoughts?

 ~ Matt

Potential new auto-learning strategy

[Gray, Richard]

I 
saw an article a while back about some DJs who were using perl as a mixing tool 
by writing perl code that edited itself while it ran in a loop. I thought this 
was kind of cool. 
 
I 
studied AI at university, and remember a good bit of discussion regarding feedback systems.
 
So, 
to combine the two, I was thinking of how to use SA in a similar structure, and 
propose a dynamic weighting system for SA rules. Consider the scores that a base 
installation of SA gives to its rules, but when shown messages to learn from, it 
modifies the score weighting of the rules rather than the bayes 
system.
 
I'll 
not throw out a discussion regarding learning rates and so, but I can imagine 
the logic being loosely based on how much influence the rule had on the total 
score, the distance of the final result from the spam/ham boundary, and the learning rate chosen by the administrator.
 
Any 
feedback?
 
R

---
This email from dns has been validated by dnsMSS Managed Email Security and is free from all known viruses.

For further information contact [EMAIL PROTECTED]

Re: X-Spam-Status/content analysis details inconsistencies.

[Justin Mason]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1


there's an upstream server *also* running SpamAssassin, and defanging
the message in some way so that your SpamAssassin server doesn't
get a chance to get the full hits.   

- --j.

jeffrey.arnold writes:
> Hi users,
> 
> I have a weird problem here that i know i am not the only one to 
> encounter, and have yet to see (in much searching) a solution for.
> 
> I am running spamassassin for all mail via spamd/spamc, and filtering on 
> the "X-Spam-Status: Yes" header. The majority of my spam is getting 
> caught, but quite a bit is getting through. The vast majority of the 
> spam that gets through is tagged as spam in the subject line (i enable 
> rewrite_header), but not tagged as spam in the X-Spam-Status header 
> line. In an example spam, i get the following spamass header:
> 
> X-Spam-Checker-Version: SpamAssassin 3.0.2 (2004-11-16) on
>  phase.nyc.analogue.net
> X-Spam-Level: 
> X-Spam-Status: No, score=4.8 required=5.0 testsºYES_50,HTML_10_20,
>  HTML_MESSAGE,RAZOR2_CF_RANGE_51_100,RAZOR2_CHECK,URIBL_SBL
>  autolearn=no version=3.0.2
> 
> The subject/body, on the other hand, contains:
> 
> Subject: SPAM(10.9) Become the man that women desire
> 
> Content analysis details:   (10.9 points, 5.0 required)
> 
>   pts rule name  description
>  -- --
>   0.0 HTML_MESSAGE   BODY: HTML included in message
>   0.0 BAYES_50   BODY: Bayesian spam probability is 40 to 60%
>  [score: 0.5508]
>   0.1 RAZOR2_CF_RANGE_51_100 BODY: Razor2 gives confidence level above
>  50%
>  [cf: 100]
>   0.2 HTML_10_20 BODY: Message is 10% to 20% HTML
>   0.2 MIME_HTML_ONLY BODY: Message only has text/html MIME parts
>   3.5 RAZOR2_CHECK   Listed in Razor2 (http://razor.sf.net/)
>   2.0 RCVD_IN_SORBS_DUL  RBL: SORBS: sent directly from dynamic IP
>  address
>  [218.80.35.217 listed in dnsbl.sorbs.net]
>   3.8 RCVD_IN_DSBL   RBL: Received via a relay in list.dsbl.org
>  []
>   0.1 RCVD_IN_NJABL_DUL  RBL: NJABL: dialup sender did non-local SMTP
>  [218.80.35.217 listed in combined.njabl.org]
>   1.0 URIBL_SBL  Contains an URL listed in the SBL blocklist
>  [URIs: heavenlyitemsbutique.com]
> 
> So it appears that spamassassin tagged this as spam with 10.9 points, 
> but left out some of the rules when writing the X-Spam-Status header, 
> claiming the email only received 4.8 points. Spamassassin is installed 
> directly out of freebsd ports on a fbsd4.11 box with perl 5.8.5. My 
> local.cf is included below. Why would spamassassin be dropping rules 
> like this?
> 
> Thanks much,
> -jba
> 
> __
>   [EMAIL PROTECTED] :: analogue.networks.nyc :: http://analogue.net
> 
> -- local.cf --
> 
> rewrite_header Subject SPAM(_SCORE_)
> skip_rbl_checks 0
> 
> use_razor2 1
> bayes_auto_learn 1
> use_bayes 1
> use_pyzor 1
> auto_learn 1
> use_dcc 1
> dcc_add_header 1
> dns_available yes
> dcc_timeout 10
> dcc_path /my/path/to/dccproc
> 
> rbl_timeout 3
> num_check_received 3
> 
> bayes_path /path/to/my/spamass/
> auto_whitelist_path /path/to/my/auto-whitelist
> 
> header   ARIX_DF rbleval:check_rbl('arix-df', 'fresh.dict.rbl.arix.com.')
> describe ARIX_DF Recent dictionary spammer
> tflags   ARIX_DF net
> 
> header   ARIX_DS rbleval:check_rbl('arix-ds', 'stale.dict.rbl.arix.com.')
> describe ARIX_DS Sender has a history of dictionary spamming
> tflags   ARIX_DS net
> 
> scoreARIX_DF 3.0
> scoreARIX_DS 0.5
> 
> trusted_networks (my trusted nets)
> internal_networks (my internal nets)
> lock_method flock
> 
> ## Optional Score Increases
> score DCC_CHECK 4.000
> score RAZOR2_CHECK 3.500
> score BAYES_99 4.300
> score BAYES_90 3.500
> score BAYES_80 3.000
> score RCVD_IN_SORBS_WEB 2.000
> score RCVD_IN_SBL 3.000
> score URIBL_SBL 1.00
> score ALL_TRUSTED 0.000
> score RCVD_IN_BL_SPAMCOP_NET 2.0
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Exmh CVS

iD8DBQFCJaR1MJF5cimLx9ARAqU9AKCnqCft6TZ4Z77DeMBfdoscRTuuHwCgqyO0
wo3Y4UEiL+1dx5PeaNoxLWU=
=SXRJ
-END PGP SIGNATURE-

Re: Confused about HELO_DYNAMIC_*

[Justin Mason]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1


It's OK to have a dyn IP as the source, alright, as long as it
doesn't HELO using that hostname.   That's what HELO_DYNAMIC_*
matches, as it's a very strong spam signature.

> Received: from h00c04f2d101a.ne.client2.attbi.com 
> (h00c04f2d101a.ne.client2.attbi.com [66.30.139.164]) 
>   by webmail.grad.brandeis.edu (IMP) with HTTP 

So IMP is doing this now?  argh, bad decision...

- --j.

Jay Levitt writes:
> Why would the HELO_DYNAMIC_* rules trigger on these headers?  Surely it's ok 
> to have a dynamic IP as the *source* of a message, 
> just not in a relay..?
> 
> Return-Path: <[EMAIL PROTECTED]>
> Received: from server.home.jay.fm ([unix socket])
>by linux.home.jay.fm (Cyrus v2.2.8) with LMTPA;
>Sun, 27 Feb 2005 23:25:34 -0500
> X-Sieve: CMU Sieve 2.2
> Received: from blanca.unet.brandeis.edu (blanca.unet.brandeis.edu 
> [129.64.99.169])
>   by server.home.jay.fm (8.13.1/8.13.1) with ESMTP id j1S4PWlk011698
>   for <[EMAIL PROTECTED]>; Sun, 27 Feb 2005 23:25:33 -0500
> Received: from blanca.unet.brandeis.edu (localhost.localdomain [127.0.0.1])
>   by blanca.unet.brandeis.edu (8.13.1/8.13.1) with ESMTP id j1S4PUer006126
>   for <[EMAIL PROTECTED]>; Sun, 27 Feb 2005 23:25:32 -0500
> Received: (from [EMAIL PROTECTED])
>   by blanca.unet.brandeis.edu (8.13.1/8.13.1/Submit) id j1S4PUhv006125
>   for [EMAIL PROTECTED]; Sun, 27 Feb 2005 23:25:30 -0500
> Received: from h00c04f2d101a.ne.client2.attbi.com 
> (h00c04f2d101a.ne.client2.attbi.com [66.30.139.164]) 
>   by webmail.grad.brandeis.edu (IMP) with HTTP 
>   for <[EMAIL PROTECTED]>; Sun, 27 Feb 2005 23:25:30 -0500
> Message-ID: <[EMAIL PROTECTED]>
> Date: Sun, 27 Feb 2005 23:25:30 -0500
> From: [EMAIL PROTECTED]
> To: Jay Levitt <[EMAIL PROTECTED]>
> Subject: Re: Wow, now I really don't know what to say
> References: <[EMAIL PROTECTED]>
> In-Reply-To: <[EMAIL PROTECTED]>
> MIME-Version: 1.0
> Content-Type: text/plain; charset=ISO-8859-1
> Content-Transfer-Encoding: 8bit
> User-Agent: Internet Messaging Program (IMP) 3.2.6
> X-Spam-Score: 5.555 (*) 
> BAYES_00,HELO_DYNAMIC_ATTBI,HELO_DYNAMIC_IPADDR,J_CHICKENPOX_21,NO_REAL_NAME
> X-Scanned-By: MIMEDefang 2.43
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Exmh CVS

iD8DBQFCJaQnMJF5cimLx9ARApLgAJ9uN2LFTLd2qQl0TEIp21Jjd/a6ZACfZLd3
3+rMA/b/Nhq635QY9vUnLn4=
=aOCs
-END PGP SIGNATURE-

Re: I don't think the URIDNSBL is working on spams yet

[Justin Mason]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1


Jeff Chan writes:
> On Tuesday, March 1, 2005, 10:26:47 AM, Chris Santerre wrote:
> >>I just upgraded my DNS and URI, URIDNSBL appears to be working 
> >>correctly
> >>now. I'm getting all of the benefits of 3.0.2!
> >>
> >>The URIDNSBL is pure genius, thanks to all who help create and 
> >>support the
> >>SA product.
> 
> > Glad you got it fixed. Believe me, the conference call that started URIDNSBL
> > was my favorite conference call so far :)  I'm guessing only a conference
> > call from the lottery commision and the Ferrari dealership would top it. 
> 
> Minor historical note:  URIDNSBL.pm existed before SURBLs, but it
> contained just one program, uridnsbl, which is the URI domain NS
> record check against sbl.spamhaus.org.  The SpamAssassin developers
> added urirhs* to use SURBLs with URIDNSBL.pm after they saw how
> well it worked, perhaps with SpamCopURI  (That last bit of
> history I'm only guessing.  :-)

nah dude, we did it at the same time ;)   urirhs* was just a bit more
work.

- --j.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Exmh CVS

iD8DBQFCJaPCMJF5cimLx9ARAjbtAJ9EtY6FKMTXan0pTVyefImmQs4yUQCfc84L
g8gU+dZLBVqG600uyO0CgO0=
=O4TA
-END PGP SIGNATURE-

Re: Rule advice please

[Loren Wilton]

> Following discussions on this list about obfuscating words to avoid spam
> detection, and not being a ninja, I'd like some feedback about the
> possible efficacy or pitfalls on rules like the following.
[snip]

In general, there are three main ways of dealing with these obfuscations:
1.  Hand-crafted rules looking for the generally expected variants; usually
on a phrase rather than a word.
2. Chris's Obfu generator that generates and exhaustive (and exhausing, if
you try to read the result ;-) regex to catch just about any variation on
obfuscation on a word or phrase, and
3. Tripwire and related rules that will very often end up triggering pretty
heavily on the more creative obfuscations.
4. And then there is SURBL, that renders all the previous pretty moot after
the first hour or two of a new spam target domain.

As a slight subject change, there is another form of obfuscation in the wild
that tends to escape all except the SURBL test.  I'm actually rather fond of
these spams, since I can always get a good belly laugh from whatever the
spam generator managed to come up with.  I'm guessing that these are
generated by a tool that takes a phrase and then does a thesaurus lookup on
each word, with a *very* creative thesarus.  Below is an edited sample of
one such.  The message appears twice, with modifications.  Once in the text
part of the spam, once in the html part:

-- begin spam 
These pills are only similar normal lozenges but they
are specially formulated to be soft and dissolvable
under the glossa. The tablets is sorbed at the oral fissure
and gets in the bloodstream directly alternatively of rising
through with the tummytum. This effects in a quicker much more
powerful outcome which run up to 35 hours!

Our tablets are simply equal usual lozenges but they 
are specially formulated to be pliant and soluble
below the clapper. The tablets is sorbed at the oral cavity
and gets into the bloodstream straight alternatively of progressing
through with the tummytum. This results in a faster more
strong result which yet up to 39 hours!
 end spam 

Loren

Re: Obfuscation

[Martin Hepworth]

All
nice obsfu generator at..
http://sandgnat.com/cmos/cmos.jsp
--
Martin Hepworth
Snr Systems Administrator
Solid State Logic
Tel: +44 (0)1865 842300
Scott A Crosby wrote:
On Mon, 28 Feb 2005 15:34:13 +, [EMAIL PROTECTED] (Justin Mason) writes:

A paper at the spam conference suggested using an Edit Distance algorithm
with very good results; the idea being, the edit distance from "cialis" to
"C 1 a l | s" isn't as far as it is to "specialized" or so on.
if I recall correctly, someone submitted an implementation quite a while
ago on our BZ, but I think the FP rates were too high.   Given the
recent paper's published results, though, it may be there are good ways
to tweak it to get FPs at a tolerable rate.

I did an implementation of it some time ago, but I didn't get a chance
to take it far enough to test out its effectiveness. I heard remarks
that naively applying edit distance is too slow. To avoid having a FP
rate that was too high, the edit-distance costs are paramaterized, so
some edits are much cheaper than others. Eg.
# Cost of replacing a character with a punctuation in the obfu.
setreps ("bcdfghijklmnpqrstvwxyz","*?.-",.08);
setreps ("aeiou","*?.-",.03);
# Cost to insert these into the obfuscated string is cheap
setins ("/\|()=-'!*`;:?+[]\"^",.01);
setins ("_,.",.01);
So, 'v.agr.' and 'v..ia...gra' both cost <.10  

Got a bugzilla# that I can attach the prototype code to?  (Also, is it
possible to report a bug/attach the code without creating a bugzilla
account?)
Scott
**

This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

This footnote confirms that this email message has been swept
for the presence of computer viruses and is believed to be clean.   

**

Re: Obfuscation

[Justin Mason]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1


Scott A Crosby writes:
> On Mon, 28 Feb 2005 15:34:13 +, [EMAIL PROTECTED] (Justin Mason) writes:
> 
> > A paper at the spam conference suggested using an Edit Distance algorithm
> > with very good results; the idea being, the edit distance from "cialis" to
> > "C 1 a l | s" isn't as far as it is to "specialized" or so on.
> >
> > if I recall correctly, someone submitted an implementation quite a while
> > ago on our BZ, but I think the FP rates were too high.   Given the
> > recent paper's published results, though, it may be there are good ways
> > to tweak it to get FPs at a tolerable rate.
> 
> I did an implementation of it some time ago, but I didn't get a chance
> to take it far enough to test out its effectiveness. I heard remarks
> that naively applying edit distance is too slow. To avoid having a FP
> rate that was too high, the edit-distance costs are paramaterized, so
> some edits are much cheaper than others. Eg.
> 
> # Cost of replacing a character with a punctuation in the obfu.
> setreps ("bcdfghijklmnpqrstvwxyz","*?.-",.08);
> setreps ("aeiou","*?.-",.03);
> 
> # Cost to insert these into the obfuscated string is cheap
> setins ("/\|()=-'!*`;:?+[]\"^",.01);
> setins ("_,.",.01);
> 
> So, 'v.agr.' and 'v..ia...gra' both cost <.10  
> 
> Got a bugzilla# that I can attach the prototype code to?  (Also, is it
> possible to report a bug/attach the code without creating a bugzilla
> account?)

Hi Scott -- looks like there isn't a BZ entry that'd be suitable (at
a glance).  Also, it sounds like it's "new" enough that throwing
it into an existing patch might be confusing anyway.  So I'd say
go ahead and create one.

btw, you don't have a BZ account?  how did you manage *that*! ;)

we do prefer stuff go through BZ, since our CLA-tracking is implemented in
there, and it's the de-facto-standard way for us to find old
conversations, assertions, assumptions, and so on regarding implementation
details -- we can simply say "see bug " in the changelog, and the
entire thread of discussion can then be found with one click from that.
But in an emergency you could always mail it to dev and hopefully some
one will pick it up and create the bug for you.

- --j.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Exmh CVS

iD8DBQFCJY6aMJF5cimLx9ARAg1VAKCbDVxK/02yiEm1ZUFjjU7plLD1TwCfUw8v
K6U9f9DS1SJn6KXgLkidx7s=
=L0Yv
-END PGP SIGNATURE-

Re: another request for RECEIVED[x] array

[List Mail User]

>>From [EMAIL PROTECTED] Tue Mar  1 22:15:46 2005
>Mailing-List: contact [EMAIL PROTECTED]; run by ezmlm
>...
>To: List Mail User <[EMAIL PROTECTED]>
>Cc: [EMAIL PROTECTED], [EMAIL PROTECTED]
>Subject: Re: another request for RECEIVED[x] array
>References: <[EMAIL PROTECTED]>
>In-Reply-To: <[EMAIL PROTECTED]>
>Content-Type: text/plain; charset=ISO-8859-1
>Content-Transfer-Encoding: 7bit
>X-Virus-Checked: Checked
>
>On 3/2/2005 12:33 AM, List Mail User wrote:
>
>> you can do whatever you like, I'd recommend at least doing more than just a
>> simple check for the names being identical
>
>I don't care if the names don't match (although somebody else might). My
>goal with this particular lookup is to weight identifiers that don't have
>any reverse DNS, that is all. Postfix supports this internally already but
>there are way too many FPs for these checks, there are too many possible
>errors (including delegation burps), and so forth. Given that SA is
>supposed to be about managing probabilities ... seems like a good place to
>do the checking is inside SA.
>
>Same thing goes for SMTP versus ESMTP. If they are using HELO they are
>probably malware, and I'd like to be able to test on this.
>
>I'd like to be able to poke at TLS data so that I can assigne a negative
>weight, since functional TLS is a good indicator of a well-managed network
>(not proof, but again, we are talking about probabilities)
>
>
>-- 
>Eric A. Hallhttp://www.ehsco.com/
>Internet Core Protocols  http://www.oreilly.com/catalog/coreprot/
>
Eric,

It all sounds good to me, especially if you go beyond the MTA's
checking of the last relay only and check the full `received' chain;  I
find most of the spam that gets past my first level of filtering does so,
because it targets a high numbered/back-up MX not directly under my control;
Thus my servers, for those cases, see a perfectly acceptable client/EHLO
match (coming from a "trusted" host, though not in the SA sense - it is a
'MX' for my domain(s)), despite an obvious mismatch one level back.  Still,
I personally will leave the Postfix checks in place (I have them skipped
for some accounts, but very few - same thing for the black lists; But my
"extremism" has been a topic on this list before).

Oddly, and probably out of habits over twenty years old, when I
connect to servers "by hand" (which I do quite often), I almost always
type "HELO", not "EHLO".  I have noticed that SMTP scanners usually scan
using HELOs instead of EHLOs and about 1% of my connections get dropped
due to improper pipelining following a HELO (after a EHLO, pipelining is
generally allowed).  I don't generate exactly the stats you might want,
but from my own reports I see ~3.5% of connections dropped voluntarily
(i.e. by the client, not by me) dropped immediately after a connect,
~0.65% after an EHLO and ~1.05% after a HELO;  I would interpret this as
scanners (who do anything beyond checking for a connectable port) are using
HELO over EHLO at about a 3 to 2 ratio:  I am sure that your supposition
about most HELOs being malware is likely correct, I doubt there still exist
many unpatched SunOS servers in production enviroments.

Bye,

Paul Shupak
[EMAIL PROTECTED]

Re: X-Spam-Status/content analysis details inconsistencies.

[jeffrey.arnold]

On Wed, 2 Mar 2005, Alan Premselaar wrote:

::   This type of symptom seems to be common to mail being scanned twice (or
:: more) by spamassasin.  how do you have the call to spamd/spamc implemented?
:: 

Hi Alan,

I am running qmail-1.0.3, and run spamc piped through to qmail-queue by 
replacing the qmail-queue binary with a shell script that does the 
following:

#!/bin/sh
/usr/local/bin/spamc -f -s 1024000 | /var/qmail/bin/qmail-queue.orig

I have spamd logging turned up, and i haven't seen any dupes so far. Any 
clues?

Thanks much,
-jba
__
 [EMAIL PROTECTED] :: analogue.networks.nyc :: http://analogue.net

Re: another request for RECEIVED[x] array

[Eric A. Hall]

On 3/2/2005 12:33 AM, List Mail User wrote:

> you can do whatever you like, I'd recommend at least doing more than just a
> simple check for the names being identical

I don't care if the names don't match (although somebody else might). My
goal with this particular lookup is to weight identifiers that don't have
any reverse DNS, that is all. Postfix supports this internally already but
there are way too many FPs for these checks, there are too many possible
errors (including delegation burps), and so forth. Given that SA is
supposed to be about managing probabilities ... seems like a good place to
do the checking is inside SA.

Same thing goes for SMTP versus ESMTP. If they are using HELO they are
probably malware, and I'd like to be able to test on this.

I'd like to be able to poke at TLS data so that I can assigne a negative
weight, since functional TLS is a good indicator of a well-managed network
(not proof, but again, we are talking about probabilities)


-- 
Eric A. Hallhttp://www.ehsco.com/
Internet Core Protocols  http://www.oreilly.com/catalog/coreprot/

Re: X-Spam-Status/content analysis details inconsistencies.

[Alan Premselaar]

jeffrey.arnold wrote:
Hi users,
I have a weird problem here that i know i am not the only one to 
encounter, and have yet to see (in much searching) a solution for.

I am running spamassassin for all mail via spamd/spamc, and filtering on 
the "X-Spam-Status: Yes" header. The majority of my spam is getting 
caught, but quite a bit is getting through. The vast majority of the 
spam that gets through is tagged as spam in the subject line (i enable 
rewrite_header), but not tagged as spam in the X-Spam-Status header 
line. In an example spam, i get the following spamass header:

X-Spam-Checker-Version: SpamAssassin 3.0.2 (2004-11-16) on
phase.nyc.analogue.net
X-Spam-Level: 
X-Spam-Status: No, score=4.8 required=5.0 tests=BAYES_50,HTML_10_20,
HTML_MESSAGE,RAZOR2_CF_RANGE_51_100,RAZOR2_CHECK,URIBL_SBL
autolearn=no version=3.0.2

The subject/body, on the other hand, contains:
Subject: SPAM(10.9) Become the man that women desire
Content analysis details:   (10.9 points, 5.0 required)
...snip...
So it appears that spamassassin tagged this as spam with 10.9 points, 
but left out some of the rules when writing the X-Spam-Status header, 
claiming the email only received 4.8 points. Spamassassin is installed 
directly out of freebsd ports on a fbsd4.11 box with perl 5.8.5. My 
local.cf is included below. Why would spamassassin be dropping rules 
like this?

Thanks much,
-jba
Jeffrey,
  This type of symptom seems to be common to mail being scanned twice 
(or more) by spamassasin.  how do you have the call to spamd/spamc 
implemented?

alan

Re: another request for RECEIVED[x] array

[List Mail User]

>>From [EMAIL PROTECTED] Tue Mar  1 18:30:49 2005
>Date: Tue, 01 Mar 2005 21:30:33 -0500
>From: "Eric A. Hall" <[EMAIL PROTECTED]>
>User-Agent: Mozilla Thunderbird 1.0 (Windows/20041206)
>X-Accept-Language: en-us, en
>MIME-Version: 1.0
>To: List Mail User <[EMAIL PROTECTED]>
>Cc: [EMAIL PROTECTED], [EMAIL PROTECTED]
>Subject: Re: another request for RECEIVED[x] array
>References: <[EMAIL PROTECTED]>
>In-Reply-To: <[EMAIL PROTECTED]>
>Content-Type: text/plain; charset=ISO-8859-1
>Content-Transfer-Encoding: 7bit
>
>On 3/1/2005 8:39 PM, List Mail User wrote:
>
>>  Please note that is section 4.1.4 of RFC2821 it says:
>
>> An SMTP server MAY verify that the domain name parameter in the EHLO
>>command actually corresponds to the IP address of the client.
>>However, the server MUST NOT refuse to accept a message for this
>>reason if the verification fails: the information about verification
>>failure is for logging and tracing only.
>
>Exactly right, and this is one of the reasons that I do indeed accept the
>mail, and I have even made this same argument myself many times when the
>subject comes up.
>
>OTOH, none of the RFCs say anything about (and have no jurisdiction over)
>whatever weighting I might assign such events locally.
>
>That's the canonical reason for why I want to do most of these tests to SA
>in the first place.
>
>-- 
>Eric A. Hallhttp://www.ehsco.com/
>Internet Core Protocols  http://www.oreilly.com/catalog/coreprot/
>
Eric,

In that case, I see nothing wrong with your desire for rules to
check this;  In many environments, you could probably even give a fairly
high score to non-matching of the client and HELO/EHLO.  Personally, I
find that for my site, just for today, almost 45% of the connections had
no rDNS for either the client or had none for the HELO/EHLO (looking back
two weeks shows much higher number for missing rDNS, about 65%) and were
refused on that basis, another 10% were on various blacklists and also were
refused - of the remainder, a quick check shows that only about 65% matched,
but over 75% of the accepted mail was from mailing lists; Of the remainder
from individuals, it's running ~50% for exact matches, but nearly 95% if
you allow CNAME matches, MX records and IP aliases.  I would have no problem
with my traffic with a score on the order of 2.0 or less (and probably be
safe even with a much higher score) - and you are indeed free to do whatever
suits you and your environment (which certainly is quite different than mine).

I know I gave one example in my previous message, but another I see
from the past week is a very common poster to this list, Jeff C. of SURBL;
His mail comes through a server which EHLO's with smtp1.supranet.net, but
resolves to horse.supranet.net - In this case again both of the names again
resolve to the same IP address, but here they are true aliases - i.e. both
hostnames have 'A' records for the same IP, neither is a CNAME.  Again, while
you can do whatever you like, I'd recommend at least doing more than just a
simple check for the names being identical (how complex you want to make it
is, of course, a matter of both how much time you want to spend coding and
how many DNS queries you feel are reasonable).

Good luck and I really like to see whatever you come up with,

Paul Shupak
[EMAIL PROTECTED]

X-Spam-Status/content analysis details inconsistencies.

[jeffrey.arnold]

Hi users,
I have a weird problem here that i know i am not the only one to 
encounter, and have yet to see (in much searching) a solution for.

I am running spamassassin for all mail via spamd/spamc, and filtering on 
the "X-Spam-Status: Yes" header. The majority of my spam is getting 
caught, but quite a bit is getting through. The vast majority of the 
spam that gets through is tagged as spam in the subject line (i enable 
rewrite_header), but not tagged as spam in the X-Spam-Status header 
line. In an example spam, i get the following spamass header:

X-Spam-Checker-Version: SpamAssassin 3.0.2 (2004-11-16) on
phase.nyc.analogue.net
X-Spam-Level: 
X-Spam-Status: No, score=4.8 required=5.0 tests=BAYES_50,HTML_10_20,
HTML_MESSAGE,RAZOR2_CF_RANGE_51_100,RAZOR2_CHECK,URIBL_SBL
autolearn=no version=3.0.2

The subject/body, on the other hand, contains:
Subject: SPAM(10.9) Become the man that women desire
Content analysis details:   (10.9 points, 5.0 required)
 pts rule name  description
 -- --
 0.0 HTML_MESSAGE   BODY: HTML included in message
 0.0 BAYES_50   BODY: Bayesian spam probability is 40 to 60%
[score: 0.5508]
 0.1 RAZOR2_CF_RANGE_51_100 BODY: Razor2 gives confidence level above
50%
[cf: 100]
 0.2 HTML_10_20 BODY: Message is 10% to 20% HTML
 0.2 MIME_HTML_ONLY BODY: Message only has text/html MIME parts
 3.5 RAZOR2_CHECK   Listed in Razor2 (http://razor.sf.net/)
 2.0 RCVD_IN_SORBS_DUL  RBL: SORBS: sent directly from dynamic IP
address
[218.80.35.217 listed in dnsbl.sorbs.net]
 3.8 RCVD_IN_DSBL   RBL: Received via a relay in list.dsbl.org
[]
 0.1 RCVD_IN_NJABL_DUL  RBL: NJABL: dialup sender did non-local SMTP
[218.80.35.217 listed in combined.njabl.org]
 1.0 URIBL_SBL  Contains an URL listed in the SBL blocklist
[URIs: heavenlyitemsbutique.com]
So it appears that spamassassin tagged this as spam with 10.9 points, 
but left out some of the rules when writing the X-Spam-Status header, 
claiming the email only received 4.8 points. Spamassassin is installed 
directly out of freebsd ports on a fbsd4.11 box with perl 5.8.5. My 
local.cf is included below. Why would spamassassin be dropping rules 
like this?

Thanks much,
-jba
__
 [EMAIL PROTECTED] :: analogue.networks.nyc :: http://analogue.net
-- local.cf --
rewrite_header Subject SPAM(_SCORE_)
skip_rbl_checks 0
use_razor2 1
bayes_auto_learn 1
use_bayes 1
use_pyzor 1
auto_learn 1
use_dcc 1
dcc_add_header 1
dns_available yes
dcc_timeout 10
dcc_path /my/path/to/dccproc
rbl_timeout 3
num_check_received 3
bayes_path /path/to/my/spamass/
auto_whitelist_path /path/to/my/auto-whitelist
header   ARIX_DF rbleval:check_rbl('arix-df', 'fresh.dict.rbl.arix.com.')
describe ARIX_DF Recent dictionary spammer
tflags   ARIX_DF net
header   ARIX_DS rbleval:check_rbl('arix-ds', 'stale.dict.rbl.arix.com.')
describe ARIX_DS Sender has a history of dictionary spamming
tflags   ARIX_DS net
scoreARIX_DF 3.0
scoreARIX_DS 0.5
trusted_networks (my trusted nets)
internal_networks (my internal nets)
lock_method flock
## Optional Score Increases
score DCC_CHECK 4.000
score RAZOR2_CHECK 3.500
score BAYES_99 4.300
score BAYES_90 3.500
score BAYES_80 3.000
score RCVD_IN_SORBS_WEB 2.000
score RCVD_IN_SBL 3.000
score URIBL_SBL 1.00
score ALL_TRUSTED 0.000
score RCVD_IN_BL_SPAMCOP_NET 2.0

Re: Confused about HELO_DYNAMIC_*

[Matt Kettler]

At 10:43 PM 3/1/2005, Jay Levitt wrote:
Why would the HELO_DYNAMIC_* rules trigger on these headers?  Surely it's 
ok to have a dynamic IP as the *source* of a message, just not in a relay..?
It looks like it might be a trust path issue.. are the brandeis.edu hosts 
trusted? If so, SA would be correct in deciding a dynamic node from 
attbi.com dropped mail off directly.

What do the *.home.jay.fm hosts resolve as when the machine running SA does 
a DNS lookup? are they reserved IP's? If so, you'll have trust path issues 
and need to manualy define trusted_networks.

Re: ASCII-Art like spam?!

[Robert Menschel]

Hello Nick,

Tuesday, March 1, 2005, 8:04:31 AM, you wrote:

NB> Attached are two spams I got in the last two days, jebsu! ASCII
NB> ART SPAM!

You need http://www.rulesemporium.com/rules/99_FVGT_Tripwire.cf

First one scored 14 just on this one rules file.

Bob Menschel

Confused about HELO_DYNAMIC_*

[Jay Levitt]

Why would the HELO_DYNAMIC_* rules trigger on these headers?  Surely it's ok to have a dynamic IP as the *source* of a message, 
just not in a relay..?

Return-Path: <[EMAIL PROTECTED]>
Received: from server.home.jay.fm ([unix socket])
	 by linux.home.jay.fm (Cyrus v2.2.8) with LMTPA;
	 Sun, 27 Feb 2005 23:25:34 -0500
X-Sieve: CMU Sieve 2.2
Received: from blanca.unet.brandeis.edu (blanca.unet.brandeis.edu [129.64.99.169])
	by server.home.jay.fm (8.13.1/8.13.1) with ESMTP id j1S4PWlk011698
	for <[EMAIL PROTECTED]>; Sun, 27 Feb 2005 23:25:33 -0500
Received: from blanca.unet.brandeis.edu (localhost.localdomain [127.0.0.1])
	by blanca.unet.brandeis.edu (8.13.1/8.13.1) with ESMTP id j1S4PUer006126
	for <[EMAIL PROTECTED]>; Sun, 27 Feb 2005 23:25:32 -0500
Received: (from [EMAIL PROTECTED])
	by blanca.unet.brandeis.edu (8.13.1/8.13.1/Submit) id j1S4PUhv006125
	for [EMAIL PROTECTED]; Sun, 27 Feb 2005 23:25:30 -0500
Received: from h00c04f2d101a.ne.client2.attbi.com (h00c04f2d101a.ne.client2.attbi.com [66.30.139.164]) 
	by webmail.grad.brandeis.edu (IMP) with HTTP 
	for <[EMAIL PROTECTED]>; Sun, 27 Feb 2005 23:25:30 -0500
Message-ID: <[EMAIL PROTECTED]>
Date: Sun, 27 Feb 2005 23:25:30 -0500
From: [EMAIL PROTECTED]
To: Jay Levitt <[EMAIL PROTECTED]>
Subject: Re: Wow, now I really don't know what to say
References: <[EMAIL PROTECTED]>
In-Reply-To: <[EMAIL PROTECTED]>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
User-Agent: Internet Messaging Program (IMP) 3.2.6
X-Spam-Score: 5.555 (*) BAYES_00,HELO_DYNAMIC_ATTBI,HELO_DYNAMIC_IPADDR,J_CHICKENPOX_21,NO_REAL_NAME
X-Scanned-By: MIMEDefang 2.43

Re: another request for RECEIVED[x] array

[Eric A. Hall]

On 3/1/2005 8:39 PM, List Mail User wrote:

>   Please note that is section 4.1.4 of RFC2821 it says:

> An SMTP server MAY verify that the domain name parameter in the EHLO
>command actually corresponds to the IP address of the client.
>However, the server MUST NOT refuse to accept a message for this
>reason if the verification fails: the information about verification
>failure is for logging and tracing only.

Exactly right, and this is one of the reasons that I do indeed accept the
mail, and I have even made this same argument myself many times when the
subject comes up.

OTOH, none of the RFCs say anything about (and have no jurisdiction over)
whatever weighting I might assign such events locally.

That's the canonical reason for why I want to do most of these tests to SA
in the first place.

-- 
Eric A. Hallhttp://www.ehsco.com/
Internet Core Protocols  http://www.oreilly.com/catalog/coreprot/

Re: another request for RECEIVED[x] array

[List Mail User]

>...
>List-Id: 
>Delivered-To: mailing list users@spamassassin.apache.org
>Delivered-To: [EMAIL PROTECTED]
>...
>Date: Tue, 01 Mar 2005 19:32:22 -0500
>From: "Eric A. Hall" <[EMAIL PROTECTED]>
>User-Agent: Mozilla Thunderbird 1.0 (Windows/20041206)
>X-Accept-Language: en-us, en
>MIME-Version: 1.0
>To: Justin Mason <[EMAIL PROTECTED]>
>Cc: [EMAIL PROTECTED]
>Subject: Re: another request for RECEIVED[x] array
>References: <[EMAIL PROTECTED]>
>In-Reply-To: <[EMAIL PROTECTED]>
>Content-Type: text/plain; charset=ISO-8859-1
>Content-Transfer-Encoding: 7bit
>X-Virus-Checked: Checked
>
>
>On 3/1/2005 12:37 PM, Justin Mason wrote:
>
>> actually, there is such a thing in SpamAssassin 3.0.x ;)  e.g.:
>
>> debug: metadata: X-Spam-Relays-Untrusted: [ ip=199.172.62.20
>> rdns=europe.std.com helo=europe.std.com by=mail.netnoteinc.com ident=
>> envfrom= intl=0 id=392E1114061 auth= ]
>
>This doesn't record the 'with' parameter (as in "with SMTP" or "with
>ESMTP"), or any TLS info.
>
>> I want to have available to me are:
>>> 
>>> 1) Check for a reverse-DNS match
>>> 
>>> 2) Check for HELO (versus EHLO)
>>> 
>>> 3) Check for TLS
>
>I already have a reverse-DNS rule coded up but I'd like to consolidate it
>with the other two, but again, I really need a better view into the
>transfer protocol from inside SA.
>
>-- 
>Eric A. Hallhttp://www.ehsco.com/
>Internet Core Protocols  http://www.oreilly.com/catalog/coreprot/
>

Eric,

Please note that is section 4.1.4 of RFC2821 it says:

"...
An SMTP server MAY verify that the domain name parameter in the EHLO
   command actually corresponds to the IP address of the client.
   However, the server MUST NOT refuse to accept a message for this
   reason if the verification fails: the information about verification
   failure is for logging and tracing only.
..."

The capitalization is in the original, not added. Reference from:
ftp://ftp.isi.edu/in-notes/rfc2821.txt page 38, 6th paragraph of section 4.1.4

On the other hand, both sendmail and postfix can be configured to
refuse messages if there is no rDNS at all for the HELO/EHLO or if the
client address doesn't have any rDNS.  I *would* recommend these, but
since the most common MTAs already have the capability, there shouldn't
be any need for SA to duplicate it.  Also, the number of FPs if you attempt
to enforce an exact match would be *very* high;  Whenever a MTA hosts multiple
domains, only one could possibly pass the test - but it is reasonable for
any host which is an 'MX' for a domain to use that domain as an argument to
the HELO/EHLO (and possibly - definititely allowed - for any host in a domain
to use the domain name as the argument to the HELO/EHLO) .  Also, note that
an 'A' record isn't required (though it is recommended) - see section 4.1.1.1
and note that both sendmail and postfix will accept a "resolvable" domain as
defined there (i.e. at least one 'MX' or 'A' record, and it need not match
the actual client's IP address).

You also have the problem of dealing with IP literals, and users
running dynamic DNS which still has stale DNS data (so the response should
be a 4xx code not a 5xx code, if you do something like this).  If you use
an SA rule, the score should be fairly low.  As an example, recent mail I
have exchanged with [EMAIL PROTECTED] has used escalate.hk.outblaze.com
as the argument to EHLO, but the rDNS resolves to escalate.outblaze.com, and
they *certainly* are not spammers (notice also, that both names resolve in
different ways, but to the same IP address).

Paul Shupak
[EMAIL PROTECTED]

Re: I don't think the URIDNSBL is working on spams yet

[Jeff Chan]

On Tuesday, March 1, 2005, 10:26:47 AM, Chris Santerre wrote:
>>I just upgraded my DNS and URI, URIDNSBL appears to be working 
>>correctly
>>now. I'm getting all of the benefits of 3.0.2!
>>
>>The URIDNSBL is pure genius, thanks to all who help create and 
>>support the
>>SA product.

> Glad you got it fixed. Believe me, the conference call that started URIDNSBL
> was my favorite conference call so far :)  I'm guessing only a conference
> call from the lottery commision and the Ferrari dealership would top it. 

Minor historical note:  URIDNSBL.pm existed before SURBLs, but it
contained just one program, uridnsbl, which is the URI domain NS
record check against sbl.spamhaus.org.  The SpamAssassin developers
added urirhs* to use SURBLs with URIDNSBL.pm after they saw how
well it worked, perhaps with SpamCopURI  (That last bit of
history I'm only guessing.  :-)

Cheers,

Jeff C.
-- 
Jeff Chan
mailto:[EMAIL PROTECTED]
http://www.surbl.org/

Re: Obfuscation

[Scott A Crosby]

On Mon, 28 Feb 2005 15:34:13 +, [EMAIL PROTECTED] (Justin Mason) writes:

> A paper at the spam conference suggested using an Edit Distance algorithm
> with very good results; the idea being, the edit distance from "cialis" to
> "C 1 a l | s" isn't as far as it is to "specialized" or so on.
>
> if I recall correctly, someone submitted an implementation quite a while
> ago on our BZ, but I think the FP rates were too high.   Given the
> recent paper's published results, though, it may be there are good ways
> to tweak it to get FPs at a tolerable rate.

I did an implementation of it some time ago, but I didn't get a chance
to take it far enough to test out its effectiveness. I heard remarks
that naively applying edit distance is too slow. To avoid having a FP
rate that was too high, the edit-distance costs are paramaterized, so
some edits are much cheaper than others. Eg.

# Cost of replacing a character with a punctuation in the obfu.
setreps ("bcdfghijklmnpqrstvwxyz","*?.-",.08);
setreps ("aeiou","*?.-",.03);

# Cost to insert these into the obfuscated string is cheap
setins ("/\|()=-'!*`;:?+[]\"^",.01);
setins ("_,.",.01);

So, 'v.agr.' and 'v..ia...gra' both cost <.10  


Got a bugzilla# that I can attach the prototype code to?  (Also, is it
possible to report a bug/attach the code without creating a bugzilla
account?)

Scott

Re: another request for RECEIVED[x] array

[Eric A. Hall]

On 3/1/2005 12:37 PM, Justin Mason wrote:

> actually, there is such a thing in SpamAssassin 3.0.x ;)  e.g.:

> debug: metadata: X-Spam-Relays-Untrusted: [ ip=199.172.62.20
> rdns=europe.std.com helo=europe.std.com by=mail.netnoteinc.com ident=
> envfrom= intl=0 id=392E1114061 auth= ]

This doesn't record the 'with' parameter (as in "with SMTP" or "with
ESMTP"), or any TLS info.

> I want to have available to me are:
>> 
>> 1) Check for a reverse-DNS match
>> 
>> 2) Check for HELO (versus EHLO)
>> 
>> 3) Check for TLS

I already have a reverse-DNS rule coded up but I'd like to consolidate it
with the other two, but again, I really need a better view into the
transfer protocol from inside SA.

-- 
Eric A. Hallhttp://www.ehsco.com/
Internet Core Protocols  http://www.oreilly.com/catalog/coreprot/