SPF for avoiding newsletter FPs

2006-04-24 Thread Ramprasad
Hi,
  We get considerable number of newsletter mails with spammy content.
How do people tackle Fp's from newsletters ? typically the stock
newsletters , the bank promotional newsletters etc 
I would like know if this is possible ( I am using SA3.1 + Mailscanner +
postfix ) 
1) Maintain a list of newsletters ( this would grow with time )
2) For each of these newlsetter mails if their SPF records match give a
high negative score. 

Atleast those newsletters from domains who *have* SPF records will not
have problems.

Thanks
Ram





Charity spam - is this a new kind of 419?

2006-04-24 Thread Peter Campion-Bye

Received the message below at the weekend. I could be completely wrong and
this is a genuine misguided attempt at recruiting charity workers, but it
looks to me like a new kind of 419 scam - if you show an interest I suspect
they will want bank account details and/or money up front.
Suspicious that she can't even decide how to spell her own surname!


Return-Path: [EMAIL PROTECTED]
Received: from mx0.pandasys.net (mx0.pandasys.net [81.187.228.199])
 by newpennan.pandasys.net (8.13.6/8.13.4) with ESMTP id k3N1AJGU014087
 for [EMAIL PROTECTED]; Sun, 23 Apr 2006 02:10:19 +0100
Received: from phpnet.org (lb.phpnet.org [87.98.197.87])
 by mx0.pandasys.net (8.13.6/8.13.6) with SMTP id k3N1AHjL004631
 for [EMAIL PROTECTED]; Sun, 23 Apr 2006 02:10:17 +0100
Received: (qmail 20821 invoked by uid 89); 23 Apr 2006 01:03:15 -
Received: from unknown (HELO nobody.nothing.phpnet.org) (10.0.0.42)
 by phpnet.org with SMTP; 23 Apr 2006 01:03:15 -
Received: (qmail 10793 invoked by uid 500); 23 Apr 2006 01:03:01 -
Date: 23 Apr 2006 01:03:01 -
Message-ID: [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
ScriptPath: mastacrew.com/page.php
Subject: Charity Work
From: Save the Children Charity Work [EMAIL PROTECTED]
Reply-To: [EMAIL PROTECTED]
MIME-Version: 1.0
Content-Type: text/html
Content-Transfer-Encoding: 8bit
X-Spam-Score: 2.52
BAYES_50,HTML_00_10,HTML_MESSAGE,HTML_MIME_NO_HTML_TAG,MIME_HTML_ONLY,SARE_HTML_EMPTY,SPF_HELO_PASS
X-Scanned-By: MIMEDefang 2.54 on 81.187.228.199

-- Original Message --
Subject: Charity Work
From:Save the Children Charity Work [EMAIL PROTECTED]
Date:Sun, April 23, 2006 02:03
To:  [EMAIL PROTECTED]
--


 Hello,
   I am Helen from Save the Children Charity Work.Save the children is a child
charity that works in the uk and worldwide. find out
 how you can do volunteering, fundraising and make a donation.
   We are presently looking for people from United Kingdom,United
States,Canada,Australia and Ireland who can work online with our Branch in
Africa.We are willing to make arranging for payment on everyone who is
ready to part-take under this umbrella of our Charity Work(Save The
Children).
We want to make sure that Children are safe and secured from every bad
diseases occuring around the world now, and this Organization will be
making payment for everybody working under it but it depends on how many
people you can bring into this Organization.
Payment for single/new person who just join this Save the Children Health
Organization is 400pounds per week and the payment will be made in
cheque/money order or directly into your account everyweek as a part of
this Organization.
 We are pleased to welcome you as a member of this Children Health
Organization which is made for schools and everybody in the world can
part-take as member because we need just 20 more people to be member/workers
of this Organization and this Organization need people who can make
themselves avaliable at least twice a week for the work because we may need
any member to reach places where help is needed.
   I hope this is more comprehensive and you are highly welcome to be a
member/worker under this Children Health Organization.
 You can contact the Ass. Coordinator for more informations through this
mailto: [EMAIL PROTECTED]
 We are very pleased to invite you to part-take as a member/worker in this
Children Health Organization and you read more from our other branch website
under united States (www.savethechildren.org)
 Thanks
   Mrs Helen Cockran
   Ass. Coordinator
   NB: mailto: [EMAIL PROTECTED]



Re: Adding headers from SQL userprefs

2006-04-24 Thread mouss

Mike Galvez wrote:

Hello,

I have searched the archives, but I can't find an answer to why I
can't add headers such as X-Spam-Score.

I'm using SpamAssassin 3.1.1, Sendmail and spamass-milter-0.3.0_1

My local.cf has:

add_header all Score _SCORE_
version=_VERSION_
add_header spam Flag _YESNOCAPS_
report_safe 2
lock_method flock
required_score 5.5
use_bayes 1
bayes_auto_learn 1
bayes_ignore_header X-Bogosity
bayes_ignore_header X-Spam-Flag
bayes_ignore_header X-Spam-Status


In the headers, I get:

X-Spam-Level: *
X-Spam-Status: Yes, score=9.2 required=5.5 tests=AWL=-0.591,
DRUGS_ERECTILE=0.1,DRUGS_ERECTILE_OBFU=2.046,GAPPY_SUBJECT=1.625,
MANY_EXCLAMATIONS=0,PLING_PLING=0.461,SUBJECT_DRUG_GAP_C=2.88,
SUBJECT_FUZZY_VPILL=1.644,UPPERCASE_75_100=1.04 autolearn=no 
version=3.1.1

X-Spam-Flag: YES
X-Spam-Checker-Version: SpamAssassin 3.1.1 (2006-03-10) on

But no X-Spam-Score

  
Did you try it manually (spamassassin -t  message.eml)? if it works 
manually, then you probably need to restart something for the change to 
take effect.


I am pulling in user_pref rules from an sql database. Using debug, 
I know it's checking the sql table and parsing rules. The only pref

in the sql table that affects the header is:
rewrite_header Subject [SPAM-_HITS_]- 


I appreciate any help you can provide.

Thanks

  




URI Basics

2006-04-24 Thread Dan Patnode

Another Newbie question here,

So IRIs find links in the body.  I'm trying to get a handle on URI  
syntax and have found several disparate examples:



1) uri HTTP_CTRL_CHARS_HOST		/^https?\:\/\/[^\/\s]*[\x00-\x08\x0b\x0c 
\x0e-\x1f]/


2) uri NORMAL_HTTP_TO_IPm{^https?://\d+\.\d+\.\d+\.\d+}i

3) uri URI_4YOU [EMAIL 
PROTECTED](?:https?://|mailto:)[^\/[EMAIL PROTECTED]

4) uri HTTP_77  /http:\/\/.{0,2}\%77/

5) uri BARGAIN_URL  /bargain([sz]|-\S+)?\.(?:com|biz)/

6) uri URI_OFFERS   m/offer([sz]|-\S+)?\.(?:com|bi?z)/i

7) uri URI_AFFILIATE/aff\w+id=/i


I have a few questions and welcome other tips.  What do m{, m/, and  
m@ mean?  Are m||, m(), and m{} interchangeable or does each mean  
something different?  Does it matter if the ^ is on the outside (3)  
or the inside (12) of the beginning?  I see the value of URIs with  
5-7 so an anchor is not needed, is there an improvement over rawbody  
when http is used as in 1-4?


Thanks,
Dan


Re: Adding headers from SQL userprefs

2006-04-24 Thread Mike Galvez
On Sun, Apr 23, 2006 at 11:23:49AM +0200, mouss wrote:
 Mike Galvez wrote:
 Hello,
 
 I have searched the archives, but I can't find an answer to why I
 can't add headers such as X-Spam-Score.
 
 I'm using SpamAssassin 3.1.1, Sendmail and spamass-milter-0.3.0_1
 
 My local.cf has:
 
 add_header all Score _SCORE_
 version=_VERSION_
 add_header spam Flag _YESNOCAPS_
 report_safe 2
 lock_method flock
 required_score 5.5
 use_bayes 1
 bayes_auto_learn 1
 bayes_ignore_header X-Bogosity
 bayes_ignore_header X-Spam-Flag
 bayes_ignore_header X-Spam-Status
 
 
 In the headers, I get:
 
 X-Spam-Level: *
 X-Spam-Status: Yes, score=9.2 required=5.5 tests=AWL=-0.591,
 DRUGS_ERECTILE=0.1,DRUGS_ERECTILE_OBFU=2.046,GAPPY_SUBJECT=1.625,
 MANY_EXCLAMATIONS=0,PLING_PLING=0.461,SUBJECT_DRUG_GAP_C=2.88,
 SUBJECT_FUZZY_VPILL=1.644,UPPERCASE_75_100=1.04 autolearn=no 
 version=3.1.1
 X-Spam-Flag: YES
 X-Spam-Checker-Version: SpamAssassin 3.1.1 (2006-03-10) on
 
 But no X-Spam-Score
 
   
 Did you try it manually (spamassassin -t  message.eml)? if it works 
 manually, then you probably need to restart something for the change to 
 take effect.

Thanks for the reply and suggestion. 

Using cat spam1.txt | spamc -u username, I see that the X-Spam-Score is 
written:


From: Creativity Courses [EMAIL PROTECTED] [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Subject: [SPAM-13.6]- $100 off Early Registration Promotion expires soon 
Date: Tue, 10 Jan 2006 12:46:57 + (GMT)
Message-Id: [EMAIL PROTECTED]
X-Spam-Score: 13.6
X-Spam-Flag: YES
X-Spam-Checker-Version: SpamAssassin 3.1.1 (2006-03-10) on 

X-Spam-Level: *
X-Spam-score: 13.6
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary=--=_444BA993.2FBD218F


Sending from another domain I see that Milter is not adding the X-Spam-Score 
header

sm-mta[42788]: k3NHW0wP042788: Milter add: header: X-Spam-Flag: YES
sm-mta[42788]: k3NHW0wP042788: Milter add: header: X-Spam-Level: **
sm-mta[42788]: k3NHW0wP042788: Milter add: header: X-Spam-Checker-Version: 
SpamAssassin 3.1.1 (2006-03-10) on
sm-mta[42788]: k3NHW0wP042788: Milter change: header Subject: from Buy Them 
Now! to [SPAM-10.5]- Buy Them Now!
sm-mta[42788]: k3NHW0wP042788: Milter add: header: Content-Type: 
multipart/mixed; boundary=--=_444BBA16.97D41A46
sm-mta[42788]: k3NHW0wP042788: Milter message: body replaced


 
 I am pulling in user_pref rules from an sql database. Using debug, 
 I know it's checking the sql table and parsing rules. The only pref
 in the sql table that affects the header is:
 rewrite_header Subject [SPAM-_HITS_]- 
 
 I appreciate any help you can provide.
 
 Thanks
 
   
 

-- 
Michael Galvez


Re: sa-learn not learning with sudo

2006-04-24 Thread Chr. v. Stuckrad
On Sat, Apr 22, 2006 at 10:55:29AM +0200, Michael Monnerie wrote:
...
 # sudo -H -u vscan sa-learn --dump
...
 But when I do
 # su -l vscan
...
 # sudo -H -u vscan sa-learn --dump
...
 Now why is there a diff between sudo as a user or directly logging in as 

One of the differences will be all the commands in the
User's shell-startup-Files!  Those are ignored, if you
run the command directly by sudo.

It also depends on the version of 'sudo', because one
of the latest changes *dropped* the HOME-Variable
from the environment (at least if you run the command
directly from sudo!).

Lots of our automated cron-scripts suddenly failed
by this 'security fix' and we had to replace
OLD:  sudo command
NEW:  sudo env HOME=$HOME command
to 'bridge the gap' and re-use the *current* HOME
'inside of sudo'.

May be the 'sudo -l vscan' also sets the missing HOME!

YoursStucki   (postmaster hit by the same? :-)



Re: Charity spam - is this a new kind of 419?

2006-04-24 Thread Craig McLean
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Peter Campion-Bye wrote:
 Received the message below at the weekend. I could be completely wrong and
 this is a genuine misguided attempt at recruiting charity workers, but it
 looks to me like a new kind of 419 scam - if you show an interest I suspect
 they will want bank account details and/or money up front.
 Suspicious that she can't even decide how to spell her own surname!
 
[snip headers]
 
  Hello,
I am Helen from Save the Children Charity Work.Save the children is a child
 charity that works in the uk and worldwide. find out
  how you can do volunteering, fundraising and make a donation.
We are presently looking for people from United Kingdom,United
 States,Canada,Australia and Ireland who can work online with our Branch in
 Africa.We are willing to make arranging for payment on everyone who is
 ready to part-take under this umbrella of our Charity Work(Save The
 Children).
 We want to make sure that Children are safe and secured from every bad
 diseases occuring around the world now, and this Organization will be
 making payment for everybody working under it but it depends on how many
 people you can bring into this Organization.
 Payment for single/new person who just join this Save the Children Health
 Organization is 400pounds per week and the payment will be made in
 cheque/money order or directly into your account everyweek as a part of
 this Organization.
  We are pleased to welcome you as a member of this Children Health
 Organization which is made for schools and everybody in the world can
 part-take as member because we need just 20 more people to be member/workers
 of this Organization and this Organization need people who can make
 themselves avaliable at least twice a week for the work because we may need
 any member to reach places where help is needed.
I hope this is more comprehensive and you are highly welcome to be a
 member/worker under this Children Health Organization.
  You can contact the Ass. Coordinator for more informations through this
 mailto: [EMAIL PROTECTED]
  We are very pleased to invite you to part-take as a member/worker in this
 Children Health Organization and you read more from our other branch website
 under united States (www.savethechildren.org)
  Thanks
Mrs Helen Cockran
Ass. Coordinator
NB: mailto: [EMAIL PROTECTED]

Smells like 419 to me, given (among other things) the level of literacy
displayed.  If you have no objections I'll drop the sender a line and
see what the scam is...

C.

- --
Craig McLeanhttp://fukka.co.uk
[EMAIL PROTECTED]   Where the fun never starts
Powered by FreeBSD, and GIN!
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.3 (GNU/Linux)

iD8DBQFETMenMDDagS2VwJ4RAl6/AKD4yjnzQRvWCe0L6Q5zgWBCy/8tRQCgtx5R
7vLF9MtcUV9eokJxU1uVt3s=
=LWPB
-END PGP SIGNATURE-


Re: URI Basics

2006-04-24 Thread Matt Kettler
Dan Patnode wrote:
 Another Newbie question here,

 So IRIs find links in the body.  I'm trying to get a handle on URI
 syntax and have found several disparate examples:


 1) uri HTTP_CTRL_CHARS_HOST   
 /^https?\:\/\/[^\/\s]*[\x00-\x08\x0b\x0c\x0e-\x1f]/

 2) uri NORMAL_HTTP_TO_IPm{^https?://\d+\.\d+\.\d+\.\d+}i

 3) uri URI_4YOU[EMAIL PROTECTED](?:https?://|mailto:)[^\/[EMAIL 
 PROTECTED]

 4) uri HTTP_77/http:\/\/.{0,2}\%77/

 5) uri BARGAIN_URL/bargain([sz]|-\S+)?\.(?:com|biz)/

 6) uri URI_OFFERSm/offer([sz]|-\S+)?\.(?:com|bi?z)/i

 7) uri URI_AFFILIATE/aff\w+id=/i


 I have a few questions and welcome other tips.  What do m{, m/, and m@
 mean?  
Those are the match operator.. It's basically used so you can use
something other than / to delimit the start and end of your regex. It is
very common to do this for URIs so you can do http:// instead of having
to escape it into http:\/\/, as in example 4.

Why example 6 uses m/ is beyond me, as / is the default.

 Are m||, m(), and m{} interchangeable or does each mean something
 different?  
Interchangeable
 Does it matter if the ^ is on the outside (3) or the inside (12) of
 the beginning?
In 3 ^ is the first character of the regex, just as it is in 1 and 2. It
is also inside the delimiters, just like 1 and 2. In example 3 @ is
being used as a delimiter,  and ^ is the first character after it. You
can't put a ^ outside your delimiter and have it act as an anchor.
 I see the value of URIs with 5-7 so an anchor is not needed,
I don't believe the use of anchors is a significant performance penalty.
In general, they may actually cause a rule to run faster than one
without. That said, make your choice about anchors based on accuracy
needs, not performance.
 is there an improvement over rawbody when http is used as in 1-4? 

There is definitely a VERY significant performance penalty to using
rawbody over URI, for any rule.

Consider the size of input. A rawbody regex must be run against the
entire text of the body after QP decoding. A uri regex must be run
against all the text of the URIs that SA found. There is likely to be at
least a 100:1 difference in size of input. There's no penalty for
using a uri rule, as SA will always extract all the URIs and build the
input text, even if you aren't using it.

However, there are some cases where rawbody is useful, particularly when
you want to examine the formatting of newlines inserted into a HTML tag.

rawbody is also useful when you're looking for a new trick the
obfuscates URIs in such a way that SA can't parse them, but outlook can
still open them. This used to be common enough that most folks used
rawbody for all their URI type rules. However, nowadays most of them are
caught.


 Thanks,
 Dan




Re: SPF for avoiding newsletter FPs

2006-04-24 Thread Matt Kettler
Ramprasad wrote:
 Hi,
   We get considerable number of newsletter mails with spammy content.
 How do people tackle Fp's from newsletters ? typically the stock
 newsletters , the bank promotional newsletters etc 
 I would like know if this is possible ( I am using SA3.1 + Mailscanner +
 postfix ) 
 1) Maintain a list of newsletters ( this would grow with time )
 2) For each of these newlsetter mails if their SPF records match give a
 high negative score. 

 Atleast those newsletters from domains who *have* SPF records will not
 have problems.

whitelist_from_spf is your friend here. This effectively allows you to
whitelist mail based on From address, but only if it passes SPF.

http://spamassassin.apache.org/full/3.1.x/dist/doc/Mail_SpamAssassin_Plugin_SPF.html


SPF Problems

2006-04-24 Thread Jeferson Pessoa Santana

Hello list,

I'm using SPF and if the sender domain have a SPF record, then the 
header of the message show the SPF_PASS but if doesn't, the header don't 
show the SPF_FAIL or SOFTFAIL. This thing already happened with someone 
on the list?


Thanks,

Jeff


Re: SPF Problems

2006-04-24 Thread Theo Van Dinter
On Mon, Apr 24, 2006 at 11:58:00AM -0300, Jeferson Pessoa Santana wrote:
 I'm using SPF and if the sender domain have a SPF record, then the 
 header of the message show the SPF_PASS but if doesn't, the header don't 
 show the SPF_FAIL or SOFTFAIL. This thing already happened with someone 
 on the list?

If the sending domain doesn't have a SPF record, the message can't fail
an SPF check.  There has to be a record for a check to occur.

-- 
Randomly Generated Tagline:
A bug is a bug.  Even if it is not a hole, it should be hunted down
 and squashed, because one or more bugs can combine to become one or
 more holes...- Theo de Raadt


pgpE5WfkVOgrs.pgp
Description: PGP signature


Re: Charity spam - is this a new kind of 419?

2006-04-24 Thread John D. Hardin
On Mon, 24 Apr 2006, Peter Campion-Bye wrote:

 looks to me like a new kind of 419 scam

 States,Canada,Australia and Ireland who can work online with our
 Branch in Africa.

Either that or one of the Make Big Bucks Laundering Money At Home
In Your Spare Time schemes.

--
 John Hardin KA7OHZICQ#15735746http://www.impsec.org/~jhardin/
 [EMAIL PROTECTED]FALaholic #11174pgpk -a [EMAIL PROTECTED]
 key: 0xB8732E79 - 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
 Senator, when you took your oath of office, you placed your hand on
 the Bible and swore to uphold the Constitution. You didn't place your
 hand on the Constitution and swear to uphold the Bible.
-- Jamie Raskin, Professor of Law at American
University, testifying before the Maryland Senate
---



Re: SPF Problems

2006-04-24 Thread Jeferson Pessoa Santana




I'm a little confuse now. When I Get SPF_FAIL? I think that when the
domain don't have a SPF record, spamassassin scores the message with
FAIL. 



Theo Van Dinter wrote:

  On Mon, Apr 24, 2006 at 11:58:00AM -0300, Jeferson Pessoa Santana wrote:
  
  
I'm using SPF and if the sender domain have a SPF record, then the 
header of the message show the SPF_PASS but if doesn't, the header don't 
show the SPF_FAIL or SOFTFAIL. This thing already happened with someone 
on the list?

  
  
If the sending domain doesn't have a SPF record, the message can't fail
an SPF check.  There has to be a record for a check to occur.

  






Re: Pyzor

2006-04-24 Thread Stuart Johnston

M.Lewis wrote:
Is there a way to check that Pyzor (and Razor) are working? I'm running 
SA 3.1.1.


I never see any Razor or Pyzor information in the headers of spam.

spamassassin -D --lint  shows in part:

[8310] dbg: plugin: registering glue method for check_pyzor 
(Mail::SpamAssassin::Plugin::Pyzor=HASH(0x9dfdd80))
[8310] dbg: util: current PATH is: 
/usr/kerberos/sbin:/usr/kerberos/bin:/opt/jre1.5.0_06/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin 


[8310] dbg: util: executable for pyzor was found at /usr/bin/pyzor
[8310] dbg: pyzor: pyzor is available: /usr/bin/pyzor
[8310] dbg: info: entering helper-app run mode
[8310] dbg: pyzor: opening pipe: /usr/bin/pyzor  check  
/tmp/.spamassassin8310IHD3gbtmp

[8312] dbg: util: setuid: ruid=0 euid=0
[8310] dbg: pyzor: killed stale helper [8312]
[8310] dbg: pyzor: [8312] terminated:  exit=0x000f
[8310] dbg: info: leaving helper-app run mode
[8310] dbg: pyzor: check timed out after 5 seconds




There is only one pyzor server and it does tend to timeout a lot even if 
you are configured correctly.  I've set my pyzor_timeout to 1 second to 
avoid the wasted lookups.


To make sure you are configured correctly, login as the same user spamd 
runs as and run 'pyzor discover'.  Then run 'pyzor ping' a couple of 
times.  If you get: 66.250.40.33:24441  (200, 'OK'), then it is working.


-Stuart



Re: SPF Problems

2006-04-24 Thread Matt Kettler
Jeferson Pessoa Santana wrote:
 I'm a little confuse now. When I Get SPF_FAIL? 

You'll get a FAIL when the sending domain has a SPF record, but the sending
machine is not listed in the SPF record, AND the all clause is set to fail, or
softfail.

SPF failure isn't by default. It's by declaration. For example, my domain has a
SPF record. If you were to send mail and forge my email address you'd generate a
SPF failure.

Here's my record for evi-inc.com:

v=spf1 mx ptr ip4:162.84.101.0/24 ip4:208.39.141.80/28 ip4:208.39.140.174/32 
-all

Which states that anyone sending mail with an envelope return of evi-inc.com
must be in one of those 3 IP blocks, or match the MX of the domain, or have a
PTR record matching evi-inc.com. Anything else is forgery, and is declared to be
failure.



 I think that when the
 domain don't have a SPF record, spamassassin scores the message with FAIL.

It does not. You should get no SPF rules at all for a no-spf-record domain.




RelayCountry does not work

2006-04-24 Thread Igor Chudov
I use Spamassassin 3.1.1, and specified the following in my local.cf:

loadplugin Mail::SpamAssassin::Plugin::RelayCountry
add_header all Relay-Country _RELAYCOUNTRY_

When I run spamassassin from command line, it does set the
Spam-Relay-Country header, BUT its value is always
empty.

I do have IP::Country::Fast and I did run spamassassin -D, without
noticing anything interesting.

I want to delete all email from certain countries, any help will be
appreciated. 

i 


Re: RelayCountry does not work

2006-04-24 Thread Andrzej Adam Filip
Igor Chudov [EMAIL PROTECTED] writes:

 I use Spamassassin 3.1.1, and specified the following in my local.cf:

 loadplugin Mail::SpamAssassin::Plugin::RelayCountry
 add_header all Relay-Country _RELAYCOUNTRY_

 When I run spamassassin from command line, it does set the
 Spam-Relay-Country header, BUT its value is always
 empty.

 I do have IP::Country::Fast and I did run spamassassin -D, without
 noticing anything interesting.

 I want to delete all email from certain countries, any help will be
 appreciated. 

Could you post topmost Received: headers form sample message from
internet?

One possible explanation would be masking IP of the true relay 
e.g. by email gateway to internal mail server transfer.

-- 
[pl2en Andrew] Andrzej Adam Filip : [EMAIL PROTECTED] : [EMAIL PROTECTED]
http://anfi.homeunix.net/


Re: RelayCountry does not work

2006-04-24 Thread Matt Kettler
Igor Chudov wrote:
 I use Spamassassin 3.1.1, and specified the following in my local.cf:
 
 loadplugin Mail::SpamAssassin::Plugin::RelayCountry

First: DO NOT put ANY loadplugin statements in your local.cf, unless you
understand the side-effects and intentionaly don't want the rules for the plugin
to be loaded.

Edit your init.pre for this one. All your loadplugin statements should be in
init.pre or v310.pre. The files should even have the statements in them already,
all you need to do is change which ones are commented out.

 add_header all Relay-Country _RELAYCOUNTRY_
 
 When I run spamassassin from command line, it does set the
 Spam-Relay-Country header, BUT its value is always
 empty.

Where did you get the idea that _RELAYCOUNTRY_ would work here?

That feature is not present in SA 3.1.1 but is due to be released when SA 3.1.2
comes out.

http://issues.apache.org/SpamAssassin/show_bug.cgi?id=3815

However, SA does create a temporary fake header called X-Relay-Countries. You
can write SA rules that will match this header, but it gets removed when SA is
done with the message.

At present there's no way for SA 3.1.1 or older to create a permanent header
with this info in it.



Re: RelayCountry does not work

2006-04-24 Thread Igor Chudov
On Mon, Apr 24, 2006 at 01:41:47PM -0400, Matt Kettler wrote:
 Igor Chudov wrote:
  I use Spamassassin 3.1.1, and specified the following in my local.cf:
  
  loadplugin Mail::SpamAssassin::Plugin::RelayCountry
 
 First: DO NOT put ANY loadplugin statements in your local.cf, unless you
 understand the side-effects and intentionaly don't want the rules for the 
 plugin
 to be loaded.

Thanks. I made the change and moved these declarations to init.pre.

 Edit your init.pre for this one. All your loadplugin statements should be in
 init.pre or v310.pre. The files should even have the statements in them 
 already,
 all you need to do is change which ones are commented out.
 
  add_header all Relay-Country _RELAYCOUNTRY_
  
  When I run spamassassin from command line, it does set the
  Spam-Relay-Country header, BUT its value is always
  empty.
 
 Where did you get the idea that _RELAYCOUNTRY_ would work here?

It is mentioned in many places, for example 

http://search.cpan.org/dist/Mail-SpamAssassin/lib/Mail/SpamAssassin/Plugin/RelayCountry.pm

 That feature is not present in SA 3.1.1 but is due to be released when SA 
 3.1.2
 comes out.
 
 http://issues.apache.org/SpamAssassin/show_bug.cgi?id=3815
 
 However, SA does create a temporary fake header called X-Relay-Countries. 
 You
 can write SA rules that will match this header, but it gets removed when SA is
 done with the message.
 

That would be interesting, how can I add that header now?

I would just handle it with procmail or my own mail filter that I
have. 

 At present there's no way for SA 3.1.1 or older to create a permanent header
 with this info in it.

any help on getting this going now will be appreciated. 

i


RE: Reference manual

2006-04-24 Thread Chris Santerre
Title: RE: Reference manual






 
 Steve Sargent wrote:
  Is there a reference manual with SpamAssassin, and if so were do I
  get a copy of it?
 
 
 Its not all inclusive, but I've found this book to be a handy 
 beginning:
 http://www.packtpub.com/book/spamassassin
 
 Dan



I was waiting for someone else to recommend it :) 


Chris Santerre
SysAdmin and SARE/URIBL ninja
http://www.uribl.com
http://www.rulesemporium.com





Re: RelayCountry does not work

2006-04-24 Thread Matt Kettler
Igor Chudov wrote:
 On Mon, Apr 24, 2006 at 01:41:47PM -0400, Matt Kettler wrote:
 Igor Chudov wrote:
 I use Spamassassin 3.1.1, and specified the following in my local.cf:

 loadplugin Mail::SpamAssassin::Plugin::RelayCountry
 First: DO NOT put ANY loadplugin statements in your local.cf, unless you
 understand the side-effects and intentionaly don't want the rules for the 
 plugin
 to be loaded.
 
 Thanks. I made the change and moved these declarations to init.pre.
 
 Edit your init.pre for this one. All your loadplugin statements should be in
 init.pre or v310.pre. The files should even have the statements in them 
 already,
 all you need to do is change which ones are commented out.

 add_header all Relay-Country _RELAYCOUNTRY_

 When I run spamassassin from command line, it does set the
 Spam-Relay-Country header, BUT its value is always
 empty.
 Where did you get the idea that _RELAYCOUNTRY_ would work here?
 
 It is mentioned in many places, for example 
 
 http://search.cpan.org/dist/Mail-SpamAssassin/lib/Mail/SpamAssassin/Plugin/RelayCountry.pm
 

Hmm, you're right.. it was apparently added to the docs for SA 3.1.1, but wasn't
correctly implemented.

 That feature is not present in SA 3.1.1 but is due to be released when SA 
 3.1.2
 comes out.

 http://issues.apache.org/SpamAssassin/show_bug.cgi?id=3815

 However, SA does create a temporary fake header called X-Relay-Countries. 
 You
 can write SA rules that will match this header, but it gets removed when SA 
 is
 done with the message.

 
 That would be interesting, how can I add that header now?

X-Relay-Countries is already created. But it's also never added to the message
itself. You can't force this header into the message.

 
 I would just handle it with procmail or my own mail filter that I
 have. 
 
 At present there's no way for SA 3.1.1 or older to create a permanent header
 with this info in it.
 
 any help on getting this going now will be appreciated. 

You'd have to patch your copy of SA with the fix patch from the bug report
above. However, this assumes you're comfortable with diff/patch tools.

You can get the patch at:
http://issues.apache.org/SpamAssassin/attachment.cgi?id=3444action=view


This is a hard-coded problem. There's no configuration options in SA 3.1.1 that
can fix it.




Re: RelayCountry does not work

2006-04-24 Thread Igor Chudov
On Mon, Apr 24, 2006 at 07:41:15PM +0200, Andrzej Adam Filip wrote:
 Igor Chudov [EMAIL PROTECTED] writes:
 
  I use Spamassassin 3.1.1, and specified the following in my local.cf:
 
  loadplugin Mail::SpamAssassin::Plugin::RelayCountry
  add_header all Relay-Country _RELAYCOUNTRY_
 
  When I run spamassassin from command line, it does set the
  Spam-Relay-Country header, BUT its value is always
  empty.
 
  I do have IP::Country::Fast and I did run spamassassin -D, without
  noticing anything interesting.
 
  I want to delete all email from certain countries, any help will be
  appreciated. 
 
 Could you post topmost Received: headers form sample message from
 internet?
 
 One possible explanation would be masking IP of the true relay 
 e.g. by email gateway to internal mail server transfer.
 

Sure, here is the sample spam:

*From [EMAIL PROTECTED]  Mon Apr 24 09:23:43 2006
Return-Path: [EMAIL PROTECTED]
X-Spam-Checker-Version: SpamAssassin 3.1.1 (2006-03-10) on 
manifold.algebra.com
X-Spam-Level: 
X-Spam-Status: No, score=0.8 required=3.0 tests=INFO_TLD,UNPARSEABLE_RELAY 
autolearn=disabled version=3.1.1
Received: from ak74.algebra.com (ak74.algebra.com [65.182.171.162])
by manifold.algebra.com (8.13.6/8.13.6) with ESMTP id k3OENhEZ002505
for [EMAIL PROTECTED]; Mon, 24 Apr 2006 09:23:43 -0500
Received: from a38198.upc-a.chello.nl (a38198.upc-a.chello.nl [62.163.38.198])
by ak74.algebra.com (8.13.6/8.13.1) with SMTP id k3OENUFQ005425
for [EMAIL PROTECTED]; Mon, 24 Apr 2006 09:23:37 -0500
Received: from mail.netelligent.ca
by a38198.upc-a.chello.nl (8.9.3/8.9.3) with ESMTP id CZ7TBbscc5oV
for [EMAIL PROTECTED]; Mon, 24 Apr 2006 17:30:47 -0700
Received: from ([EMAIL PROTECTED]) 
by mail.netelligent.ca with Microsoft SMTPSVC(5.0.2195.5329)
for [EMAIL PROTECTED]; Mon, 24 Apr 2006 17:30:47 -0700
Date: Mon, 24 Apr 2006 17:30:47 -0700
From: Ramon Chu [EMAIL PROTECTED]
Reply-To: Ramon Chu [EMAIL PROTECTED]
Message-ID: [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Subject: Hardcoore incesst Content!
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: 7bit
Status: RO
Content-Length: 69
Lines: 4


hardccore family banging!

http://wifiplayarea.info/bxfamilynig.htm



Re: Should My Install of SA Be Catching These?

2006-04-24 Thread Mike Jackson

I would appreciate any guidance that you feel would make my SA setup
stronger.  These types of messages (attached) keep squeaking through...
is my setup weak or have I broken something?  To the layman's eye, they
look pretty spammy.

I am running v3.0.2 and I just went through all the SARE updates about
2 weeks ago, but these messages still score under my 4.5 threshold for
spam.  In my setup they score as follows:



From my experience, the two best ways to catch spam with SpamAssassin:


1. A thoroughly trained Bayes database. Feed the messages to it, and watch 
them get caught! I see at least the first message had a Bayes score of 0. 
There's your culprit right there!


2. Collaborative databases like Razor and Pyzor. I'll bet either one 
would've caught those messages.


Spammers learn too quickly from the SARE rules for them to be truly 
effective - after a while, they become edge cases, not the norm. The Spamcop 
top 200 list is nice, but I rarely see spam come from the same source 
twice - I ran a test one weekend where I fed all the dictionary-attack spam 
that a certain domain I host received (and it gets a LOT of dictionary 
attack spam) into a homebrew RBL. It listed thousands of IPs, not a single 
one of which made more than one SMTP connection (therefore the homebrew RBL 
was a total bust). But, there's not much they can do against the mighty 
power of Bayes and Razor.


You might also investigate using RBLs at the SMTP level, as long as you 
trust them to be accurate. I use the SBL and XBL lists from Spamhaus, and 
have never once heard a legitimate complaint about them creating false 
positives (and that's at three different providers over 4-5 years). I also 
use the bogusmx and DSN lists from rfc-ignorant.org; you're running a higher 
risk of causing false positives (but honestly, any false positives you see 
should simply be addressed with the admin of the responsible network, 
because they're just being stupid), but you're also going to catch quite a 
bit of spam. 



RE: Should My Install of SA Be Catching These?

2006-04-24 Thread Bowie Bailey
Clay Davis wrote:
 I would appreciate any guidance that you feel would make my SA setup
 stronger.  These types of messages (attached) keep squeaking
 through... is my setup weak or have I broken something?  To the
 layman's eye, they look pretty spammy.
 
 I am running v3.0.2 and I just went through all the SARE updates about
 2 weeks ago, but these messages still score under my 4.5 threshold for
 spam.  In my setup they score as follows:
 
 viagra.txt
 
  pts rule name  description
  -- --
  0.0 BAYES_00   BODY: Bayesian spam probability is 0 to 1%
 [score: 0.]
  0.0 HTML_MESSAGE   BODY: HTML included in message
  0.0 AWLAWL: From: address is in the auto
 white-list
 
 
 PillGraphic.txt
 
  pts rule name  description
  -- --
  0.5 SARE_HTML_URI_LHOST30  URI: Long unbroken string within URI
  0.1 HTML_80_90 BODY: Message is 80% to 90% HTML
  0.5 BAYES_40   BODY: Bayesian spam probability is 20 to
 40%
 [score: 0.2135]
  0.5 HTML_IMAGE_ONLY_24 BODY: HTML: images with 2000-2400 bytes of
 words
  0.0 HTML_MESSAGE   BODY: HTML included in message
  0.2 MIME_HTML_ONLY BODY: Message only has text/html MIME
 parts
  0.8 SARE_GIF_ATTACHFULL: Email has a inline gif
  0.0 MIME_BOUND_NEXTPARTSpam tool pattern in MIME boundary
  1.7 SARE_GIF_STOX  Inline Gif with little HTML

My first attempt was rejected by the list, so let me try again with
the URIs stripped out...

You should be catching these easily.

The first thing I would do is fix your Bayes database.  If it is
assigning BAYES_00 to a spam message, then something is seriously
wrong.  Once you have fixed it, you should put back the default
scores.  BAYES_00 should score negative under normal conditions.

Razor, DCC, Pyzor, and URIBL are also useful against these types of
spams.

This is what I got on those two messages.  Note that Razor2, URIBL,
and a properly functioning Bayes database tore them apart.


Viagra.txt:

X-Spam-Status: Yes, score=41.1 ...
X-Spam-Report:
*  0.1 FORGED_RCVD_HELO Received: contains a forged HELO
*  0.0 HTML_MESSAGE BODY: HTML included in message
*  3.5 BAYES_99 BODY: Bayesian spam probability is 99 to 100%
*  [score: 1.]
*  1.5 RAZOR2_CF_RANGE_E8_51_100 Razor2 gives engine 8 confidence
level
*  above 50%
*  [cf: 100]
*  0.5 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/)
*  0.5 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level above
50%
*  [cf: 100]
*  2.2 DCC_CHECK Listed in DCC (http://rhyolite.com/anti-spam/dcc/)
*  3.9 RCVD_IN_XBL RBL: Received via a relay in Spamhaus XBL
*  2.0 RCVD_IN_SORBS_DUL RBL: SORBS: sent directly from dynamic IP
address
*  1.9 DNS_FROM_RFC_BOGUSMX RBL: Envelope sender in
*  bogusmx.rfc-ignorant.org
*  1.9 RCVD_IN_NJABL_DUL RBL: NJABL: dialup sender did non-local
SMTP
*  1.6 URIBL_SBL Contains an URL listed in the SBL blocklist
*  3.8 URIBL_AB_SURBL Contains an URL listed in the AB SURBL
blocklist
*  4.1 URIBL_JP_SURBL Contains an URL listed in the JP SURBL
blocklist
*  3.0 URIBL_BLACK Contains an URL listed in the URIBL blacklist
*  2.1 URIBL_WS_SURBL Contains an URL listed in the WS SURBL
blocklist
*  3.0 URIBL_OB_SURBL Contains an URL listed in the OB SURBL
blocklist
*  4.5 URIBL_SC_SURBL Contains an URL listed in the SC SURBL
blocklist
*  0.8 DIGEST_MULTIPLE Message hits more than one network digest
check


PillGraphic.txt:

X-Spam-Status: Yes, score=28.8 ...
X-Spam-Report:
*  0.6 J_CHICKENPOX_27 BODY: 2alpha-pock-7alpha
*  0.9 URI_NOVOWEL URI: URI hostname has long non-vowel sequence
*  1.8 HTML_IMAGE_ONLY_24 BODY: HTML: images with 2000-2400 bytes of
words
*  0.0 HTML_MESSAGE BODY: HTML included in message
*  3.0 BAYES_95 BODY: Bayesian spam probability is 95 to 99%
*  [score: 0.9723]
*  0.0 MIME_HTML_ONLY BODY: Message only has text/html MIME parts
*  0.8 SARE_GIF_ATTACH FULL: Email has a inline gif
*  1.5 RAZOR2_CF_RANGE_E8_51_100 Razor2 gives engine 8 confidence
level
*  above 50%
*  [cf: 100]
*  0.5 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/)
*  0.5 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level above
50%
*  [cf: 100]
*  0.2 DNS_FROM_RFC_ABUSE RBL: Envelope sender in
abuse.rfc-ignorant.org
*  3.9 RCVD_IN_XBL RBL: Received via a relay in Spamhaus XBL
*  1.6 URIBL_SBL Contains an URL listed in the SBL blocklist
*  4.1 URIBL_JP_SURBL Contains an URL listed in the JP SURBL
blocklist
*  

Re: RelayCountry does not work

2006-04-24 Thread Matt Kettler
Igor Chudov wrote:
 I have this statement in init.pre
 
 add_header all Relay-Country _RELAYCOUNTRY_ 

*sigh*.. that should be in your local.cf

Put the loadplugin statements in your .pre files, and nowhere else, but also
don't put anything else there.



Re: RelayCountry does not work

2006-04-24 Thread Matt Kettler
Igor Chudov wrote:
 On Mon, Apr 24, 2006 at 04:38:38PM -0400, Matt Kettler wrote:
 Igor Chudov wrote:
 I have this statement in init.pre

 add_header all Relay-Country _RELAYCOUNTRY_ 
 *sigh*.. that should be in your local.cf

 Put the loadplugin statements in your .pre files, and nowhere else, but also
 don't put anything else there.
 
 Beautiful. I moved the add_header statement to init.pre, and
 magically, spamassassin works (at least when invoked from commmand
 line, not via spamc as I normally do).


In order for spamc to reflect the config changes you'll need to restart spamd.
Spamd only parses the /etc/mail/spamassassin and /usr/share/spamassassin config
files at startup.

 



Re: RelayCountry does not work

2006-04-24 Thread Igor Chudov
On Mon, Apr 24, 2006 at 04:46:40PM -0400, Matt Kettler wrote:
 Igor Chudov wrote:
  On Mon, Apr 24, 2006 at 04:38:38PM -0400, Matt Kettler wrote:
  Igor Chudov wrote:
  I have this statement in init.pre
 
  add_header all Relay-Country _RELAYCOUNTRY_ 
  *sigh*.. that should be in your local.cf
 
  Put the loadplugin statements in your .pre files, and nowhere else, but 
  also
  don't put anything else there.
  
  Beautiful. I moved the add_header statement to init.pre, and
  magically, spamassassin works (at least when invoked from commmand
  line, not via spamc as I normally do).
 
 
 In order for spamc to reflect the config changes you'll need to restart spamd.
 Spamd only parses the /etc/mail/spamassassin and /usr/share/spamassassin 
 config
 files at startup.

Yes, I did HUP spamc and I see that it works. Thank you Matt! I am
very happy, now I can start banning countries.

i


Re: RelayCountry does not work

2006-04-24 Thread Matt Kettler
Igor Chudov wrote:

 Yes, I did HUP spamc and I see that it works. Thank you Matt! I am
 very happy, now I can start banning countries.

Fair enough.. Just remember to unsubscribe yourself from global mailing lists,
like this one, first... After all, you never know what country an answer to a
question will come from, and you'll look rather silly having half a 
conversation.

The way RelayCountry works, you'll pick up ALL countries a message went through,
not just the country of the host that dropped it off to your network. Thus,
mailing list messages will also match the country of origin.

Moral of the story: blocking countries is fairly dangerous.. Tread carefully 
here.






Re: RelayCountry does not work

2006-04-24 Thread Igor Chudov
On Mon, Apr 24, 2006 at 04:57:20PM -0400, Matt Kettler wrote:
 Igor Chudov wrote:
 
  Yes, I did HUP spamc and I see that it works. Thank you Matt! I am
  very happy, now I can start banning countries.
 
 Fair enough.. Just remember to unsubscribe yourself from global mailing lists,
 like this one, first... After all, you never know what country an answer to a
 question will come from, and you'll look rather silly having half a 
 conversation.
 
 The way RelayCountry works, you'll pick up ALL countries a message went 
 through,
 not just the country of the host that dropped it off to your network. Thus,
 mailing list messages will also match the country of origin.
 
 Moral of the story: blocking countries is fairly dangerous.. Tread carefully 
 here.

Thanks. I will start with China and Korea, I have never received a
legitimate message from there. 

In any case, my spam is never deleted, it goes into junk folders,
which I review every several days.

i


Re: RelayCountry does not work

2006-04-24 Thread Matt Kettler
Igor Chudov wrote:

 Moral of the story: blocking countries is fairly dangerous.. Tread carefully 
 here.
 
 Thanks. I will start with China and Korea, I have never received a
 legitimate message from there. 

Are you sure? I've received dozens legitimate messages via this very list from
Chinese IPs. There are at least 3 different users on this list that match my
RELAY_CN rule (based on RelayCountry), and one of them has posted 23 messages
since February.

Note it's not always easy to tell where the person sending mail is really
located. They could work for a US company, but be away on travel and sending via
SMTP-Auth from China.

Ditto for gmail, yahoo, etc.. Have you checked the IPs in the headers to see
what countries they were in? Reverse DNS names won't help you here either.



 
 In any case, my spam is never deleted, it goes into junk folders,
 which I review every several days.

That's a good idea. Do some playing, you'll likely find that you get a lot more
international mail than you think..



Re: URI Basics

2006-04-24 Thread Theo Van Dinter
On Mon, Apr 24, 2006 at 05:18:23PM -0700, Dan wrote:
 Are you saying that in URIs, any character (@ in this case) can serve  
 as the delimiter, so long as it displays after the m and again at the  
 end of the entry?

Yes.  Take a look at the perlre and perlop (specifically the m// operator)
documentation.  Mastering Regular Expressions from O'Reilly may be a
good book to read as well. :)

-- 
Randomly Generated Tagline:
I'll just change into someone more comfortable.


pgpH4h3DIrQn4.pgp
Description: PGP signature


Re: URI Basics

2006-04-24 Thread John Rudd


On Apr 24, 2006, at 5:18 PM, Dan wrote:
I'm beginning to realize how many of my learning curve issues are 
attempts to understand the very structure of a system created with a 
bare minimum of structure.


Specifically, you're learning perl regular expressions, and perl is a 
language that gives you a million different ways to skin a cat, so to 
speak.  As the quote goes all things are permissible, but not all 
things are beneficial.


It's also a programming language that many people tend to describe as 
looking like line noise (if you ever used an old dial up line, in 
terminal mode instead of as a SLIP/PPP link, you may actually get the 
joke ... especially if you had call waiting turned on).


Between the two, yes, it feels very unstructured.

In addition to the other book that was recommended, it might be a good 
idea to pick up Learning Perl.  It's easier to understand a thing when 
you know how it thinks.




Re: URI Basics

2006-04-24 Thread Matt Kettler
Dan wrote:
 In 3 ^ is the first character of the regex, just as it is in 1 and 2. It
 is also inside the delimiters, just like 1 and 2. In example 3 @ is
 being used as a delimiter,  and ^ is the first character after it.

 Are you saying that in URIs, any character (@ in this case) can serve
 as the delimiter, so long as it displays after the m and again at the
 end of the entry?
Well, any non-alphanumeric non-whitespace can be used. i.e. any punctuation.

Actually This actually is true of ANY SA rule, not just URIs. The use of
m to set up a regex delimiter is just part of the perl regex syntax,
which SA supports all of. It's called the match operator.

So
 /foo/
m/foo/
m!foo!

Just be warry of what you use as a delimiter. Choosing something other
than / should only done to make things easier to read. It also
over-rides that character's normal uses until the end of the regex.

You can find a lot of detail about using the match operator (m) for this
purpose in section 7.4.3 of:

http://www.unix.org.ua/orelly/perl/learn/ch07_04.htm

(note: that page is general perl programing oriented, so a lot of things
in there are not so relevant.



 I'm beginning to realize how many of my learning curve issues are
 attempts to understand the very structure of a system created with a
 bare minimum of structure.
Heh, it's not that bad.. but there are a lot of advanced quirks you'll
see people using from their knowledge of heavy perl wizzardry.


 There is definitely a VERY significant performance penalty to using
 rawbody over URI, for any rule.

 Consider the size of input. A rawbody regex must be run against the
 entire text of the body after QP decoding. A uri regex must be run
 against all the text of the URIs that SA found. There is likely to be at
 least a 100:1 difference in size of input. There's no penalty for
 using a uri rule, as SA will always extract all the URIs and build the
 input text, even if you aren't using it.

 Great information Matt, thanks. 
No problem.


Re: URI Basics

2006-04-24 Thread Dan

Follow up question:

Is URI the way to go when tracking obsfucation, as in:
uri __LINKAGE_A284 [EMAIL PROTECTED]

...or will URI's translation get in the way, requiring something more  
like?:

rawbody __LINKAGE_A284 [EMAIL PROTECTED]

Thanks,
Dan



Re: URI Basics

2006-04-24 Thread Matt Kettler
Dan wrote:
 Follow up question:

 Is URI the way to go when tracking obsfucation, as in:
 uri __LINKAGE_A284 [EMAIL PROTECTED]

 ...or will URI's translation get in the way, requiring something more
 like?:
 rawbody __LINKAGE_A284 [EMAIL PROTECTED]

Neither of the above will work.. Both uri and rawbody rules are run
after QP (and base 64) decoding is done.

There's some proposals to have a more configurable set of choices but
right now raw is really half cooked, and uri is fully cooked just
like body.




Re: URI Basics

2006-04-24 Thread Theo Van Dinter
On Mon, Apr 24, 2006 at 09:27:47PM -0400, Matt Kettler wrote:
  Is URI the way to go when tracking obsfucation, as in:
  uri __LINKAGE_A284 [EMAIL PROTECTED]

Yes.  The uri rules run over both the raw version and the decoded versions.

 Neither of the above will work.. Both uri and rawbody rules are run
 after QP (and base 64) decoding is done.

FWIW, the character encoding (w = %77) isn't QP or base64, it's just encoding.

 There's some proposals to have a more configurable set of choices but
 right now raw is really half cooked, and uri is fully cooked just
 like body.

uri is a large array of all the uris found in the mail.  for each raw
one found in the mail, SA goes through and canonicalizes them (remove
obfuscation, find redirector patterns, etc,) and then all of those
(raw and canonical) are run through by the uri rules.

-- 
Randomly Generated Tagline:
Well, last time I checked, I wasn't a trout ... - rei.com radio ad


pgp1PlMpVocDb.pgp
Description: PGP signature


Re: URI Basics

2006-04-24 Thread Dan

Gentlemen,

Thank you for the all the great input.


Specifically, you're learning perl regular expressions, and perl is  
a language that gives you a million different ways to skin a cat,  
so to speak.  As the quote goes all things are permissible, but  
not all things are beneficial.


It's also a programming language that many people tend to describe  
as looking like line noise (if you ever used an old dial up line,  
in terminal mode instead of as a SLIP/PPP link, you may actually  
get the joke ... especially if you had call waiting turned on).


I'm new to regex and SA (and open source for that matter) but I'm  
actually old school tech.  I remember well, the thrill of upgrading  
from 2400 to 14.4k bps.  One my fondest tech memories is bringing  
online my own ISDN based 56k RAS in the 90's.  And that thing has a  
CLI that would make SA blush.




Between the two, yes, it feels very unstructured.

In addition to the other book that was recommended, it might be a  
good idea to pick up Learning Perl.  It's easier to understand a  
thing when you know how it thinks.


I know what you mean.  Being new to both, its been tough not knowing  
when regex ends and SA begins.  I'm used to being able to make  
systems sing so coming in cold to a system this big and well  
established (even while understanding the principals being used) is  
intimidating.  With your help, I'll have SA breaking a sweat in no time.



Dan


Messages Not detected as Spam

2006-04-24 Thread Paul Wetter




For the last week now I have been receiving 
severalvery similar messages that are spam and not being detected as 
spam. I have done an sa-learn on every one of them but they still come in 
not even being tagged. Is there something wrong with my bayes 
detection? Is there any way to log what spamassassin is doing to see if it 
finds anything?

I call spamassassin's spam checks through 
amavisd-new which controls a couple virtual domains.

Thanks in advance,
Paul


Re: Messages Not detected as Spam

2006-04-24 Thread Paul Wetter



I forgot to note thatI have flagged 50+ of 
these similar emails. It seems to me that something is not working 
correctly.

  - Original Message - 
  From: 
  Paul 
  Wetter 
  To: users@spamassassin.apache.org 
  
  Sent: Monday, April 24, 2006 10:30 
  PM
  Subject: Messages Not detected as 
  Spam
  
  
  For the last week now I have been receiving 
  severalvery similar messages that are spam and not being detected as 
  spam. I have done an sa-learn on every one of them but they still come 
  in not even being tagged. Is there something wrong with my bayes 
  detection? Is there any way to log what spamassassin is doing to see if 
  it finds anything?
  
  I call spamassassin's spam checks through 
  amavisd-new which controls a couple virtual domains.
  
  Thanks in advance,
  Paul


Re: Messages Not detected as Spam

2006-04-24 Thread Matt Kettler
Paul Wetter wrote:
 For the last week now I have been receiving several very similar
 messages that are spam and not being detected as spam.  I have done an
 sa-learn on every one of them but they still come in not even being
 tagged.  Is there something wrong with my bayes detection?  Is there
 any way to log what spamassassin is doing to see if it finds anything?
  
 I call spamassassin's spam checks through amavisd-new which controls a
 couple virtual domains.

First step, try running one of them manually through spamassassin -t..
what rule hits do you get? (post the X-Spam-Status SA generates).

Next step, modify amavis to always add an X-Spam-Status header (ie: set
tagged_above to -1000.). Compare the results, or post here along with
the above..





Re: Messages Not detected as Spam

2006-04-24 Thread Paul Wetter

Here is what I get when I reproduce the email:
X-Spam-Status: No, hits=0.002 tagged_above=-1 required=1.5
tests=[BAYES_50=0.001, HTML_MESSAGE=0.001]


spamassassin -t gives me this:

Content analysis details:   (9.1 points, 2.5 required)

pts rule name  description
 -- --
0.1 FORGED_RCVD_HELO   Received: contains a forged HELO
0.0 UNPARSEABLE_RELAY  Informational: message has unparseable relay 
lines

0.0 HTML_MESSAGE   BODY: HTML included in message
3.5 BAYES_99   BODY: Bayesian spam probability is 99 to 100%
   [score: 1.]
1.6 RCVD_IN_BL_SPAMCOP_NET RBL: Received via a relay in bl.spamcop.net
  [Blocked - see 
http://www.spamcop.net/bl.shtml?81.121.100.79]

3.9 RCVD_IN_XBLRBL: Received via a relay in Spamhaus XBL
   [81.121.100.79 listed in sbl-xbl.spamhaus.org]


They are very different!
Where do we go from here?

Thanks again!!
-Paul


- Original Message - 
From: Matt Kettler [EMAIL PROTECTED]

To: Paul Wetter [EMAIL PROTECTED]
Cc: users@spamassassin.apache.org
Sent: Monday, April 24, 2006 10:46 PM
Subject: Re: Messages Not detected as Spam



Paul Wetter wrote:

For the last week now I have been receiving several very similar
messages that are spam and not being detected as spam.  I have done an
sa-learn on every one of them but they still come in not even being
tagged.  Is there something wrong with my bayes detection?  Is there
any way to log what spamassassin is doing to see if it finds anything?

I call spamassassin's spam checks through amavisd-new which controls a
couple virtual domains.


First step, try running one of them manually through spamassassin -t..
what rule hits do you get? (post the X-Spam-Status SA generates).

Next step, modify amavis to always add an X-Spam-Status header (ie: set
tagged_above to -1000.). Compare the results, or post here along with
the above..










Re: RelayCountry does not work

2006-04-24 Thread jdow

From: Matt Kettler [EMAIL PROTECTED]


Igor Chudov wrote:


Yes, I did HUP spamc and I see that it works. Thank you Matt! I am
very happy, now I can start banning countries.


Fair enough.. Just remember to unsubscribe yourself from global mailing lists,
like this one, first... After all, you never know what country an answer to a
question will come from, and you'll look rather silly having half a 
conversation.

The way RelayCountry works, you'll pick up ALL countries a message went through,
not just the country of the host that dropped it off to your network. Thus,
mailing list messages will also match the country of origin.

Moral of the story: blocking countries is fairly dangerous.. Tread carefully 
here.


And we DO have a nice fellow from China on this list who can offer some
insights into REALLY large SpamAssassin installations. (Ones that dwarf
the likes of AOL.)

{^_-}


Permission errors

2006-04-24 Thread Igor Chudov
Doing some housecleaning...

I am running spamd as root, at which point it reverts to 'nobody'.

It then proceeds to complain, understandably, that it does not have
permission to write to users' directories. 


Apr 24 23:56:57 manifold spamd[21442]: spamd: still running as root:
user not specified with -u, not found, or set to root, falling back to
nobody at /usr/bin/spamd line 1152, GEN353 line 4. 
Apr 24 23:56:57 manifold spamd[21442]: spamd: processing message
[EMAIL PROTECTED] for root:99 
Apr 24 23:56:58 manifold spamd[21442]: locker: safe_lock: cannot
create tmp lockfile
/root/.spamassassin/auto-whitelist.lock.manifold.algebra.com.21442 for
/root/.spamassassin/auto-whitelist.lock: Permission denied 
Apr 24 23:56:58 manifold spamd[21442]: auto-whitelist: open of
auto-whitelist file failed: locker: safe_lock: cannot create tmp
lockfile
/root/.spamassassin/auto-whitelist.lock.manifold.algebra.com.21442 for
/root/.spamassassin/auto-whitelist.lock: Permission denied 


I am in a cleanup mode and would like to get rid of these errors, but
this one has me stumped. How can it expect to access inside root's
directory, if it runs as nobody???

i