Re: whitelist_from questions

[John Wilcock]

Le 24/07/2009 04:09, MySQL Student a écrit :

I don't doubt that if we removed a substantial amount of them that SA
would do what's right, but there doesn't seem to be any scientific way
to do that successfully.


Can't you just look at the scores that the whitelisted messages are 
getting and see whether any would be close to being considered as spam 
without the -100 of the whitelist? [How best to do that depends on how 
you've integrated spamassassin into your mail setup, but grepping 
through logs ought to do it in most cases].


And perhaps a few carefully-chosen negative-scoring rules (for words or 
phrases common to your customer's business) might be a far more 
effective way of handling the rest.



Is there a way to script that for the 1000 or so entries, to see which
have SPF records?


There are no doubt lots of ways, but how about:

egrep 'whitelist_from[^_]' local.cf | awk '{FS="@"; print $2" TXT";}' | 
xargs dig | grep "v=spf1"


John.

--
-- Over 4000 webcams from ski resorts around the world - www.snoweye.com
-- Translate your technical documents and web pages- www.tradoc.fr

Re: Certain spam not parsed by spamd!

[Jari Fredriksson]

> I've recently implemented relaycountry and seen 90%+
> improvement in our ability to trap spam but there is one
> email which seems capable of avoiding getting parsed by
> spamd. 
> 
> All other messages get the x-spam headers added
> successfully but this one for some reason completely
> slips through without any such headers. It carries a
> trojan too, which is odd because clamav should pick that
> up. clamd is updated daily. 
> 
> The headers of the strange spam are:
> 
> Return-path: 
> Envelope-to: u...@host.co.uk
> Delivery-date: Fri, 24 Jul 2009 11:12:38 +0800
> Received: from [190.144.0.42] (helo=CWXNQKBTZ)
>by s1.host.info with esmtp (Exim 4.67)
>(envelope-from )
>id 1MUBD2-0002wE-2i
>for u...@host.co.uk; Fri, 24 Jul 2009 11:12:38
> +0800 
> Received: from 190.144.0.42 by red3.redtong.com; Thu, 23
> Jul 2009 22:24:55 -0500
> Message-ID: <000d01ca0c0e$50804720$6400a...@banacha55>
> From:  
> To: u...@host.co.uk
> Subject: You have received an eCard
> Date: Thu, 23 Jul 2009 22:24:55 -0500
> MIME-Version: 1.0
> Content-Type: multipart/mixed;
>  boundary="=_NextPart_000_0006_01CA0C0E.50804720"
> X-Priority: 3
> X-MSMail-Priority: Normal
> X-Mailer: Microsoft Outlook Express 6.00.2900.2180
> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180
> 
> The above email contained a .zip file.
> 
> This was not random, as I've received three similar
> emails this morning and none of them have x-spam headers
> all other emails are fine. 

It apparently was never seen by SpamAssassin, if there were no X-Spam-* 
-headers.

How you call SpamAssassin? Any whitelisting there, do you call SpamAssassin for 
your own mail? It seems the sender address is same as receiver address. 
Whitelisted somehow, and maybe not inspected by SpamAssassin?

Certain spam not parsed by spamd!

[snowweb]

I've recently implemented relaycountry and seen 90%+ improvement in our
ability to trap spam but there is one email which seems capable of avoiding
getting parsed by spamd.

All other messages get the x-spam headers added successfully but this one
for some reason completely slips through without any such headers. It
carries a trojan too, which is odd because clamav should pick that up. clamd
is updated daily.

The headers of the strange spam are:

Return-path: 
Envelope-to: u...@host.co.uk
Delivery-date: Fri, 24 Jul 2009 11:12:38 +0800
Received: from [190.144.0.42] (helo=CWXNQKBTZ)
by s1.host.info with esmtp (Exim 4.67)
(envelope-from )
id 1MUBD2-0002wE-2i
for u...@host.co.uk; Fri, 24 Jul 2009 11:12:38 +0800
Received: from 190.144.0.42 by red3.redtong.com; Thu, 23 Jul 2009 22:24:55
-0500
Message-ID: <000d01ca0c0e$50804720$6400a...@banacha55>
From:   
To: u...@host.co.uk
Subject: You have received an eCard
Date: Thu, 23 Jul 2009 22:24:55 -0500
MIME-Version: 1.0
Content-Type: multipart/mixed;
  boundary="=_NextPart_000_0006_01CA0C0E.50804720"
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 6.00.2900.2180
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180

The above email contained a .zip file.

This was not random, as I've received three similar emails this morning and
none of them have x-spam headers all other emails are fine.

pete
-- 
View this message in context: 
http://www.nabble.com/Certain-spam-not-parsed-by-spamd%21-tp24638560p24638560.html
Sent from the SpamAssassin - Users mailing list archive at Nabble.com.

Re: whitelist_from questions

[MySQL Student]

Hi,

> Firstly, before you convert all these to whitelist_from_rcvd, perhaps you
> ought to ask yourself whether you really need 1000 entries on your
> whitelist.

I'm surprised you were the first to make that very comment, so thanks.

> Does mail from these addresses actually get miscategorised as
> spam, or would SA get it right without the whitelist?

Mail was being tagged as spam, and the organization became concerned
that others would be tagged, so it seemed anytime there was a
high-profile external business contact that they couldn't risk being
tagged, they had it added to the whitelist.

The list used to be much larger until we spent quite a while (months
and months) going through it with them to prune it.

I don't doubt that if we removed a substantial amount of them that SA
would do what's right, but there doesn't seem to be any scientific way
to do that successfully.

> Secondly, don't forget about whitelist_from_spf. If a domain has an SPF
> record, this is a better solution than whitelist_from_rcvd as it avoids the
> need for *you* to work out which are the outgoing servers.

Is there a way to script that for the 1000 or so entries, to see which
have SPF records?

> Lastly, if you do use whitelist_from_rcvd, remember that there may be
> multiple outgoing servers for a given domain, and worse they may change over
> time.

Yeah, I thought of that too, so it doesn't sound like that's going to
work well here.

Thanks,
Alex

Re: Lotto/Money & email address spam

[MySQL Student]

Hi,

> sa-update lint checks the rules in a sandbox, and does not update the
> local channel, if there are any issues. Moreover, do NOT copy these
> updates to your site config dir -- but keep it in the update dir where
> sa-update puts them [1]. SA knows how to use them instead of the
> "install-time" default conf.

Okay, great. That is what I have now done. I actually have multiple
mail servers, none of which have direct access to the Internet other
than inbound SMTP, so I have sa-update running on another box, which
creates a tarball, which is then scp'd to the mail servers and
extracted.

For me, this now means the sa-update channels are in
/var/lib/spamassassin/3.0005/ and my local site-config is
/etc/mail/spamassassin, where local.cf and init.pre reside.

I also spent much of the day reading docs. I've worked with Linux now
for many years, and have been involved with SA, just not to the level
that I'm involved now.

> It's a rather bizarre picture I'm sensing here. From your recent posts I
> understand you are running a mail server for a large organization. Yet
> there is this cannonade with rather basic questions...

guenther, I knew you were a smart guy :-)

Yes, there is a bigger picture; hopefully I get some cred for trying
to tackle this on my own (with the help of others more experienced).

Anyway, I'm trying to use sa-update to install the SOUGHT rules, and
linting them shows this:

[17021] warn: config: invalid regexp for rule __SEEK_AY2NNY: /This
place is so exclusive, how did you get an invite\x{e2}\x{80}\x{a6} /:
/This place is so exclusive, how did you get an
invite\x{e2}\x{80}\x{a6} /: Can't use \x{} without 'use utf8'
declaration

I'm using perl-5.6.0; is that the cause?

Thanks again,
Alex

Re: Lotto/Money & email address spam

[MySQL Student]

Hi,

>> Please don't paste examples to this list.
>>
>> Please post them to pastebin (or a similar service) and then include the
>> link.
..

Yes, understood. FWIW, I know enough to not post an entire message
with headers to the list -- I'm sure half the time it would be
filtered anyway. This time it was just a snippet, but in the future
I'll post even those online, too.

Thanks,
Alex

Re: [NEW SPAM FLOOD] www.shopXX.net

[Kevin Parris]

(apologies for top posting, but the email software here does not really do 
quoting in a way that works out well otherwise)

If your mail contains SpamAssassin headers then it was (obviously) processed 
through SpamAssassin.  Just because you have BL checks in your MTA does not 
necessarily mean that all spam items will be blocked at that level.  Lots of 
spam can pass some BL checks and then be scored high as the result of other 
things.

My comments were not meant to say that BL checks stop spam.  I was responding 
specifically to your inquiry about a rule being 'overlooked' if there happened 
to be a message it would hit that also had something in it that would hit a 
blacklist too.  I think you're reading too much complexity into things.  Or 
maybe not enough.

The basic idea is something like this:

 a) You have some stuff specified for Postfix to do, it starts doing those 
things, and if it gets through them (without deciding to reject the message) to 
the point where you specify a call to SA, then it passes the item to SA for 
scoring.

 b)  SA applies the rules (which usually include querying various blacklists 
based on things found within the message) and tallies up the score, then it 
gives the results to whatever asked it to analyze the message.

 c)  Then whatever that was (in your case, Postfix) looks at the results and 
decides what to do next, based on what you specified for it.

SpamAssassin does not block mail.  SpamAssassin analyzes a message and assigns 
a score.   Mail handlers reject/quarantine/discard/deliver mail.  SpamAssassin 
is not a mail handler.

If you don't understand the effects of entries in your Postfix configuration, 
you probably will get better assistance in a Postfix-specific forum.

>>> Dan Schaefer  07/23/09 10:22 AM >>>

> It means that if you were using BL at MTA level your SA might never have seen 
> the message at all.
>
> No your rule would not be "overlooked" 'because the site is in a blacklist' 
> *unless* you were using the BL in your MTA and rejected the transaction from 
> a blacklisted IP address and, thus, never submitted it to SA at all.
>
>   
If this is the case, then why does my email have the X-* headers in it? I have 
nothing in my postfix header_checks to discard the BL rules. Does anyone have a 
detailed flow chart of SA/postfix setup and describes blacklisting? Or even a 
webpage describing the process?

Re: [NEW SPAM FLOOD] www.shopXX.net

[John Hardin]

On Thu, 23 Jul 2009, Dan Schaefer wrote:

> >   Are you quite sure that an upstream copy of SA, e.g. in your ISP 
> >   or at a sender site that scans for outgoing spam, hasn't already 
> >   added X-* headers to the message?
> 
>  No. Is that even possible to track down?


 There would probably be an X-Spam-Checker-Version header in your
 inbound mail stream.


X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on 
pony.performanceadmin.com


That is my server.


You'd have to check for that _before_ your local SA got a crack at the 
message. Whether you can grab a copy of mail before SA depends on your 
glue.


--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  Win95: Where do you want to go today?
  Vista: Where will Microsoft allow you to go today?
---
 12 days since a sunspot last seen - EPA blames CO2 emissions

Re: Lotto/Money & email address spam

[Mike Cappella]

Hi Karsten,

On 7/23/2009 10:00 AM, Karsten Bräckelmann wrote:

On Thu, 2009-07-23 at 09:22 -0700, Mike Cappella wrote:

Just as an FYI.  I have had (only) one experience where a positive
download contained a corrupted SOUGHT rule file, and an amavis restart
failed.


What exactly do you mean by "corrupt rule file"? Sounds to me like lint
testing should have failed already.


I didn't take the time to ascertain exactly what was wrong with the 
file.  I recall looking at my logs to determine what events took place 
at the time amavis failed.  Only the sought rule files were updated.  I 
attempted to start amavis, but it failed.  I removed the sought file(s), 
and amavis started.




Also, how exactly do you restart amavis after an sa-update? Are you
listening to the exit codes?


Yes, only on exit code 0.  On exit code 1, 2, and others, I output a 
diagnostic.





http://marc.info/?l=amavis-user&m=116189238122548&w=2


Hmm, the advice in the first post is wrong. I don't use amavis myself,
but AFAIK you actually do need to restart amavis after any rule changes,
contrary to what the post claims. Same with spamd, FWIW.


This is correct - amavis absolutely does need to be restarted for any SA 
changes (eg. rules, code, configuration changes).  I read the poster's 
response "AFAIK, no." as a response to the statement:


  According to << the wiki URL >>,
  SA will use them "automatically".

I didn't judge that script to be complete, or the only script acting 
upon the problem (there could be another, that evaluates the output). 
Regardless, the general idea was there.




Also, one does not need to call 'spamassassin' to lint check. sa-update
does this. (Any lint check is not strictly necessary anyway, but a very
sensible safety measure.)



http://marc.info/?l=amavis-user&m=116213281631150&w=2


And here someone mixes up the dirs. sa-update does not use the site-
config dir for its channels -- that is, not unless one specifically
tells it to, which isn't a good idea.


I recall at that time there being much confusion on the amavis list by 
many people as to exact what happened with sa-update, regarding new 
rules and which directories were used various platforms, and what needed 
to occur for amavis to pick up the changes.  Eventually, it was sorted out.


Mike

Re: Lotto/Money & email address spam

[Karsten Bräckelmann]

On Thu, 2009-07-23 at 09:22 -0700, Mike Cappella wrote:
> Just as an FYI.  I have had (only) one experience where a positive 
> download contained a corrupted SOUGHT rule file, and an amavis restart 
> failed.

What exactly do you mean by "corrupt rule file"? Sounds to me like lint
testing should have failed already.

Also, how exactly do you restart amavis after an sa-update? Are you
listening to the exit codes?


> http://marc.info/?l=amavis-user&m=116189238122548&w=2

Hmm, the advice in the first post is wrong. I don't use amavis myself,
but AFAIK you actually do need to restart amavis after any rule changes,
contrary to what the post claims. Same with spamd, FWIW.

Also, one does not need to call 'spamassassin' to lint check. sa-update
does this. (Any lint check is not strictly necessary anyway, but a very
sensible safety measure.)

> http://marc.info/?l=amavis-user&m=116213281631150&w=2

And here someone mixes up the dirs. sa-update does not use the site-
config dir for its channels -- that is, not unless one specifically
tells it to, which isn't a good idea.


-- 
char *t="\10pse\0r\0dtu...@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}

Re: [NEW SPAM FLOOD] www.shopXX.net

[Martin Gregorie]

On Thu, 2009-07-23 at 12:25 -0400, Dan Schaefer wrote:
> > Are you quite sure that an upstream copy of SA, e.g. in your ISP or at a
> > sender site that scans for outgoing spam, hasn't already added X-*
> > headers to the message?
> >
> >
> > Martin
> >
> >   
> No. Is that even possible to track down?
> 
Sure - look at any incoming message's headers to see if there are any
that didn't come from your copy of SA. Each set has a
X-spam-checker-version header that gives the name of the SA host that
added that header set. If that's a possibility, just make sure your
filter ignores header sets that aren't yours. AFAIK your SA header set
it always the first in the message headers.


Martin
 

Re: [NEW SPAM FLOOD] www.shopXX.net

[John Hardin]

On Thu, 23 Jul 2009, Dan Schaefer wrote:


 Are you quite sure that an upstream copy of SA, e.g. in your ISP or at
 a sender site that scans for outgoing spam, hasn't already added X-*
 headers to the message?


No. Is that even possible to track down?


There would probably be an X-Spam-Checker-Version header in your inbound 
mail stream.


--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  Perfect Security and Absolute Safety are unattainable; beware
  those who would try to sell them to you, regardless of the cost,
  for they are trying to sell you your own slavery.
---
 12 days since a sunspot last seen - EPA blames CO2 emissions

Re: [NEW SPAM FLOOD] www.shopXX.net

[Dan Schaefer]

 Are you quite sure that an upstream copy of SA, e.g. in your ISP or at
 a sender site that scans for outgoing spam, hasn't already added X-*
 headers to the message?


No. Is that even possible to track down?


There would probably be an X-Spam-Checker-Version header in your 
inbound mail stream.


X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on 
pony.performanceadmin.com


That is my server.

--
Dan Schaefer
Web Developer/Systems Analyst
Performance Administration Corp.

Re: [NEW SPAM FLOOD] www.shopXX.net

[Dan Schaefer]

Are you quite sure that an upstream copy of SA, e.g. in your ISP or at a
sender site that scans for outgoing spam, hasn't already added X-*
headers to the message?


Martin

  

No. Is that even possible to track down?

--
Dan Schaefer
Web Developer/Systems Analyst
Performance Administration Corp.

Re: Lotto/Money & email address spam

[Mike Cappella]

On 7/23/2009 3:34 AM, Karsten Bräckelmann wrote:

Yes.  SA will use the updates as provided by sa-update, when available.
All you need to do is to restart your daemon, IFF there have been any
updates.


Just as an FYI.  I have had (only) one experience where a positive 
download contained a corrupted SOUGHT rule file, and an amavis restart 
failed.


For the OP, here are two links to scripts that may help you:

http://marc.info/?l=amavis-user&m=116189238122548&w=2
http://marc.info/?l=amavis-user&m=116213281631150&w=2

---
Mike

Re: [NEW SPAM FLOOD] www.shopXX.net

[Martin Gregorie]

Dan Schaefer wrote:
>
> If this is the case, then why does my email have the X-* headers in 
> it? I have nothing in my postfix header_checks to discard the BL 
> rules. Does anyone have a detailed flow chart of SA/postfix setup and 
> describes blacklisting? Or even a webpage describing the process?
> 
Are you quite sure that an upstream copy of SA, e.g. in your ISP or at a
sender site that scans for outgoing spam, hasn't already added X-*
headers to the message?


Martin

Re: copy spam mail to separate mailbox

[Martin Gregorie]

On Thu, 2009-07-23 at 10:48 -0400, Dan Schaefer wrote:
> > I have a postfix/SA setup and I was wondering if anyone knew how to 
> > COPY an email marked as spam instead of redirecting.
> > Not this:
> > /^X-Spam-Flag: YES/   REDIRECT spam...@example.com
> >
> This should work, right?
> http://onetforum.com/fourm/viewtopic.php?f=2&t=34
> 
The spamfilter script in it looks to be incomplete. Take a look at this
example instead: http://onetforum.com/fourm/viewtopic.php?t=26

You need to replace the pipeline in the script with commands that:

1) save the script's arguments 
2) store the spamc output in a temporary file.
3) look at the stored message with grep or awk to see if its spam.
   Either will do the job, but awk will be faster because it can
   be scripted to just look at the headers.
4) If the message is spam, replace the last argument (the recipient)
   with the address that should receive spam.
5) use sendmail to pass the message back to Postfix using the
   arguments stored in (1) and modified in (4)
6) delete the file.

Of course, if performance is important you can write this as a C program
that stores the message in memory while its deciding what to do with it.


Martin

Re: [NEW SPAM FLOOD] www.shopXX.net

[Charles Gregory]

On Wed, 22 Jul 2009, Dan Schaefer wrote:

For those of you that manage these rules,
URI_OBFU_X9_WS, URI_OBFU_WWW, AE_MEDS38, AE_MEDS39 did not mark this 
email as spam

http://pastebin.com/m40f7cff4


The URI is not obfuscated, therefore it triggered the URIBL tests 
properly (and scored 3 additional points from them).


- C

Re: rule counter

[Karsten Bräckelmann]

On Thu, 2009-07-23 at 09:05 -0400, Dan Schaefer wrote:
> I don't have a test server to try this, so maybe someone could test it 
> for me or maybe someone has tried this before...

You don't need a dedicated test server for that. Access to SA is
sufficient. You *do* however need some sort of testing your rules before
pushing them live. Can't outsource that to us.

> I want to create a rule that counts the number of rules that have a 
> score and add my score. I don't want to count the total score. Can I use 
> !! instead of the rule name.
> 
> I may not have explained it well, so here's an example. I want at least 
> 3 out of 5 rules to pass before adding my 5 points.
> 
> #(!!1.25) = true = 1? in SA
> meta   MY_RULE   (!!RULE_ONE + !!RULE_TWO + !!RULE_THREE + !!RULE_FOUR + 
> !!RULE_FIVE >= 3)
> score   MY_RULE   5

Creative. ;)  I didn't try it, but I guess that should work. Anyway...

> instead of the following, because the total score would be counted

This is wrong.

> meta   MY_RULE   (RULE_ONE + RULE_TWO + RULE_THREE + RULE_FOUR + 
> RULE_FIVE >= 3)

This simple rule already does exactly what you want. SA does *not* use
the rule's scores when adding. See the Conf docs on meta rules.

You can verify this easily yourself (output snipped).

$ echo -e "\n" | spamassassin --cf="meta FOO (MISSING_MID + MISSING_DATE > 1)"
*  0.0 MISSING_MID Missing Message-Id: header
*  0.0 MISSING_DATE Missing Date: header
*  1.0 FOO FOO


Note though, that meta and eval rules in your expression *might* have a
value other than 0 or 1. I believe, however, that eval rules generally
do return a boolean value. Also, meta rules are most likely coded to
return a boolean value, too. (Like your rule does, e.g.)

And of course, tflags multiple rules will return the number they hit.
Other than these caveats, in that expression above, any rule will indeed
evaluate to a boolean value. *Not* the rule's score, in no case.


Actually looking at other folks rules might proof enlightening, too.
Like this example, that does what you are asking for.

$ grep '^meta DIGEST_MULTIPLE' 20_net_tests.cf 
meta DIGEST_MULTIPLERAZOR2_CHECK + DCC_CHECK + PYZOR_CHECK > 1


-- 
char *t="\10pse\0r\0dtu...@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}

Re: [NEW SPAM FLOOD] www.shopXX.net

[Bowie Bailey]

Dan Schaefer wrote:


It means that if you were using BL at MTA level your SA might never 
have seen the message at all.


No your rule would not be "overlooked" 'because the site is in a 
blacklist' *unless* you were using the BL in your MTA and rejected 
the transaction from a blacklisted IP address and, thus, never 
submitted it to SA at all.


  
If this is the case, then why does my email have the X-* headers in 
it? I have nothing in my postfix header_checks to discard the BL 
rules. Does anyone have a detailed flow chart of SA/postfix setup and 
describes blacklisting? Or even a webpage describing the process?


It's very simple with Postfix or any other MTA.

1) Connection request comes to Postfix.
2) Postfix checks the sending server against its blacklists.  If it 
matches, the mail is refused.
3) Postfix checks its normal rules and if the sender/recipient/etc is 
ok, the message is accepted.

4) Postfix sends the message to SA.
5) SA scores the message and returns it to Postfix (SA blacklists simply 
score 100 points).
6) Postfix can now deliver, quarantine or delete the message based on 
the score or spam/ham designation returned by SA.


--
Bowie

Re: copy spam mail to separate mailbox

[Dan Schaefer]

I have a postfix/SA setup and I was wondering if anyone knew how to 
COPY an email marked as spam instead of redirecting.

Not this:
/^X-Spam-Flag: YES/   REDIRECT spam...@example.com


This should work, right?
http://onetforum.com/fourm/viewtopic.php?f=2&t=34

--
Dan Schaefer
Web Developer/Systems Analyst
Performance Administration Corp.

Re: [NEW SPAM FLOOD] www.shopXX.net

[Dan Schaefer]

It means that if you were using BL at MTA level your SA might never have seen 
the message at all.

No your rule would not be "overlooked" 'because the site is in a blacklist' 
*unless* you were using the BL in your MTA and rejected the transaction from a 
blacklisted IP address and, thus, never submitted it to SA at all.

  
If this is the case, then why does my email have the X-* headers in it? 
I have nothing in my postfix header_checks to discard the BL rules. Does 
anyone have a detailed flow chart of SA/postfix setup and describes 
blacklisting? Or even a webpage describing the process?


--
Dan Schaefer
Web Developer/Systems Analyst
Performance Administration Corp.

Re: Pyzor or DCC

[Jeff Mincy]

   From: Jonas Eckerman 
   Date: Thu, 23 Jul 2009 15:37:11 +0200
   
   Michael Hutchinson wrote:
   
   >> I saw a test
   >> message with just the word test in the subject hit DCC once.
   
   > That's really strange, I don't see how DCC would fire on the subject..
   > the checksum of the message must have somehow matched some Spam.. 
   
   That's perfectly normal. DCC doen't just match spam, it matches things 
   that has been seen before. That means it matches bulk, but also anything 
   that happens to be very common for other reasons.

yep.
   
   I imagine that an empty message with the subject "test" is pretty 
   common, so it's perfectly reasonable for DCC to have seen such messages 
   many times before.
   
   I don't know if DCC cares about the subject att all. If it doesn't, it's 
   even more liekey that it would hit on an empty test message.
   
   /Jonas

DCC does hit on empty messages.   The empty messages can be
whitelisted.   The DCC distribution includes a fetch-testmsg-whitelist
script:

% head /usr/src/dcc-1.3.111/misc/fetch-testmsg-whitelist
#!/bin/sh

# Fetch a list of "empty" mail messages for whitelisting.  Many free mail
#   service providers add HTML or other text to mail.  That causes empty
#   and nearly empty mail messages to have valid DCC checksums and not be
#   ignored by DCC clients.

# The fetched file can be included in whiteclnt files.  For example, the
#   following line in /var/dccwhiteclnt would whitelist many common
#   empty messages

Re: unsubscribe

[Evan Platt]

At 06:17 AM 7/23/2009, Rick Duval wrote:

unsubscribe


As is stated in every header:

list-unsubscribe:  

Re: spamd socket partial read

[Filippo Carletti]

> This plugin sends the full mail to spamd, but spamd reads only the
> headers and wait for 300 seconds for the body (already sent by
> qpsmtpd).

Solved.

The ipp2p (p2p block) firewall rules were active on localhost and
blocking connection to spamd via socket on port 783 for that
particular mail.
Sorry for the noise.

-- 
Ciao,
Filippo

Re: Pyzor or DCC

[Jonas Eckerman]

Michael Hutchinson wrote:


I saw a test
message with just the word test in the subject hit DCC once.



That's really strange, I don't see how DCC would fire on the subject..
the checksum of the message must have somehow matched some Spam.. 


That's perfectly normal. DCC doen't just match spam, it matches things 
that has been seen before. That means it matches bulk, but also anything 
that happens to be very common for other reasons.


I imagine that an empty message with the subject "test" is pretty 
common, so it's perfectly reasonable for DCC to have seen such messages 
many times before.


I don't know if DCC cares about the subject att all. If it doesn't, it's 
even more liekey that it would hit on an empty test message.


/Jonas
--
Jonas Eckerman
Fruktträdet & Förbundet Sveriges Dövblinda
http://www.fsdb.org/
http://www.frukt.org/
http://whatever.frukt.org/

unsubscribe

[Rick Duval]

unsubscribe

Re: [NEW SPAM FLOOD] www.shopXX.net

[Dan Schaefer]

>For those of you that manage these rules,
>URI_OBFU_X9_WS, URI_OBFU_WWW, AE_MEDS38, AE_MEDS39 did not mark this 
email as spam


I'm up to AE_MED45, so I wouldn't expect AE_MEDS38 and 39 to be 
hitting anything currently.


>http://pastebin.com/m40f7cff4

This is not an obfuscated domain.  You can see that it hit two URIBLs 
- JP and WS.  I would have expected it to be in URIBL_BLACK (or at 
least GOLD) as well as Invaluement's URIBL.  There are plenty of 
mechanisms to catch valid URIs - that's not the purpose of the 
obfuscation rules.


And, you still got 15 points - so, what's the problem?

Relax. I don't have a problem. I was just pointing out a potential flaw. 
I was just trying to help out. I just misunderstood the whole blacklist 
thing, that's all.


--
Dan Schaefer
Web Developer/Systems Analyst
Performance Administration Corp.

rule counter

[Dan Schaefer]

I don't have a test server to try this, so maybe someone could test it 
for me or maybe someone has tried this before...
I want to create a rule that counts the number of rules that have a 
score and add my score. I don't want to count the total score. Can I use 
!! instead of the rule name.


I may not have explained it well, so here's an example. I want at least 
3 out of 5 rules to pass before adding my 5 points.


#(!!1.25) = true = 1? in SA
meta   MY_RULE   (!!RULE_ONE + !!RULE_TWO + !!RULE_THREE + !!RULE_FOUR + 
!!RULE_FIVE >= 3)

score   MY_RULE   5

instead of the following, because the total score would be counted
meta   MY_RULE   (RULE_ONE + RULE_TWO + RULE_THREE + RULE_FOUR + 
RULE_FIVE >= 3)


Thanks in advance,
Dan Schaefer
Web Developer/Systems Analyst
Performance Administration Corp.

Re: [NEW SPAM FLOOD] www.shopXX.net

[Daniel J McDonald]

On Thu, 2009-07-23 at 07:34 +0100, rich...@buzzhost.co.uk wrote:
> It's catching on :-)

this new obfuscation is already caught by AE_MED45, but I can foresee a
variant that might not match...

How about:

body__MED_OB
/\bw{2,3}(?:[[:punct:][:space:]]{1,5}|[[:space:][:punct:]]{1,3}dot[[:space:][:punct:]]{1,3})[[:alpha:]]{0,6}\d{2,6}(?:[[:punct:][:space:]]{1,5}|[[:space:][:punct:]]{1,3}dot[[:space:][:punct:]]{1,3})(?:c\s?o\s?m|n\s?e\s?t|o\s?r\s?g)[[:punct:]]?\b/i
body__MED_NOT_OB/\bw{2,3}\.[[:alpha:]]{0,6}\d{2,6}\.(?:com|net|org)\b/i
metaAE_MED46(__MED_OB && ! __MED_NOT_OB)
describeAE_MED46Shorter rule to catch spam obfuscation
score   AE_MED464.0

-- 
Dan McDonald, CCIE #2495, CISSP# 78281, CNX
www.austinenergy.com

Re: URL Block Lists

[Adam Stephens]

MySQL Student wrote:

Hi,

What is the preferred list of URL block lists that everyone uses? I'm
currently using SURBL and a few others, often times there are URLs
like 'learningbetter.net' that isn't tagged.

We've set up our own internal URL block list that gets trained
manually by inspecting email visually, until the URL is added to URIBL
or SURBL, but I must be missing something, because lately there are
far too many not being tagged.

Thanks,
Alex
  


there are a few others, but they're not as effective as URIBL or SURBL:

uribl.swinog.ch
url.rbl.jp
block.rhs.mailpolice.com

Instead of running a local URL blocklist, you can submit to URIBL 
yourself; requests are generally processed promptly and that way 
everyone benefits.


cheers,
Adam.

--

Adam Stephens
Network Specialist - Email & DNS
adam.steph...@bristol.ac.uk

Re: Lotto/Money & email address spam

[Karsten Bräckelmann]

On Wed, 2009-07-22 at 22:34 -0400, some Alex wrote:
> Okay, I have configured sa-update to download the following rulesets:

> Do people have a script that lints the rules, copies them to
> /etc/mail/spamassassin/ and restarts amavisd?

NO.

sa-update lint checks the rules in a sandbox, and does not update the
local channel, if there are any issues. Moreover, do NOT copy these
updates to your site config dir -- but keep it in the update dir where
sa-update puts them [1]. SA knows how to use them instead of the
"install-time" default conf.

See 'man spamassassin', section Configuration Files. And of course the
wiki docs, specifically about sa-update.

> SA should automatically pick up on the new rules, correct?

Yes.  SA will use the updates as provided by sa-update, when available.
All you need to do is to restart your daemon, IFF there have been any
updates.

I strongly suggest you do some documentation reading.


It's a rather bizarre picture I'm sensing here. From your recent posts I
understand you are running a mail server for a large organization. Yet
there is this cannonade with rather basic questions...

  guenther


[1] And get rid of the ones you already copied. Site config overwrites
default config.

-- 
char *t="\10pse\0r\0dtu...@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}

Re: Avoid processing of email with specific headers

[Martin Gregorie]

On Thu, 2009-07-23 at 04:14 +0300, Jari Fredriksson wrote:
> If you postfix calls  SpamAssasin directly as configured
> in /etc/postfix/master.cf I have no ideas.
> 
I think you can run a script that calls spamc rather than spamc itself.
The script could use grep or (better) awk[1] to search for specific
headers, though of course this will impact performance.
 
Caution: I haven't tried this, so proceed with caution.

[1] unlike grep, awk can be limited to scanning headers only.


Martin

Re: Spam troubleshooting

[Matus UHLAR - fantomas]

> >> Can I also ask where the best place to start with to implement razor
> >> and/or pyzor in SA3.2 on Linux with postfix?
> >
> > EHM? implement it on your mailserver...

On 22.07.09 22:38, MySQL Student wrote:
> Heh, no, I mean where can I go to learn how to implement it? Where's
> the docs? :-)

well, install razor, configure SA to load razor plugin ... that's all.

> I think I'm headed towards razor first, as it doesn't require python
> and appears to be simpler and more effective, even?

yes.
-- 
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
REALITY.SYS corrupted. Press any key to reboot Universe.

Re: Pyzor or DCC

[Matus UHLAR - fantomas]

> > > If you get an E-Mail scoring in both Pyzor and DCC, the chances are
> > > very high that the message is Spam. We only deal with around 90,000
> > > incoming delivery attempts per day - but have not had a false
> > > positive from Pyzor or DCC yet, and have been using both for some
> > > years.
> > >
> > That's odd, I get quite a lot of DCC FPs and a few Pyzor FPs on a
> > relatively small amount of email. They tend to hit on bulk mail, like
> > newsletters, automated mail and very generic mails. I saw a test
> > message with just the word test in the subject hit DCC once.

On 23.07.09 12:06, Michael Hutchinson wrote:
> That's really strange, I don't see how DCC would fire on the subject..
> the checksum of the message must have somehow matched some Spam.. 

we can assume that empty messages are quite common for any reason, so they
really may get listed in those services. Don't send empty mail...

> Actually now that you mention it, I had to deal with a newsletter FP
> some time ago that was being hit by Pyzor, and then was being pushed
> over the threshold thanks to the formatting of the E-Mail. I had
> whitelisted the address for a while, and after testing about a month
> later, Pzyor was no longer hitting it, and whitelisting was removed.

did you whitelist it locally or at pyzor? As I mentioned before, some of
pyzor reporters seem to report automatically, which may list other types of
bulk messages that are not really spams..

> > I've not seen anything FP on both though, and BAYES would
> > probably prevent an FP if it did.
> 
> Agreed, although the score assignments of Pyzor and DCC may warrant a
> review by some mail administrators - to be sure to be sure (that mail
> does flow).

pyzor interface needs rewriting to work with whitelist count too, not just
report count ...
-- 
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Micro$oft random number generator: 0, 0, 0, 4.33e+67, 0, 0, 0...