RE: [Mimedefang] Filtering based on X-Mailer or X-MIMEOLE header?
Just noticed that someone has been doing this within SpamAssassin already - sa_update grabbed this in 80_additional.cf : meta SPAMMY_XMAILER (__XM_OL_29196700||__XM_OL_41332400||__XM_OL_48071700||__XM_OL_28001441||__XM_OL_29196600||__XM_OL_49631700||__XM_OL_48072300||__XM_OL_28004682||__XM_OL_10_0_4115||__XM_OL_4_72_2106_4) describe SPAMMY_XMAILER X-Mailer string is common in spam and not in ham score SPAMMY_XMAILER 1.0 Paul. -- --- Paul Murphy Head of I.T. Argenta Discovery Tel. 01279 645 554 Fax. 01279 645 646 ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] Filtering based on X-Mailer or X-MIMEOLE header?
> -Original Message- > From: John Rudd > Shouldn't that last bit be: > > [12456789]\d$/ > > > As you've got it, it will reject an offset of xx30, which, as > you pointed out, is valid. Actually, the vast majority would be caught with [2468]0$/ I've quarantined thousands of these since I put the rule in on Tuesday. Only about 10 have been xx30, the rest would have been caught by this modification. Also, I don't add enough points to block based on this rule alone. The Received rule, yes. This one is just sort of a placeholder so I can investigate if I start seeing a bunch that match on the Date rule but no longer match on the forged Received rule. ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Filtering based on X-Mailer or X-MIMEOLE header?
John Rudd wrote:
>>header ECC_ODD_TZ Date =~
>>/^\s*(?:Sun|Mon|Tue|Wed|Thu|Fri|Sat)\,\s\d{1,2}\s(?:Jan|Feb|Mar|Apr|Jun|
>>Jul|Aug|Sep|Oct|Nov|Dec)\s\d{4}\s\d{2}(?:\:\d{2}){1,2}\s[\+\-]?\d{2}[123
>>456789]\d$/
>>
>>
>>
>
>Shouldn't that last bit be:
>
>[12456789]\d$/
>
>
>As you've got it, it will reject an offset of xx30, which, as you
>pointed out, is valid.
>
>
It will reject 0545 as well...
-Philip
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID. You may ignore it.
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Filtering based on X-Mailer or X-MIMEOLE header?
header ECC_ODD_TZ Date =~
/^\s*(?:Sun|Mon|Tue|Wed|Thu|Fri|Sat)\,\s\d{1,2}\s(?:Jan|Feb|Mar|Apr|Jun|
Jul|Aug|Sep|Oct|Nov|Dec)\s\d{4}\s\d{2}(?:\:\d{2}){1,2}\s[\+\-]?\d{2}[123
456789]\d$/
Shouldn't that last bit be:
[12456789]\d$/
As you've got it, it will reject an offset of xx30, which, as you
pointed out, is valid.
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID. You may ignore it.
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] Filtering based on X-Mailer or X-MIMEOLE header?
Mark Damrose wrote:
> I've found that most of the stock spam have a unique Received
header.
> Two rules that have been doing extremely well for me are:
> header ECC_FORGED_ELGIN_RCVD Received =~ /by elgin.edu with esmtp
> \(.+\)\s+id\s\S+\s+for/
> header ECC_ODD_TZ Date =~
>
/^\s*(?:Sun|Mon|Tue|Wed|Thu|Fri|Sat)\,\s\d{1,2}\s(?:Jan|Feb|Mar|Apr|Jun|
>
Jul|Aug|Sep|Oct|Nov|Dec)\s\d{4}\s\d{2}(?:\:\d{2}){1,2}\s[\+\-]?\d{2}[123
> 456789]\d$/
Well spotted! That's very useful for me, and certainly almost all of
my recent examples match on this.
Interestingly, they also predominantly have "The Bat!" as the X-Mailer,
although the version details are variable. Maybe 25% have a version of
Outlook instead.
Paul.
--
---
Paul Murphy
Head of I.T.
Argenta Discovery
Tel. 01279 645 554
Fax. 01279 645 646
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID. You may ignore it.
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] Filtering based on X-Mailer or X-MIMEOLE header?
--On Saturday, November 25, 2006 12:07 AM -0600 "Damrose, Mark" <[EMAIL PROTECTED]> wrote: There are time zones that are not an even hour offset from UTC, but the only ones I know of are 30 minutes, and a value of 60 or more makes no sense. Nepal is +0545. Some time ago I implemented the same check as you describe, and thanks to Columbia University's worldwide scope we ran into that one within the first week! Joseph Brennan Lead Email Systems Engineer Columbia University Information Technology ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] Filtering based on X-Mailer or X-MIMEOLE header?
> -Original Message-
> From: Jim McCullars
>I feel your pain. I have gotten to where I check my work
> email at night to see what the latest pump-and-dump stock
> spam is and update SA accordingly. Ugh.
I've found that most of the stock spam have a unique Received header.
Some examples:
Received: from 213.56.31.142 (HELO smtp.oleane.net) by elgin.edu
with esmtp (30,,1N(4829S +/QM) id LLX8Z5-/084()-I* for
[EMAIL PROTECTED]; Fri, 24 Nov 2006 10:31:31 -0060
Received: from 63.149.130.78 (HELO barracuda.1-stopnet.com) by
elgin.edu with esmtp (A+*33AUUHE*U +K686) id 6OM2K4-172DAP-Q/
for [EMAIL PROTECTED]; Fri, 24 Nov 2006 10:43:06 -0480
Received: from 216.122.69.112 (HELO mail.safeserver.com) by
elgin.edu with esmtp ((1+? @C28) id 7;+LH;-FY(844-:7 for
[EMAIL PROTECTED]; Fri, 24 Nov 2006 12:44:18 -0060
Received: from 64.214.48.68 (HELO mdegw01.mgipharma.com) by
elgin.edu with esmtp (942,L96+'P )J4J+,) id QMRGJ0-:PKD)6--L for
[EMAIL PROTECTED]; Fri, 24 Nov 2006 12:49:20 -0060
Received: from 216.35.197.77 (HELO mail.zytronic.com) by elgin.edu
with esmtp (IK-24*R3 U)4UJ) id /ST525-0PO+(5->V for
[EMAIL PROTECTED]; Fri, 24 Nov 2006 12:49:22 -0060
Note the bare IP with no brackets (not the IP of the bot).
HELO random hostname in parentheses.
elgin.edu is my domain, but I do not have a host at the domain level
that relays mail.
Also note the UTC offset in the date format. That field should be HHMM.
There are time zones that are not an even hour offset from UTC, but the
only ones I know of are 30 minutes, and a value of 60 or more makes no
sense.
The Date headers also have the odd UTC offset.
Date: Fri, 24 Nov 2006 10:31:31 -0060
Date: Fri, 24 Nov 2006 10:43:06 -0480
Date: Fri, 24 Nov 2006 10:48:01 -0120
Date: Fri, 24 Nov 2006 11:08:20 -0480
Date: Fri, 24 Nov 2006 11:49:46 -0060
Date: Fri, 24 Nov 2006 12:38:42 -0060
Date: Fri, 24 Nov 2006 12:42:20 -0060
Date: Fri, 24 Nov 2006 12:44:18 -0060
Date: Fri, 24 Nov 2006 12:49:20 -0060
Date: Fri, 24 Nov 2006 12:49:22 -0060
Two rules that have been doing extremely well for me are:
header ECC_FORGED_ELGIN_RCVD Received =~ /by elgin.edu with esmtp
\(.+\)\s+id\s\S+\s+for/
header ECC_ODD_TZ Date =~
/^\s*(?:Sun|Mon|Tue|Wed|Thu|Fri|Sat)\,\s\d{1,2}\s(?:Jan|Feb|Mar|Apr|Jun|
Jul|Aug|Sep|Oct|Nov|Dec)\s\d{4}\s\d{2}(?:\:\d{2}){1,2}\s[\+\-]?\d{2}[123
456789]\d$/
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID. You may ignore it.
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Filtering based on X-Mailer or X-MIMEOLE header?
On Fri, 24 Nov 2006, Paul Murphy wrote: > while I'm updating my SA rules daily, I never seem able to keep ahead of I feel your pain. I have gotten to where I check my work email at night to see what the latest pump-and-dump stock spam is and update SA accordingly. Ugh. Jim McCullars University of Alabama in Huntsville ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Filtering based on X-Mailer or X-MIMEOLE header?
Hi, In the ever-escalating war, I'm having problems with some spammers sending stock scams with large chunks of random text either side, and while I'm updating my SA rules daily, I never seem able to keep ahead of the game with these. Eventually DCC and Razor2 catch up, but the first couple of hours is always a problem and I end up cobbling together my own rules to block specific spams. I've been considering alternative approaches, and one which seems attractive on the surface is to further analyse the message headers for indications of spammyness - we've already got the Received headers, the sender and recipient, and the Subject covered, but can the others be used to provide an indication that the content is spam? SA already considers some of these in deciding things like whether the message claims to be sent using Outlook but doesn't have the correct headers to support this claim, but this is highly specific and not generally very helpful. I decided to look at the X-Mailer and X-MIMEOLE headers specifically, and to extract these in a fairly ad-hoc way for each message, and then add the details of the message, SA score, and mailer to a database table. I'd then propose to adjust the SA score based on an analysis of the history data. The adjustment would be equal to 10% of (the mean score minus one standard deviation), so a small offset in most cases. Obviously my policy is up to me, so no-one has to do this, but I thought I'd share my thoughts and experiences. Applying this across a day's worth of traffic here produces some promising results: select count(*) as cnt, round(sum(score),2) as total, round(avg(score),2) as mean, round(min(score),2) as min, round(stddev(score),2) as stddev, round((avg(score)-stddev(score))/10,2) as adj, left(mailer,50) as mailer from mail_msg where mailer is not null group by left(mailer,50) order by mean; +-+-++-++---++ | cnt | total | mean | min | stddev | adj | mailer | +-+-++-++---++ | 1 | -11.43 | -11.43 | -11.43 | 0.00 | -1.14 | StrongMail Enterprise 3.1.5(2.00.223) | | 22 | -226.31 | -10.29 | -101.21 | 28.81 | -3.91 | Microsoft CDO for Windows 2000 | | 4 | -16.95 | -4.24 | -5.23 | 0.80 | -0.50 | Kana Connect 6 | | 2 | -5.65 | -2.82 | -2.97 | 0.15 | -0.30 | Roving Constant Contact 0 (http//www.constantconta | | 1 | -2.60 | -2.60 | -2.60 | 0.00 | -0.26 | Microsoft Outlook, Build 10.0.6626 | | 1 | -2.60 | -2.60 | -2.60 | 0.00 | -0.26 | Microsoft Outlook IMO, Build 9.0.6604 (9.0.2911.0) | | 4 | -10.05 | -2.51 | -2.58 | 0.04 | -0.25 | Novell GroupWise Internet Agent 6.5.4 | | 1 | -2.51 | -2.51 | -2.51 | 0.00 | -0.25 | Lotus Notes Release 6.5.1 January 21, 2004 | | 1 | -2.47 | -2.47 | -2.47 | 0.00 | -0.25 | BBC EBS Custom Mailer v2 | | 1 | -2.46 | -2.46 | -2.46 | 0.00 | -0.25 | Microsoft Outlook, Build 10.0.4024 | | 2 | -4.76 | -2.38 | -2.41 | 0.03 | -0.24 | Lotus Notes Release 6.5.4 March 27, 2005 | | 1 | -2.35 | -2.35 | -2.35 | 0.00 | -0.23 | Apple Mail (2.750) | | 1 | -2.33 | -2.33 | -2.33 | 0.00 | -0.23 | GlobalCrossing | | 1 | -2.30 | -2.30 | -2.30 | 0.00 | -0.23 | Internet Mail Service (5.5.2658.27)| | 4 | -8.95 | -2.24 | -2.34 | 0.08 | -0.23 | Internet Mail Service (5.5.2653.19)| | 6 | -13.27 | -2.21 | -2.53 | 0.44 | -0.26 | Microsoft Office Outlook 11| | 7 | -14.66 | -2.09 | -2.50 | 0.50 | -0.26 | Microsoft Exchange V6.0.6603.0 | | 10 | -19.32 | -1.93 | -2.22 | 0.12 | -0.21 | Internet Mail Service (5.5.2658.3) | | 2 | -3.85 | -1.92 | -1.93 | 0.01 | -0.19 | Microsoft MimeOLE V6.00.3790.504 | | 2 | -3.50 | -1.75 | -2.60 | 0.85 | -0.26 | Lotus Notes Release 6.5.3 September 14, 2004 | | 10 | -17.15 | -1.72 | -2.09 | 0.22 | -0.19 | ColdFusion MX Application Server | | 6 | -10.25 | -1.71 | -2.30 | 0.77 | -0.25 | Microsoft Exchange V6.5.6944.0 | | 1 | -1.67 | -1.67 | -1.67 | 0.00 | -0.17 | pyroclasticmailsplatterer 0.0.1| | 1 | -1.64 | -1.64 | -1.64 | 0.00 | -0.16 | AOL Email 22250 | | 1 | -1.64 | -1.64 | -1.64 | 0.00 | -0.16 | Microsoft MimeOLE V6.00.2800.1807 | | 1 | -1.62 | -1.62 | -1.62 | 0.00 | -0.16 | Lotus Notes 653HF860 Ju
