Subject: Fusemail Technical Support for Case - 03278437 ref:_00D301Siv._50060Ppgo5:ref – Re: URL spam and RP_MATCHES_RCVD
On 16/04/13 14:28, Ned Slider wrote: In an ideal world, yes. But I suspect the SA auto-generated scoring system is far from an ideal world due to the limited number of contributors to the spam/ham corpus and the fact that your or my mail streams might not accurately reflect those contributed to the corpus. So, if a high scoring (positive or negative) rule hit is clearly causing FP's or FN's in your mail stream then it's generally better to simply nullify that rule, either disabling it by setting the score to zero or by assigning an arbitrary low score for informational purposes thus allowing you to continue to track it's performance whilst not otherwise affecting the overall scoring of the system. Perhaps someone could unsubscribe the role account @fusemail.com that appears to open a support ticket and auto-replies in response to posts to this mailing list :-D Original Message Subject: Subject: Fusemail Technical Support for Case - 03278437 ref:_00D301Siv._50060Ppgo5:ref – Re: URL spam and RP_MATCHES_RCVD Date: Tue, 16 Apr 2013 13:29:34 + (GMT) From: techsupp...@fusemail.com To: n...@unixmail.co.uk Thank you for your inquiry Customer, Thank you for contacting FuseMail® Technical Support with your inquiry. A support agent will respond as soon as possible with more information to help you resolve your issue. In the meantime if you need to contact us again regarding this issue please refer to Case - 03278437. We appreciate your time and consideration and look forward to speaking with you soon. Sincerely, The FuseMail® team www.fusemail.com This email, its contents and attachments contain information from j2 Global Communications, Inc. and/or its affiliates which may be privileged, confidential or otherwise protected from disclosure. The information is intended to be for the addressee(s) only. If you're not an addressee, any disclosure, copy, distribution, or use of the contents of this message is prohibited. If you've received this email in error please notify the sender by reply e-mail and delete the original message and any copies. Thank you. www.j2.com.
Re: URL spam and RP_MATCHES_RCVD
On 16/04/13 00:07, Alex wrote: Hi, But I stand by my local.cf entry reducing RP_MATCHES_RCVD to an advisory -0.001; it may be useful in combination with other rules, but I don't think it's valuable enough on its own to have even -0.5 points. I can't say I've seen any evidence in the mail stream I deal with that scoring down like that is causing either FPs or FNs. These values are automatically generated from presumably tens or hundreds of thousands of messages to determine exactly how it should be weighted, just like all other stock rules, correct? I can understand adjusting the values slightly for each user's particular situation, but doesn't it generally throw off the balance of how the email as a whole is weighted when you adjust values in that way? In an ideal world, yes. But I suspect the SA auto-generated scoring system is far from an ideal world due to the limited number of contributors to the spam/ham corpus and the fact that your or my mail streams might not accurately reflect those contributed to the corpus. So, if a high scoring (positive or negative) rule hit is clearly causing FP's or FN's in your mail stream then it's generally better to simply nullify that rule, either disabling it by setting the score to zero or by assigning an arbitrary low score for informational purposes thus allowing you to continue to track it's performance whilst not otherwise affecting the overall scoring of the system.
Re: URL spam and RP_MATCHES_RCVD
On 15/04/13 18:46, Niamh Holding wrote: Hello Kris, Friday, April 12, 2013, 4:23:55 PM, you wrote: KD> score RP_MATCHES_RCVD -0.551 -1.344 -0.551 -1.344 I'm seeing- score RP_MATCHES_RCVD -0.552 -2.373 -0.552 -2.373 But perhaps there is something odd, I'm seeing that my current version is higher than thr new version- Apr 15 18:44:52.484 [17403] dbg: channel: current version is 1463883, new version is 1462428, skipping channel I was seeing that too, but the latest update (as of now) gives: score RP_MATCHES_RCVD -0.550 -0.556 -0.550 -0.556 dbg: channel: current version is 1467748, new version is 1467748, skipping channel
Re: Checking for email attachment name for containing Javscript code that could get potentially executed when displayed on a webpage.
On 08/03/13 14:05, Sharma, Ashish wrote: Can you pastebin an example? Not sure what you mean with the attachment *name* contains JS code. Here is the requested sample http://pastebin.com/DN7PRnH4 The attachment name contains the javascript code at the bottom of the pasted file. thanks Ashish You could try this (untested): mimeheader L_CT_DOCWRITE Content-Type =~ /document\.write/ score L_CT_DOCWRITE 1 describeL_CT_DOCWRITE Content-type contains document.write Score as you see fit. -Original Message- From: Axb [mailto:axb.li...@gmail.com] Sent: Wednesday, March 06, 2013 3:59 PM To: users@spamassassin.apache.org Subject: Re: Checking for email attachment name for containing Javscript code that could get potentially executed when displayed on a webpage. On 03/06/2013 11:20 AM, Sharma, Ashish wrote: All, I have a mail receiving server that parses incoming emails for email attachment and the files are listed on a web page for users to see. Here I need to check for email attachment name for containing Javscript code that could get potentially executed when displayed on a webpage. Is there a way in Spamassassin conf that can help me in testing for the above mentioned scenario? Can you pastebin an example? Not sure what you mean with the attachment *name* contains JS code.
Re: Yahoo single-link spam common elements
On 02/03/13 01:40, John Hardin wrote:
On Sat, 2 Mar 2013, Ned Slider wrote:
On 01/03/13 19:55, Alexandre Boyer wrote:
The famous 5 recipients...
I had a (very) few exceptions while having the very same pattern in
body. With 4 recipients instead of 5, and sometimes one among the 5
with
no To:address, just To:name, wich was harder to count...
I removed the similar rule as your __RP_D_00040 from my systems to
avoid
false negatives.
And no FP for a long time on this rule (this is an old bot, first saw
last summer, but probably older but unnoticed).
The example I posted earlier today had 7 recipients listed in To:
(sorry, I redacted them).
Rather than using a rule specifically for 5 recipients, I would use
the existing __MANY_RECIPS rule in the meta rule.
That said, I just checked my example, and __MANY_RECIPS failed to
fire. Here's the current rule:
header __MANY_RECIPS ToCc =~ /(?:\@[^@]{5,30}){3}/
Can someone explain the regex and why it fails to fire for 7 recipients?
(@, followed by 5-30 non-@ characters) repeated three times.
If the username + domain name + inter-address punctuation is longer than
30 chars it won't work.
I don't see a good reason for the upper limit, or at least for one that
restrictive. The To and Cc headers aren't going to be unboundedly long.
Thanks John (and others).
Yes, the limit of 30 looks too small. I just created a local rule upping
the limit to 100 and it now fires as expected.
John - can we increase the limit?
Re: Yahoo single-link spam common elements
On 01/03/13 19:55, Alexandre Boyer wrote:
The famous 5 recipients...
I had a (very) few exceptions while having the very same pattern in
body. With 4 recipients instead of 5, and sometimes one among the 5 with
no To:address, just To:name, wich was harder to count...
I removed the similar rule as your __RP_D_00040 from my systems to avoid
false negatives.
And no FP for a long time on this rule (this is an old bot, first saw
last summer, but probably older but unnoticed).
The example I posted earlier today had 7 recipients listed in To:
(sorry, I redacted them).
Rather than using a rule specifically for 5 recipients, I would use the
existing __MANY_RECIPS rule in the meta rule.
That said, I just checked my example, and __MANY_RECIPS failed to fire.
Here's the current rule:
header __MANY_RECIPS ToCc =~ /(?:\@[^@]{5,30}){3}/
Can someone explain the regex and why it fails to fire for 7 recipients?
Re: Yahoo single link spam
On 01/03/13 17:33, David F. Skoll wrote: Somewhat OT... are people still seeing these Yahoo single-link spams? They seem to have stopped abruptly as far as I can tell. Regards, David. Here's one from this morning: http://pastebin.com/cuk595z6 that matches the pattern being discussed.
Re: TBIRD_SPOOF
On 14/02/13 14:48, John Hardin wrote: On Thu, 14 Feb 2013, Ned Slider wrote: Hi list, Is it just me or is TBIRD_SPOOF hitting pretty much all initial email sent by Thunderbird, not via a ML etc? That was an experimental rule that hasn't panned out and has been removed. It should go away with the next update. Many thanks John. And confirmed - it is now gone with the latest sa-update.
Re: hinet.net?
On 14/02/13 14:34, Robert Schetterer wrote: Am 14.02.2013 15:24, schrieb Walter Hurry: Is anyone else being plagued by unreadable nonsense from hinet.net? It originates from China, it seems. I've just had to tell procmail to send it all to the bit bucket. Just curious. Is hinet.net a known problem? yes, hinet is a problem since ever , use ie spamhaus rbl at smtp level to block them Best Regards MfG Robert Schetterer +1 - just block them at the MTA, either with Spamhaus or a local IP blocklist. Here's the list I have for HINET - might not be complete. $ cat /etc/postfix/client.cidr | grep -i hinet 59.112.0.0/14 REJECT # Hinet.net blocked 59.116.0.0/15 REJECT # Hinet.net blocked 59.120.0.0/15 REJECT # Hinet.net blocked 61.216.0.0/13 REJECT # Hinet.net blocked 61.224.0.0/13 REJECT # Hinet.net blocked 111.240.0.0/12 REJECT # Hinet.net blocked 114.24.0.0/14 REJECT # Hinet.net blocked 114.32.0.0/12 REJECT # Hinet.net blocked 118.160.0.0/15 REJECT # Hinet.net blocked 118.164.0.0/14 REJECT # Hinet.net blocked 118.168.0.0/14 REJECT # Hinet.net blocked 122.118.0.0/16 REJECT # Hinet.net blocked 122.120.0.0/13 REJECT # Hinet.net blocked 125.224.0.0/13 REJECT # Hinet.net blocked 218.160.0.0/12 REJECT # Hinet.net blocked 220.128.0.0/12 REJECT # Hinet.net blocked
Re: TBIRD_SPOOF
On 14/02/13 12:04, Ned Slider wrote:
Hi list,
Is it just me or is TBIRD_SPOOF hitting pretty much all initial email
sent by Thunderbird, not via a ML etc?
$ grep TBIRD_SPOOF *.cf
72_active.cf:##{ TBIRD_SPOOF
72_active.cf:meta TBIRD_SPOOF __MUA_TBIRD && !__HAS_IN_REPLY_TO &&
!__HAS_X_REF && !__THREADED && !__VIA_ML && !__NOT_SPOOFED &&
!__HAS_SENDER && !__HAS_ERRORS_TO && !__HAS_X_BEEN_THERE &&
!__RP_MATCHES_RCVD && !ALL_TRUSTED && !__TO_EQ_FROM_DOM
72_active.cf:describe TBIRD_SPOOF Claims Thunderbird mail client but
looks suspicious
72_active.cf:#score TBIRD_SPOOF 3.00 # limit
72_active.cf:##} TBIRD_SPOOF
72_scores.cf:score TBIRD_SPOOF 3.000 2.999 3.000 2.999
If I send myself a simple test message, routed externally so as not to
hit ALL_TRUSTED, I get a hit on this rule. Replies etc are OK but the
initial email hits TBIRD_SPOOF and scores 3pts so it's that initial
first email in a tread that gets clobbered.
Any suggestions how to fix this?
Replying to myself,
I just updated my rules with sa-learn as I hadn't updated for a few
days, and it seems this rule has disappeared. Looks like it was only out
in the wild for a short time.
Thanks to those who replied off list.
Problem solved!
Re: X-Relay-Countries
On 12/02/13 20:33, Daniel McDonald wrote: On 2/12/13 1:15 PM, "David F. Skoll" wrote: PS: Beware of penalizing other countries too much. My mail originates from Canada and the PostgreSQL mailing list is (or used to be?) hosted in Panama. Furthermore, by far the lion's share of spam originates from the US. Yes, of course. But some mail just isn't likely to originate overseas. For example, we have been getting a lot of phishes pretending to be FedEX non-delivery notices. FedEX is based in the US, so if I see "FedEX" and RELAY_NOT_US, and a couple of other spam signs, I can more safely conclude it is spam Nice idea, but why not just use SPF for fedex.com as they bother to publish an SPF record? Surely that has to be a far more reliable indicator it wasn't sent from fedex? $ dig txt fedex.com ;; ANSWER SECTION: fedex.com. 10578 IN TXT "v=spf1 redirect=_spf.infosec.fedex.com" They might sign their mail too, but as I don't have any legitimate fedex mails to hand, I can't confirm that.
TBIRD_SPOOF
Hi list,
Is it just me or is TBIRD_SPOOF hitting pretty much all initial email
sent by Thunderbird, not via a ML etc?
$ grep TBIRD_SPOOF *.cf
72_active.cf:##{ TBIRD_SPOOF
72_active.cf:meta TBIRD_SPOOF __MUA_TBIRD &&
!__HAS_IN_REPLY_TO && !__HAS_X_REF && !__THREADED && !__VIA_ML &&
!__NOT_SPOOFED && !__HAS_SENDER && !__HAS_ERRORS_TO &&
!__HAS_X_BEEN_THERE && !__RP_MATCHES_RCVD && !ALL_TRUSTED &&
!__TO_EQ_FROM_DOM
72_active.cf:describe TBIRD_SPOOF Claims Thunderbird
mail client but looks suspicious
72_active.cf:#score TBIRD_SPOOF 3.00 # limit
72_active.cf:##} TBIRD_SPOOF
72_scores.cf:score TBIRD_SPOOF 3.000 2.999
3.000 2.999
If I send myself a simple test message, routed externally so as not to
hit ALL_TRUSTED, I get a hit on this rule. Replies etc are OK but the
initial email hits TBIRD_SPOOF and scores 3pts so it's that initial
first email in a tread that gets clobbered.
Any suggestions how to fix this?
Re: Calling spamassassin directly yields very different results than calling spamassassin via amavis-new
On 10/01/13 00:03, Ben Johnson wrote: On 1/9/2013 5:36 PM, RW wrote: This is not better, it indicates that SA didn't recognise it as an email, not that it recognised it as a spam. Whatever /tmp/msg.txt was it wasn't a properly formatted email. Thanks for the quick replies, Marius and RW. I see; I saved the email message out of Thunderbird (with View -> Headers -> All), as a plain text file. Apparently, that process butchers the original message. Ben, In thunderbird, select the message and then press Ctrl-U (or from the menus: View > Message Source) and select File > Save to save the email including all headers in plain text format. You can then feed it to spamassassin as above. Hope that helps.
Re: FPs on AXB_XMAILER_MIMEOLE_OL_B054A
On 08/01/13 16:31, Kevin A. McGrail wrote: On 1/8/2013 11:27 AM, Kris Deugau wrote: Ned Slider wrote: Hi, I'd just like to note some FPs on AXB_XMAILER_MIMEOLE_OL_B054A hitting some ham. Rules in this cluster seem to target "obsolete" versions of MSOE and its descendants. See https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6844 for some discussion around a similar rule. I can see the reasoning, but all too often ISP end users do not update their systems, ever, causing these to be seen in live legitimate traffic. My $0.02. Rules often will hit on Spam and Ham so a FP should really be something that causes a Spam or Ham to be categorized incorrectly as a whole. For example, I may write a rule that scores 0.25 that hits on Spam but also some Ham. But I also have rules that are negative to negate the Ham impact. So if a score is particularly high on a single rule or it contributes to mismarking an email, it's a good thing to discuss. If it adds a small amount to a score, that's really not unexpected. So when the rule misfires on the Ham, is the ham still being overall not marked as Spam? Do you see a good amount of hits from the rule on Spam? Regards, KAM Hi Kevin, I absolutely take your point about scoring ham vs spam, and in this case the ham was indeed not misclassified as spam. Bayes was correctly scoring these, either neutrally or as ham. About the only rule hitting with any significant score was AXB_XMAILER_MIMEOLE_OL_B054A. However, in order to improve overall efficiency I do take note and try to investigate when any rule hits on ham, especially when that rule is scored at anything much higher than an informational score. This rule came to my attention as the score has very recently increased from an informational score of 0.001 to a not insignificant 2.121 (and even higher for those not running network tests and/or bayes). If as you suggest it had a score of 0.25 then it almost certainly wouldn't have caught my attention. The fact it is scoring greater than 40% of a spam classification doesn't appear justified from examination of my corpus. I see absolutely no hits in my spam corpus dating back two years and covering over 10,000 messages (I grant small by some standards). I see a small number of hits against ham dating back to June 2012 (perhaps around the time the rule was first introduced?) from a handful of senders. Ultimately it has to come down to rule efficiency and the efficiency of this rule _for me_ is pretty awful even if it's not a huge issue. I see it performs a little better in the official corpus: http://ruleqa.spamassassin.org/20130107-r1429709-n/AXB_XMAILER_MIMEOLE_OL_B054A/detail It's probably fair to say that neither my nor the SA corpus are ideal for judging the true performance of such rules but in each case it's what we have to work with. Having read the bugzilla Kris referenced I do now at least understand a little of the reasoning behind the rule :-) Thanks for the responses.
Re: FPs on AXB_XMAILER_MIMEOLE_OL_B054A
On 08/01/13 16:27, Kris Deugau wrote: Ned Slider wrote: Hi, I'd just like to note some FPs on AXB_XMAILER_MIMEOLE_OL_B054A hitting some ham. Rules in this cluster seem to target "obsolete" versions of MSOE and its descendants. See https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6844 for some discussion around a similar rule. I can see the reasoning, but all too often ISP end users do not update their systems, ever, causing these to be seen in live legitimate traffic. -kgd Thanks Kris for the pointer and discussion.
FPs on AXB_XMAILER_MIMEOLE_OL_B054A
Hi,
I'd just like to note some FPs on AXB_XMAILER_MIMEOLE_OL_B054A hitting
some ham.
# grep _OL_B054A *.cf
72_active.cf:##{ AXB_XMAILER_MIMEOLE_OL_B054A
72_active.cf:meta AXB_XMAILER_MIMEOLE_OL_B054A (__AXB_XM_OL_B054A &&
__AXB_MO_OL_B054A)
72_active.cf:##} AXB_XMAILER_MIMEOLE_OL_B054A
72_active.cf:header __AXB_MO_OL_B054A X-MimeOLE =~ /Produced\ By\
Microsoft\ MimeOLE\ V15\.4\.3555\.308/
72_active.cf:header __AXB_XM_OL_B054A X-Mailer =~ /Microsoft\ Windows\
Live\ Mail\ 15\.4\.3555\.308/
72_scores.cf:score AXB_XMAILER_MIMEOLE_OL_B054A 3.499 2.121
3.499 2.121
The scores seem pretty high for what looks like a hit against a pretty
standard X-Mailer and X-MimeOLE type, or am I missing something here?
I'm not sure I understand the strategy or thinking behind the rule.
Looking back through my mail archives it seems this rule was scoring
0.001 until very recently hence why it probably didn't hit my radar
until now.
Re: A plugin for banned words?
On 30/12/12 19:27, Jari Fredriksson wrote: 30.12.2012 21:09, RW kirjoitti: On Sun, 30 Dec 2012 19:13:01 +0200 Jari Fredriksson wrote: Finally they are getting some Bayes too, and exterbal URIBL databases are recognizing URIs in the payload. So I have now lowered the points on my rule to 5.5. Also created a local anti-DNSWL_MED for mail coming from redhat having this RCVD_IN_DNSWL_MED on. The list appears to be available at gmane.comp.java.jboss.user IIWY I'd look at how well Bayes is doing in the list, it may be that you can safely add a meta rule to boost the score for the higher scoring bayes rules in the list, and then add a few low-scoring subject rules for "Vs" and some of the other common words. I have not received ham from that list today. Bayes was very slow to adapt, and now that it finally gets usually between 60-90% it begins to work. But I am very afraid the ham from that list will get the same points! The email is pure HTML, looks like a "page" from their discussion site, and it has very much common in ham& spam. Remains to be seen how well bayes copes with it. Had you in the past trained bayes with a large amount of ham from that list? I would imagine that would explain why you would then need to train many spam from the same source before you see any change in the behaviour of bayes. Most of the tokens are going to look the same to bayes as it's all from the same source and the content differs only slightly.
Re: A plugin for banned words?
On 30/12/12 18:44, Jari Fredriksson wrote: 30.12.2012 20:25, Ned Slider kirjoitti: Where is this list? I can't find a jboss-us...@redhat.com (on Red Hat mailing lists, see https://www.redhat.com/mailman/listinfo), only a jboss-u...@lists.jboss.org on lists.jboss.org. Can you point to some example spam in the list archives please. Sorry, I was a bit inaccurate with the list name.. List-Id: The JBoss User main mailing list List-Unsubscribe:<https://lists.jboss.org/mailman/listinfo/jboss-user>, <mailto:jboss-user-requ...@lists.jboss.org?subject=unsubscribe> List-Archive:<http://lists.jboss.org/pipermail/jboss-user> List-Post:<mailto:jboss-u...@lists.jboss.org> List-Help:<mailto:jboss-user-requ...@lists.jboss.org?subject=help> List-Subscribe:<https://lists.jboss.org/mailman/listinfo/jboss-user>, <mailto:jboss-user-requ...@lists.jboss.org?subject=subscribe> It's spamming me faster than I can process it. I report to SpamCop, Pyzor& Razor via SpamAssassin's report function, and the queue just keeps growing ;) Ah yes, it's not a Red Hat run list, but rather something run by jboss.org (of course now owned by Red Hat) I suspect this is an issue that they need to sort out their end rather than something all their users need to resolve. They shouldn't be emitting spam at those levels. As Martin noted, it appears to be some forum software setup also linked to the mailing list. No doubt the admin is away for the holiday season and the spammers are running rampant. When sites become overrun like that they very quickly become unusable. If it were me I'd suspend delivery from the list until they are able to resolve their spam issue.
Re: A plugin for banned words?
On 30/12/12 17:13, Jari Fredriksson wrote: 30.12.2012 18:05, Jari Fredriksson kirjoitti: So far I have created many meta rules containing those words, but the list is endless. The words like mostly U.S. town names and U.S. sports team names. On the face of it that doesn't sound too difficult. In addition to that the spam contains words "game", "basketball", "football", "live", "vs.", "at", "tv" and "pc". That sounds really error prone - particularly the last five words. Seems to be a brain dead idea from JBoss/Red Hat to create such a system in internet - web gui for discussions - anonymize the senders and send the messages w/o any clue who really created the content Those simple words can not alone do anything, but something like this: List-Id: jboss-us...@redhat.com Subject: [jboss-users] [jBPM] - I watch New York from my TV and found a live Buffalo on my free PC Where is this list? I can't find a jboss-us...@redhat.com (on Red Hat mailing lists, see https://www.redhat.com/mailman/listinfo), only a jboss-u...@lists.jboss.org on lists.jboss.org. Can you point to some example spam in the list archives please. That might trigger my rule. But it is JBoss Business Process management tool mailing list, so that has not yet happened. Thousands of spam this day alone, not a single false positive found so far. Finally they are getting some Bayes too, and exterbal URIBL databases are recognizing URIs in the payload. So I have now lowered the points on my rule to 5.5. Also created a local anti-DNSWL_MED for mail coming from redhat having this RCVD_IN_DNSWL_MED on. I wonder what the attacker is up to. Has she just figured a way thru all anti spam measures, or trying to case damage to Red Hat somehow... It's only jBPM sub list of JBoss so far...
Re: Scoring Yahoo mail from certain continents/countries ?
On 09/12/12 10:16, Frederic De Mees wrote: Dear list, Here is the context. The French-speaking countries receive tons of e-mails, mostly fraud attempts, fake lotteries, originating from West-Africa and sent by Yahoomail users. Often those messages contain big attachments. The payload (text of the message) is embedded in a 1MB jpeg with fake certificates of a lawyer, a logo, or whatever. Spamassassin misses 100% of them because: - the sender IP (Yahoo) is genuine and has a good reputation - the analysis of the message text shows nothing bad, as the mill!ions of euros are in the picture attachment - due to the message size, the analysis is skipped anyway. If no customer of the mail server in question expect any mail from any Yahoo user in Africa, a simple 'header_checks' Postfix directive like this will match such messages if their sender IP starts with 41. /^Received: from .41\..*web.*mail.*yahoo\.com via HTTP/i I admit this is rough albeit effective. On one side, not all Africa is 41. On the other side, I do not want to block all 41. I would have loved to do it with SA. This means that the line "Received: from [ip.add.res.ss].*web.*mail.*yahoo\.com via HTTP" should be detected and analysed. The ip address should be extracted. The whois of the address should be queried. The country code of the IP address would return certain number of SA points from a list of "Yahoousers bad countries" I would manage. Have I dreamed ? Frédéric Brussels There are already 2 meta rules related to mail from 41/8 subnets: __NSL_ORIG_FROM_41 __NSL_RCVD_FROM_41 Write some meta rules combining these with a rule for mail from Yahoo! or look at scoring __FROM_41_FREEMAIL which already combines the above with FREEMAIL_FROM. That should be enough to get you started.
Re: "Fairly-Secure" Anti-SPAM Gateway Using OpenBSD, Postfix, Amavisd-new, SpamAssassin, Razor and DCC ? Can I get your opinion?
I'll expand a little on John's comments below On 29/11/12 18:44, John Hardin wrote: On Thu, 29 Nov 2012, Ed Flecko wrote: I'll be sure to check into Postgrey. Are there any special considerations to installing/configuring it or is it simply a matter of installing, reading the docs and configuring? The biggest consideration is not technical, it's managing the expectations of your users. You will need to educate your users that email is *not* instant messaging. Indeed. But do also play around with the delays in postgrey (--delay). A minimal delay of 60 seconds is enough to force a retry and is adequate - legit hosts will retry, non-legit hosts won't so a longer delay is generally unnecessary. You will probably want to put a little effort into maintaining lists of regular correspondents who can bypass greylisting. There may be tools to automate that, e.g. to whitelist someone a local user has sent mail to. Postgrey has an auto-whitelisting mechanism that can be fine tuned by reducing the number of times a client must successfully retry (--auto-whitelist-clients) before auto-whitelisting and adjusting the age of the cache (--max-age) so whitelisted clients are cached for longer. Generally after a couple weeks of normal mail flow, all regular hosts should be cached so only new contacts will get greylisted. Also don't be afraid to whitelist big clients that you receive correspondence from - you know they are legit and will resend so it's pointless greylisting them. Postgrey is very configurable and all the options above are documented in the manpage. Some users are extremely allergic to any delays in their email; you may have to maintain a list of exception destination addresses to keep them happy, or for addresses where no delay is acceptable, e.g. or
Re: "Fairly-Secure" Anti-SPAM Gateway Using OpenBSD, Postfix, Amavisd-new, SpamAssassin, Razor and DCC ? Can I get your opinion?
On 28/11/12 23:32, Ed Flecko wrote: I'm looking to set up a spam filtering server to replace our ISP's spam filtering service. I've seen this tutorial ( ftp://orn.mpg.de/pub/unix/mail/Fairly-Secure_Anti-SPAM_Gateway_Using_SpamAssassin.html#antivirus ) and I'd be very interested in YOUR opinion; do you think, fundamentally, a server with these software packages could be an effective combination at fighting spam? We're a (I guess) medium size organization with appx. 1000 end users. What about weaving clam-av into the mix? Although this tutorial uses OpenBSD, I'll probably be using FreeBSD. Thank you for your input! :-) Ed I use Postfix with Amavisd-new which allows SpamAssassin and Clam-AV to be easily integrated. I also use Postgrey for greylisting. I find this setup very flexible and efficient. Clam-AV doesn't catch a huge amount on my mail flow - email borne trojans/viruses don't seem to be overly popular these days. You can get 3rd party signatures for things like phishing although I've never tried these as I've trained SA to do a good job on catching phishing emails. I'm running on Linux (RHEL5) but I guess the base OS is largely irrelevant so I'd use what you are comfortable with. I guess there are many ways to skin this particular cat but the above setup works very well for me. In other words, I suspect you will get a number of different answers all providing effective solutions based around the use of SpamAssassin and/or Clam-AV. The difference mostly seems to be how you choose to integrate them into your mail server.
MSGID_RANDY is firing on hams from chtah.net
Hi,
MSGID_RANDY is firing on hams from chtah.net during the last couple
days, with a not insignificant score of 2.599.
Here is the current rule:
20_head_tests.cf:header __MSGID_RANDY Message-ID =~
/<[a-z\d][a-z\d\$-]{10,29}[a-z\d]\@[a-z\d][a-z\d.]{3,12}[a-z\d]>/
20_head_tests.cf:meta MSGID_RANDY (__MSGID_RANDY &&
!(__MSGID_OK_HEX || __MSGID_OK_DIGITS || __MSGID_OK_HOST))
20_head_tests.cf:describe MSGID_RANDY Message-Id has pattern
used in spam
and here are a couple example Message-ID headers from HAM that are
hitting this rule:
Message-Id:
Message-Id:
Perhaps these can be used to help tweak the rule not to fire on these
FPs, or perhaps we just need some HAM from chtah.net in the corpus to
moderate the scoring?
Both HAMs were of the type "your online statement/bill is ready", one
from Barclaycard and the other from Vodafone. I'm not able to share them
- sorry.
Anyway, just passing it along as this isn't a rule that's been on my
radar before the last week.
Re: How to report a spam botnet
On 20/11/12 20:26, Cathryn Mataga wrote: Easy enough to block #25 by default -- turn it on for anyone who asks. Indeed. I think the idea of a botnet black hole list is great, really. Spamhaus already do this. It's called the Exploits Block List (XBL): http://www.spamhaus.org/xbl/ To quote: The Spamhaus Exploits Block List (XBL) is a realtime database of IP addresses of hijacked PCs infected by illegal 3rd party exploits, including open proxies (HTTP, socks, AnalogX, wingate, etc), worms/viruses with built-in spam engines, and other types of trojan-horse exploits. /end quote Spamassassin already queries the Spamhaus lists, including the XBL - feel free to increase the scores to "black hole" any hits. Alternatively, many mail admins query Spamhaus lists during the smtp transaction and reject mail outright.
Re: Stopping abusive machiens (was Re: How to report a spam botnet)
On 20/11/12 15:17, David F. Skoll wrote: On Tue, 20 Nov 2012 15:10:57 + Ned Slider wrote: Personally I'd like to see some large corporates go after some infected home users in the courts for wilful damage. I think they'd lose. Most home users could make a compelling case that they were unaware of the infection and lacked the technical know-how to prevent it or clean it up. Ignorance is no defence, at least in the UK. I lack the technical skills to ensure my car is safe to drive on public highways but the law compels me to make it so regardless. Why should the Information Superhighway be treated any differently? The damage caused is just as costly to the victims, if not life threatening. You wouldn't be allowed to take a vehicle unfit for purpose on the public highway and cause damage to others without facing some form of recompense. So why do we allow PCs unfit for purpose on the public Internet and let the owners get away with the damage they cause? They need to be held responsible. Would you approve of a Ralph Nader-like approach of suing Microsoft for knowingly producing defective and insecure software? Detroit was shamed, bullied and sued into improving the safety of its cars; do you think that could work with Microsoft? I don't know who Ralph Nader is. I believe that case has already been fought and lost - I believe a judge has already ruled that it's unreasonable to expect something as complex as a modern OS (with many millions of lines of code) not to have flaws. Not saying I necessarily agree, but... And who would you go after to improve the safety of open-source systems like Linux? I would suspect the above ruling sets a precedent that would similarly apply to Linux, but I'm not a lawyer. I think ISPs imposing penalties for abusers is more realistic than any options through the courts. You are probably right. Major copyright holders have very publicly pursued members of the public guilty of downloading copyrighted material but yet that seems to have done little to change attitudes or stop the offense so I doubt my proposal would fair any better. Equally though, I am in absolutely no doubt that ISPs will never fine their own users unless they are legally required to do so - it would simply be bad business. ISPs simply do not care if their home users are infected - it makes absolutely no difference to them. They have the tools to detect and prevent wide scale abuse yet very few do because it adds a cost implication with little or no perceivable benefit to the customer. Probably a better proposition would be to fine the ISP for every infected PC on their network - that would get their attention. Then they are free to pass on that cost to the infected user should they wish. Or they could just choose to run a tighter ship in the first place to minimize fines/costs.
Re: How to report a spam botnet
On 20/11/12 14:30, David F. Skoll wrote: On Tue, 20 Nov 2012 14:26:49 + Martin Gregorie wrote: Nah, prevent all connections except HTML and SMTP/POP3 to the ISPs help desk and set of 'clean your act up' pages, so they can't ignore the mess their computer is in. And have escalating charges for reinstating Internet access after a machine has been compromised: First time: Free Second time: $25 Third time: $50 Subsequently: $100 Wait... are those flying pigs I see out my window? Regards, David. and therein lies the gulf between ideal and reality :-) Personally I'd like to see some large corporates go after some infected home users in the courts for wilful damage. You wouldn't be allowed to take a vehicle unfit for purpose on the public highway and cause damage to others without facing some form of recompense. So why do we allow PCs unfit for purpose on the public Internet and let the owners get away with the damage they cause? They need to be held responsible. JMHO
Re: How to report a spam botnet
On 19/11/12 06:18, Michael Monnerie wrote: [crosspost postfix-users and spamassassin-users] Am Sonntag, 18. November 2012, 14:08:08 schrieb Michael Monnerie: How should we report those IPs, is there a "anti botnet unit" somewhere? Lets concentrate back on the subject, I got this answer: normally it makes no sense to report botnets And this is what makes me worry. Botnets are todays biggest source of spam, and nobody has ever started to fight it really? There are tons of tools for every small issue, but nothing to cope with the biggest shit? As others have stated, there are many very effective ways to fight botnet spam, it's just that reporting individual IP addresses isn't one of them. To fight the spam at the recipient's end, things like Postscreen/Postgrey are hugely effective as are DNSBLs such as Spamhaus' PBL and XBL. At source, many investigators have had great success taking down botnets by targeting command and control infrastructure or by bringing legal measures against those in control of them. ISPs are also more commonly blocking outbound smtp traffic from domestic IP ranges by default forcing users to use the ISP provided smarthosts. To specifically address your query regarding reporting IP addresses - any ISP should be able to immediately see one of their hosts is spewing inordinate amounts of spam without you having to report the IP address - if they can't see this (and do something about it) then they will very quickly find their way onto DNSBLs at which point the problem generally takes care of itself. So generally there are better uses of one's time than reporting tens of thousands of infected IP addresses to their ISPs who should already have this information at their disposal.
Re: HK_LOTTO hitting ham from the UK national lottery
On 31/10/12 15:21, Axb wrote: On 10/31/2012 04:13 PM, Niamh Holding wrote: Hello Andy, Wednesday, October 31, 2012, 2:22:10 PM, you wrote: AJ> Your message scored a 7.1 on my system. Not a good score for ham :) AJ> 0.5 KAM_LOTTO1 Likely to be an e-Lotto Scam Email But it isn't... maybe 2 rules need amending so they don't hit genuine UK national lottery ham. or maybe you just lower the score for your setup. and rethink required=4.5 or: if you provide a few dozen samples of these hammy msgs , they can be included in the SA ham corpus or: you can directly contribute to rescoring by running a masscheck instance as per: http://wiki.apache.org/spamassassin/NightlyMassCheck I'll just add that I can also confirm HK_LOTTO and HK_LOTTO_NAME (currently scoring 3.599 and 0.998) regularly hit UK National Lottery emails as well as UK Health Lottery emails. I previously contacted Emailvision who is the ESP for the UK Health Lottery to advise them their emails were scoring very heavily in SA and recommend they maybe pre-screen outgoing emails with SA before sending (they were being flagged as spam at default scores). I've been working around the high scoring HK_LOTTO rules by whitelisting with SPF. Here is what I have for the UK Health Lottery: whitelist_from_spf *@healthlottery.co.uk whitelist_from_spf *@*.healthlottery.co.uk whitelist_from_spf *@health-lot.cccampaigns.com I haven't seen high/abusive levels of mail from these domains so consider whitelisting safe practice.
Re: BAYES_99 score
On 22/10/12 19:15, dar...@chaosreigns.com wrote: On 10/22, JP Kelly wrote: Should I set the BAYES_99 score high enough to trigger as spam? I get plenty of spam getting through which does not get caught because BAYES_99 is the only rule which fires and it is not set to score at or above the threshold. You could. Some people only use bayesian filtering, which would be similar. The important question is, how many false positives (non-spams flagged as spams) would that cause? SpamAssassin's automated scoring attempts to achieve 1 false positive in 2,500 non-spams, with a score threshold of 5.0. So if you don't have an absolute minimum of 2,500 representative non-spams to check for having hit BAYES_99, you risk increasing your false positives. But it's your risk to take. I have had very good success running adjusted scores for BAYES rules, but I am very careful how I train my bayes database. I've disabled auto-learning and only manually train on hand-checked ham and spam examples. Consequently, I find the extremes (BAYES_99 and BAYES_00) to be highly reliable indicators.
Re: I thought this message was rather spammy
On 17/10/12 18:51, Alexandre Boyer wrote: Right, but you have the content on the other link: http://igor.chudov.com/tmp/spam013.trace.txt It scores 5.7 and should be blocked. The message scored 2.3 when it was originally received. It only scored 5.7 when it was later reevaluated by SA at which point a URI is now hitting 2 URIBLs and thus increasing the score. These URIs have obviously just been added to the blacklists since this spam run started. Greylisting can often help here as even though it won't block the message it can delay the message long enough for offending IPs or URIs to get added to blacklists.
Re: no rDNS
On 16/10/12 16:25, Joseph Acquisto wrote: Wondering about this detection: "2.4 RDNS_NONE Delivered to internal network by a host with no rDNS" Yes, I see this on *all* SPAM marked email some of which are from legitimate senders. I have set my internal boxes as trusted and internal in the config file, but still suspect it must be my own doing. It is quite possible legitimate senders may not have set rDNS. Unusual that it would hit *all* spam. Perhaps you should post an example to pastebin of an email you feel it shouldn't have hit?
Re: Sender domain in IP space 5.0.0.0/8 triggers RCVD_ILLEGAL_IP
On 16/10/12 07:50, John Wilcock wrote: Le 16/10/2012 07:57, Frederic De Mees a écrit : When I receive mails from servers hosted in IP address space 5.0.0.0/8, SA tags them with RCVD_ILLEGAL_IP. This address space is currently heavily distributed in Europe. I have found a bug report #6810 (https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6810) that says that the bug has been fixed in June. However I still get a score of 3.4 when such mails drop in. How would you diagnose this ? The bug has definitely been fixed for SpamAssassin 3.3.2, and the bugzilla states that the same is true for all 3.3.x. The Wiki page / documentation still needs updating though: http://wiki.apache.org/spamassassin/Rules/RCVD_ILLEGAL_IP states: The valid, but unusable, address ranges are as follows: 0/8 Reserved 1/8 Unallocated 2/8 Unallocated 7/8 Live allocation - US DoD 127/8 Loopback range (excepting localhost 127.0.0/24) 223/8 Unallocated 224/4 Multicast but 1/8 and 2/8 are now in use so should be removed from the documentation.
Re: Can't locate Bignum.pm
On 09/10/12 14:52, Niamh Holding wrote: Hello maillog in showing- Oct 9 08:18:25 mail spamd[25346]: spamd: server killed by SIGTERM, shutting down Oct 9 08:18:25 mail spamd[28876]: logger: removing stderr method Oct 9 08:18:26 mail spamd[28878]: Can't locate Crypt/OpenSSL/Bignum.pm in @INC (@INC contains: /usr/lib/perl5/site_perl/5.8.6/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.6 /usr/lib/perl5/site_perl/5.8.5/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.4/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.3/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.5 /usr/lib/perl5/site_perl/5.8.4 /usr/lib/perl5/site_perl/5.8.3 /usr/lib/perl5/site_perl /usr/lib/perl5/vendor_perl/5.8.6/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.5/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.4/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.3/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.6 /usr/lib/perl5/vendor_perl/5.8.5 /usr/lib/perl5/vendor_perl/5.8.4 /usr/lib/perl5/vendor_perl/5.8.3 /usr/lib/perl5/vendor_perl /usr/lib/perl5/5.8.6/i386-linux-thread-multi /usr/lib/perl5/5.8.6) at /usr/lib/perl5/site_perl/5.8.6/i386-linux-thread-multi/Crypt/OpenS SL/RSA.pm line 17. Oct 9 08:18:31 mail spamd[28878]: spamd: server started on port 783/tcp (running version 3.3.2) Oct 9 08:18:31 mail spamd[28878]: spamd: server pid: 28878 And do you have the perl-Crypt-OpenSSL-Bignum package/module installed?
Re: SA without Mail::SPF::Query
On 06/09/12 17:45, Andreas Schulze wrote: Am 06.09.2012 17:08 schrieb Ned Slider: If your milter adds the Received-SPF header before the mail is passed to SA then maybe you could simply write a rule to check the Received-SPF header and score as you see appropriate. Yes, the Milter add a Received-SPF header. Could you point me to some sample Rules? Thanks Andreas Better still, why don't you post examples of the header your milter adds and what you'd like to do and we can help you write some rules. But to start you off, one could write a very simple rule to detect the 'Received-SPF: pass' header: header LOCAL_RCVD_SPF Received-SPF =~ /^pass/i describeLOCAL_RCVD_SPF Received-SPF contains pass score LOCAL_RCVD_SPF 0.1
Re: SA without Mail::SPF::Query
On 06/09/12 12:08, Andreas Schulze wrote: Hello, for technical reasons I have no Mail::SPF::Query. So my SA has no "view" to the spf settings of an incomming mail. But I run an SPF-Milter in front of SA without Mail::SPF::Query. That Filter adds an Received-SPF header to the mails but do not reject. Is it possible to use the result of the milter in the same way SA would do with its own SPF implementation? Than the SPF information could have an influance to the spamcore. Thanks Andreas If your milter adds the Received-SPF header before the mail is passed to SA then maybe you could simply write a rule to check the Received-SPF header and score as you see appropriate.
Re: Anyone from ReturnPath want to deal with this
On 05/09/12 15:39, Tom Bartel wrote: -Original Message- From: Ned Slider [mailto:n...@unixmail.co.uk] Sent: Wednesday, September 05, 2012 8:11 AM To: users@spamassassin.apache.org Subject: Re: Anyone from ReturnPath want to deal with this On 01/09/12 01:14, Ned Slider wrote: Hi list, Would anyone from ReturnPath care to take a look at the following: Received: from mail5.eventbrite.com (mail5.eventbrite.com [67.192.45.102]) which just spammed a contact@ address scraped off website and has -5pts awarded by ReturnPath: RCVD_IN_RP_CERTIFIED=-3 RCVD_IN_RP_SAFE=-2 sent "from" miracle_mur...@hotmail.com Compromised server/account maybe?? Happy to submit a fully unredacted sample off list. Not happy seeing spam sail through with -5pts from ReturnPath. Thanks. Bump... No one here from ReturnPath? I'm surprised ReturnPath is able to "certify" and declare "safe" IPs from a netblock that doesn't even list an abuse contact. How is one supposed to report and follow up on blatant abuse? Hi Ned - I apologize for my delayed response - I saw this yesterday - but had to get subscribed and then figure out how to reply to a message that I could only see in the archive. I've copied in Margot Romary who heads up our Compliance and Security - she and her team handle these issues. I've also copied in Kelly Molloy who works in our ISP services group as she is typically on the lookout for issues like this so we don't miss them. In terms of abuse contact, I'm seeing information in the Network Whois record for this IP and block, though maybe I'm looking in the wrong place: http://centralops.net/co/DomainDossier.aspx?addr=67.192.45.102&dom_whois=true&dom_dns=true&net_whois=true Return Path does operate both ab...@returnpath.net and postmas...@returnpath.net. Any issue with a Certification member will get to our team. Our Support site http://returnpath.net/support/ specifically lists certificat...@returnpath.net as the destination for any issues such as this. Certified members should not be sending to scraped addresses as indicated. Eventbrite allows folks to setup events and upload associated addresses. Based on the reputation data we have for them (complaints, trap hits) across our network (some of that data summarized here https://www.senderscore.org/lookup.php?lookup=67.192.45.102), this does not appear to be a widespread problem - however we will of course reach out to them to investigate and ensure they identify the offending user and handle properly. Best, Tom Many thanks Tom - I'll also reply off list with a copy of the spam for you and your colleagues to follow up on. Regards, Ned
Re: Anyone from ReturnPath want to deal with this
On 01/09/12 01:14, Ned Slider wrote: Hi list, Would anyone from ReturnPath care to take a look at the following: Received: from mail5.eventbrite.com (mail5.eventbrite.com [67.192.45.102]) which just spammed a contact@ address scraped off website and has -5pts awarded by ReturnPath: RCVD_IN_RP_CERTIFIED=-3 RCVD_IN_RP_SAFE=-2 sent "from" miracle_mur...@hotmail.com Compromised server/account maybe?? Happy to submit a fully unredacted sample off list. Not happy seeing spam sail through with -5pts from ReturnPath. Thanks. Bump... No one here from ReturnPath? I'm surprised ReturnPath is able to "certify" and declare "safe" IPs from a netblock that doesn't even list an abuse contact. How is one supposed to report and follow up on blatant abuse?
Anyone from ReturnPath want to deal with this
Hi list, Would anyone from ReturnPath care to take a look at the following: Received: from mail5.eventbrite.com (mail5.eventbrite.com [67.192.45.102]) which just spammed a contact@ address scraped off website and has -5pts awarded by ReturnPath: RCVD_IN_RP_CERTIFIED=-3 RCVD_IN_RP_SAFE=-2 sent "from" miracle_mur...@hotmail.com Compromised server/account maybe?? Happy to submit a fully unredacted sample off list. Not happy seeing spam sail through with -5pts from ReturnPath. Thanks.
Re: Somewhat OT: Is this wrong?
On 24/08/12 16:55, David F. Skoll wrote: On Fri, 24 Aug 2012 16:29:18 +0100 Ned Slider wrote: If Microsoft want to examine the From header then that is their concern. Googling shows others tend to agree with you that their implementation is broken, or in your words wrong. It is certainly wrong in SPF terms, but again it's not SPF :-) The problem is that I publish SPF records for my domain in the expectation that they'll be used correctly. By behaving incorrectly, Microsoft is making it less attractive for sites to publish SPF records lest they be misinterpreted. Regards, David. I hear you. I don't think it diminishes the effectiveness of SPF though. If your domain is regularly spoofed/phished etc then publishing an SPF record provides a really easy way for your recipients to verify your mail is legitimate and weed out the spoofs. But of course that is dependent on the receiving mail server being competently configured, something which is outside of your control.
Re: Somewhat OT: Is this wrong?
On 24/08/12 16:03, David F. Skoll wrote: On Fri, 24 Aug 2012 15:58:27 +0100 Ned Slider wrote: The Microsoft Sender ID system is not the same as SPF. The technet article I posted implied (and real-world tests seem to confirm) that MSFT Exchange 2003 really does SPF lookups against header-sender domains. Regards, David. David, It may well do. Sorry, I'm not familiar with what Microsoft does. I was merely pointing out that you shouldn't confuse a Microsoft proprietary standard that happens to have a similar name with SPF as they are not the same thing. If Microsoft want to examine the From header then that is their concern. Googling shows others tend to agree with you that their implementation is broken, or in your words wrong. It is certainly wrong in SPF terms, but again it's not SPF :-) Regards, --ned
Re: Somewhat OT: Is this wrong?
On 24/08/12 15:37, David F. Skoll wrote: Hi, Somewhat OT, but I figure there are SPF experts here: http://technet.microsoft.com/en-us/library/aa995992.aspx It appears to me that Microsoft uses header sender/from addresses to do an SPF lookup (see "How Sender ID Works") Am I the only one who thinks this is utterly wrong? To me, this is pretty clear: http://www.openspf.org/FAQ/Envelope_from_scope Regards, David. The Microsoft Sender ID system is not the same as SPF. See here: http://www.openspf.org/SPF_vs_Sender_ID Hope that helps.
Re: Rules Needed to verify bank fraud
On 23/08/12 18:18, Marc Perkel wrote: Let's take wellsfargo.com (Wells Fargo Bank) as an example. If the FCrDNS of the connecting server is *.wellsfargo.com it is ham. If wellsfargo.com is in the received lines and not forged it is ham. If wellsfargo.com is in the received headers and it is forged it is spam. If wellsfargo.com is in the received lines and there are IP in received with invalid FCrDNS then it is forged. If wellsfargo.com is not in the received headers then it is spam. Aren't you just massively over-complicating something that should be really simple. Wells Fargo publish an SPF record that will tell you if it's real or forged and SA already checks that. You really don't need to look any further. Wells Fargo are telling you with some authority, these are the IP addresses we authorise to send mail from our domain. Most all banks can be detected with 100% accuracy with these rules. For banks that let 3rd parties send email for them we can add specific exceptions including if the SFP lists it, or a list of known 3rd parties that pass the bank's email. Here's why this is important. It hits the fraud community hard. Takes the money out. Makes spam less profitable.
Re: Rules Needed to verify bank fraud
On 23/08/12 12:08, RW wrote: On Thu, 23 Aug 2012 01:33:56 +0100 Ned Slider wrote: # Fedex header __LOCAL_FROM_FEDEX Return-Path:addr =~ /\@fedex\.com$/i meta LOCAL_SPF_FEDEX ((SPF_SOFTFAIL || SPF_FAIL)&& __LOCAL_FROM_FEDEX) describeLOCAL_SPF_FEDEX Fedex SPF Fail and if I want to make sure all the legit fedex mail gets through I can further whitelist mail from fedex that passes SPF whitelist_from_spf *@fedex.com whitelist_from_spf *@*.fedex.com and that will virtually guarantee all the spam/viruses claiming to be from fedex are blocked and all the legitimate mail from fedex gets through. Provided that SPF hasn't been broken by email forwarding. If it's a plugin then just disable the plugin (or don't enable the plugin) on systems that accept mail from forwarders which break SPF.
Re: Rules Needed to verify bank fraud
On 23/08/12 04:31, Kevin A. McGrail wrote: On 8/22/2012 8:33 PM, Ned Slider wrote: So if I hit all mail claiming to be sent from fedex.com that fails SPF I can easily weed out all the fakes: # Fedex header __LOCAL_FROM_FEDEX Return-Path:addr =~ /\@fedex\.com$/i meta LOCAL_SPF_FEDEX ((SPF_SOFTFAIL || SPF_FAIL) && __LOCAL_FROM_FEDEX) describe LOCAL_SPF_FEDEX Fedex SPF Fail and if I want to make sure all the legit fedex mail gets through I can further whitelist mail from fedex that passes SPF whitelist_from_spf *@fedex.com whitelist_from_spf *@*.fedex.com and that will virtually guarantee all the spam/viruses claiming to be from fedex are blocked and all the legitimate mail from fedex gets through. It's no different with banks. However, this approach doesn't scale particularly well. It's OK for 10 or 20 domains, but for many more it would be far easier to manage using a plugin. So really want you want is a plugin that can take a list of domains that we've verified support SPF using -all and make that a stronger score? Regards, KAM I guess what we are looking for is a plugin that can take a list of commonly abused domains known to have valid SPF records or valid DKIM signatures, and to be able to apply a (stronger) score to those messages that fail the SPF and/or DKIM test. If it were a plugin then those using email forwarding that breaks SPF and/or DKIM could easily disable the plugin making the system easy to manage. IMHO, talking SPF, it doesn't really matter if it's -all or ~all (hard or soft fail), here SA isn't making a policy judgement but rather creating a set of rules that differentiate spam from ham, or in this case attempting to differentiate mail that really is sent from paypal et al versus that which isn't (I'm struggling for the correct terminology here as it really isn't spam vs ham as we really don't care if it's solicited mail or not, just that it's genuinely sent not fraudulently sent). In much the same way as SA currently scores SPF_NEUTRAL > SPF_SOFTFAIL > SPF_FAIL which is the logical reverse of what the policy suggests, but that is what the corpus scoring model says is more effective. So just as with the above, the plugin rule should look to be a good indicator of spam/abuse/fraud/phish or whatever you care to call it versus mail genuinely sent from these domains. As an example, paypal.com uses ~all, but I'm going to treat that as a FAIL in SA the same as I would treat -all. I guess it comes down to looking closely at a list of domains and deciding if there is evidence: (a) that they are commonly abused and (b) that they have an SPF and/or DKIM policy that we can leverage to differentiate legitimate mail from abuse at which point they would make a suitable candidate for inclusion on the plugins list of domains.
Re: Rules Needed to verify bank fraud
On 23/08/12 00:07, RW wrote: On Wed, 22 Aug 2012 17:40:23 +0100 Ned Slider wrote: On 22/08/12 16:22, Marc Perkel wrote: I'd like to make a suggestion as to how to block a lot of fraud. This would involve making a list of domains similar to the successful freemail list plugin. The idea is to block email that spoofs major institutions such as banks, credit cards, ebay, and other organizations that want to try to get your password. So - we start by compiling a list of banks that are often spoofed and look at the received lines. The idea being that good email from these institutions will come from host names that either match their domains, or match the SPF. If it matches it's good - if it doesn't - it's bad. I've already done it and I can confirm it works brilliantly. By default I score mail from all such domains with an arbitrary high score which will cause them to be tagged as spam (say 8 points) and then whitelist them with SPF and/or DKIM which negates the above score allowing legitimate mail to pass untagged. If commonly forged/spoofed/phished domains don't use SPF or DKIM then I have little sympathy for them and it makes it harder for us to help them. My bank outsources it's email to a email marketing company. I think this sort of thing is quite common. The received header has nothing to do with the the bank. It passes spf, but the domain is not one it uses for anything else. But mail from the bank should still be sent using one of their own domains (envelope from address) regardless of who is responsible for handing the delivery. In this case the bank should set up an SPF record on their domain that includes IP address/range of the email marketing company responsible for delivery. This is exactly what SPF is intended for. The fact mail bight be delivered by a 3rd party marketing company is completely irrelevant. If mail claiming to be from "my bank" passes SPF (or DKIM) then there is a good chance it was actually sent by them. If it fails SPF (or DKIM) then you should assume that it's not really from them (although unfortunately it seems even some banks have a hard time getting SPF right). Lots of spam passes SPF, so this sounds fairly limited to me. This is *not* what SPF is intended for. We are not proposing testing all mail with SPF/DKIM, only mail claiming to be from those domains that are regularly the targets of fraud/phish/spoofing. Probably only 20-50 domains max I'd guess. Take a recent example, and not even one that's banking related. My spamtraps are currently inundated with spam claiming to be from fedex with an attachment (virus) for me to open about a parcel delivery. A common scam for sure. Now without even knowing what a valid email from fedex looks like I can see fedex.com has an SPF record: $ dig txt fedex.com ;; ANSWER SECTION: fedex.com. 10658 IN TXT "v=spf1 redirect=_spf.infosec.fedex.com" So if I hit all mail claiming to be sent from fedex.com that fails SPF I can easily weed out all the fakes: # Fedex header __LOCAL_FROM_FEDEX Return-Path:addr =~ /\@fedex\.com$/i metaLOCAL_SPF_FEDEX ((SPF_SOFTFAIL || SPF_FAIL) && __LOCAL_FROM_FEDEX) describeLOCAL_SPF_FEDEX Fedex SPF Fail and if I want to make sure all the legit fedex mail gets through I can further whitelist mail from fedex that passes SPF whitelist_from_spf *@fedex.com whitelist_from_spf *@*.fedex.com and that will virtually guarantee all the spam/viruses claiming to be from fedex are blocked and all the legitimate mail from fedex gets through. It's no different with banks. However, this approach doesn't scale particularly well. It's OK for 10 or 20 domains, but for many more it would be far easier to manage using a plugin.
Re: Rules Needed to verify bank fraud
On 22/08/12 16:22, Marc Perkel wrote: I'd like to make a suggestion as to how to block a lot of fraud. This would involve making a list of domains similar to the successful freemail list plugin. The idea is to block email that spoofs major institutions such as banks, credit cards, ebay, and other organizations that want to try to get your password. So - we start by compiling a list of banks that are often spoofed and look at the received lines. The idea being that good email from these institutions will come from host names that either match their domains, or match the SPF. If it matches it's good - if it doesn't - it's bad. I've already done it and I can confirm it works brilliantly. By default I score mail from all such domains with an arbitrary high score which will cause them to be tagged as spam (say 8 points) and then whitelist them with SPF and/or DKIM which negates the above score allowing legitimate mail to pass untagged. If commonly forged/spoofed/phished domains don't use SPF or DKIM then I have little sympathy for them and it makes it harder for us to help them. For me the hardest thing was collating accurate information for which domains use SPF and or DKIM, especially for domains where you don't see any legitimate mail. I'm happy to share what I have. I have previously suggested on this list using a plugin similar to freemail for such usage but no one has run with it. I'd do it but unfortunately my perl skills are not up to the task so my implementation is simply based on a cobbled together set of SA rules. ...and it doesn't even need to be rocket science - you'd be amazed how well this rule works on my mail flow: header FROM_CONTAINS_BANK From =~ /\bbank(?:ing)?\b/i describeFROM_CONTAINS_BANK From contains bank
Re: SELL CVV GOOD ALL COUNTRY,Transfer WU,SHIP LAPTOP( DELL, TOSHIBA,..) IPAD2,IPHONE
On 09/06/12 20:32, jdow wrote:
I rather enjoyed it when the list passed spam. Every spam received to
the list was instant grist for the SARE mill leading to better and tighter
rules for killing similar spams.
After awhile the amount of spam from actual spammers dropped fairly
dramatically as they put spamassassin list filters in their spam tools.
Those were the days
{^_-}
I get quite a lot of spam sent to what appears to be scraped
message-...@mydomain.co.uk that I use for this list. Of course they were
being rejected as address unknown but I've since added some header rules
to my MTA to accept and redirect these to my spamtraps for bayes to feed
off :-D
Funnily enough the volume increases after each posting I make to this list.
Re: why don't banks do more against phishing?
On 24/04/12 15:23, Martin Gregorie wrote: My bank says up front and in writing that they will never ask for account or login details by e-mail. I suggest moving your account away from any bank that doesn't have the same policy and stick to it. Make sure you tell them why you're leaving, though. In addition to helping customers in this way, it would be really nice if they would similarly help mail admins to by also having a well defined email policy, clearly stating which addresses they will send email from and publishing accurate SPF records for those domains. That would make it trivial for all mail admins to detect and block bank phishing attempts. It's not rocket science!
Re: "Pill" spams
On 20/04/12 23:24, Ned Slider wrote: On 20/04/12 20:17, Ned Slider wrote: On 16/04/12 04:56, John Hardin wrote: On Tue, 10 Apr 2012, John Hardin wrote: On Tue, 10 Apr 2012, Thomas Johnson wrote: On Tue, Apr 10, 2012 at 7:08 AM, Bowie Bailey wrote: > That sounds like it might be good rule-fodder. "subject", "Subject", > and "SUBJECT" are possibly valid, but the other funky capitalizations > might be worth a few points. And how would one write a rule for that? Check the __TO_EQ_FM header rules for ideas. I'll take a shot at something this weekend if nobody does earlier. I supplied this directly to Thomas earlier in the week, and it's in my sandbox: header SUBJ_ODD_CASE ALL =~ /\n(?!(?:Subject:|SUBJECT:|subject:))(?i:subject:)/sm describe SUBJ_ODD_CASE Oddly mixed-case Subject: header Hi John, I have quite a few examples of these in my archives, and I confirm your rule catches them - nice job :-) Further, I note that to date the examples I have also appear to be omitting a space between the end of "Subject:" and the start of the actual subject text (examples on pastebin below). I wonder if this could also be leveraged into a rule? http://pastebin.com/xPf4XgJY For example, I'd expect to see: Subject: Re: Some text and not: Subject:Re:Some text Hmm, not as easy to write a rule as I first thought as it would appear that when using the header Subject, SA strips the leading white space that we are trying to test for. So I took inspiration from your rule above, and came up with: header LOCAL_SUBJ_MISSPACED ALL =~ /\n(?i:^Subject:(?!($|\s)))/sm describe LOCAL_SUBJ_MISSPACED Subject: missing whitespace which appears to work as expected. John - please could you explain the closing /sm as I'm unfamiliar with it's usage? Thanks. I found the explanation of the sm modifiers in the perlre manual. Tidying up my rule a little: header LOCAL_SUBJ_MISSPACEDALL =~ /\nSubject:(?!($|\s))/ism describeLOCAL_SUBJ_MISSPACEDSubject: missing whitespace
Re: "Pill" spams
On 20/04/12 20:17, Ned Slider wrote: On 16/04/12 04:56, John Hardin wrote: On Tue, 10 Apr 2012, John Hardin wrote: On Tue, 10 Apr 2012, Thomas Johnson wrote: On Tue, Apr 10, 2012 at 7:08 AM, Bowie Bailey wrote: > That sounds like it might be good rule-fodder. "subject", "Subject", > and "SUBJECT" are possibly valid, but the other funky capitalizations > might be worth a few points. And how would one write a rule for that? Check the __TO_EQ_FM header rules for ideas. I'll take a shot at something this weekend if nobody does earlier. I supplied this directly to Thomas earlier in the week, and it's in my sandbox: header SUBJ_ODD_CASE ALL =~ /\n(?!(?:Subject:|SUBJECT:|subject:))(?i:subject:)/sm describe SUBJ_ODD_CASE Oddly mixed-case Subject: header Hi John, I have quite a few examples of these in my archives, and I confirm your rule catches them - nice job :-) Further, I note that to date the examples I have also appear to be omitting a space between the end of "Subject:" and the start of the actual subject text (examples on pastebin below). I wonder if this could also be leveraged into a rule? http://pastebin.com/xPf4XgJY For example, I'd expect to see: Subject: Re: Some text and not: Subject:Re:Some text Hmm, not as easy to write a rule as I first thought as it would appear that when using the header Subject, SA strips the leading white space that we are trying to test for. So I took inspiration from your rule above, and came up with: header LOCAL_SUBJ_MISSPACEDALL =~ /\n(?i:^Subject:(?!($|\s)))/sm describeLOCAL_SUBJ_MISSPACEDSubject: missing whitespace which appears to work as expected. John - please could you explain the closing /sm as I'm unfamiliar with it's usage? Thanks.
Re: "Pill" spams
On 16/04/12 04:56, John Hardin wrote: On Tue, 10 Apr 2012, John Hardin wrote: On Tue, 10 Apr 2012, Thomas Johnson wrote: On Tue, Apr 10, 2012 at 7:08 AM, Bowie Bailey wrote: > That sounds like it might be good rule-fodder. "subject", "Subject", > and "SUBJECT" are possibly valid, but the other funky capitalizations > might be worth a few points. And how would one write a rule for that? Check the __TO_EQ_FM header rules for ideas. I'll take a shot at something this weekend if nobody does earlier. I supplied this directly to Thomas earlier in the week, and it's in my sandbox: header SUBJ_ODD_CASE ALL =~ /\n(?!(?:Subject:|SUBJECT:|subject:))(?i:subject:)/sm describe SUBJ_ODD_CASE Oddly mixed-case Subject: header Hi John, I have quite a few examples of these in my archives, and I confirm your rule catches them - nice job :-) Further, I note that to date the examples I have also appear to be omitting a space between the end of "Subject:" and the start of the actual subject text (examples on pastebin below). I wonder if this could also be leveraged into a rule? http://pastebin.com/xPf4XgJY For example, I'd expect to see: Subject: Re: Some text and not: Subject:Re:Some text I have ~100 further examples of these that do not have mixed case Subject (pastebin below), mostly pill spams that look like they are sent by the same broken bot mailer, just with "Subject" in more conventional case. http://pastebin.com/Zu0uvViQ Regards.
Re: Slip thu's
On 17/04/12 12:40, xTrade Assessory wrote: Ned Slider wrote: On 17/04/12 11:54, joea wrote: Getting "scanned document", "pills" and stuff with a url of "blah.blah.ru" Would emails with Russian URLs be legitimate in your organisation? Any .ru URL gets 6pts here by default - no complaints yet. Hi that certainly is not very nice. Electronic racism ... Not at all, it is based purely on statistical analysis of my mail flow. even if the spam quote may be high, there are legitimate users which should not pay for that I don't see any legitimate users here, I only see spam containing URIs in the body ending in the tld .ru - YMMV specially when you analyze better and see that the origin often is US or others, the only use .ru services to send it out ... I'm not suggesting blocking mail *from* .ru, I'm suggesting scoring mails in SA with URIs in the body ending in .ru *if* you don't expect to see such URIs in your regular mail flow. IMO when the domain and sender exist, not as known spammer or OR, you should not do that and better hang on to content analysis for evaluation/scoring That's exactly what I am doing - content evaluation/scoring based on statistical analysis of my own mail flow. A mail containing a URI ending in .ru is a very good indicator of spam on my server so I score it appropriately in SA. YMMV
Re: Slip thu's
On 17/04/12 11:54, joea wrote: Getting "scanned document", "pills" and stuff with a url of "blah.blah.ru" Would emails with Russian URLs be legitimate in your organisation? Any .ru URL gets 6pts here by default - no complaints yet. Some of these contain something like the snippet below, apparently put in by the sender or perhaps the mail provider. *** MIME-Version: 1.0 X-OriginalArrivalTime: Tue, 17 Apr 2012 05:12:23 -0300 X-SenderScore: 3 X-Envelope-From: lexlut...@evildoers.com X-SpamScore: 3 X-VirusScore: 0 X-SpamRefID: str=0001.0A010202.4F8D34EA.0094,ss=3,sh,fgs=0 X-ForwardedBy: SJL01WMAIL08B *** Would it be sane, relatively speaking, to add a rule that looks at the X-SpamScore: and/or X-SenderScore: and flag those? Not sure. Personally I don't see much value in it. In the vast majority of cases I would rather trust the results of my own scanning with SA than look at the X-Spam headers added by the outgoing mail server.
Re: New versions of Perl are slower
On 11/04/12 03:50, Julian Yap wrote: On Tue, Apr 10, 2012 at 4:28 PM, Michael Parker wrote: On Apr 10, 2012, at 4:12 PM, Julian Yap wrote: I'm running SpamAssassin 3.3.2 port revision 6 (latest from FreeBSD ports) on FreeBSD 8.2-RELEASE 64-bit. I recently upgraded my Perl from 5.10 to 5.14 but I needed to downgrade because SpamAssassin was crashing on a daily basis. See bug: https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6745 I have since downgraded my servers to Perl 5.10 and Perl 5.12. I have noticed that Perl 5.12 runs noticeably slower compared to 5.10. I think 5.14 was slow as well. Average scan times are higher and there are often more 'longer running' scans. This results in more output on servers running Perl 5.12: tail -f /var/log/maillog | grep 'identified spam .*[2-9][0-9].[0-9] seconds' Have others experienced the same thing? I think you can back that up and say that anything> 5.8 is slower. In perl 5.10 they made major changes to the regex engine which must have added some overhead that now slows things down. I've seen instances, depending on the total number of rules running, of 50% slowdowns moving from 5.8 -> 5.10 and beyond. Hmm, thanks for the info. It certainly explains things. Yeah, SpamAssassin previously used to blaze through mail scans (everything scanned in less than 3 seconds) on the same hardware. It's annoying that Perl is getting slower over time and there's no way to stop it because older versions of Perl get deprecated. - Julian I see you're not on Linux, but RHEL-5 uses Perl-5.8 and still has 5 years of product life yet (supported until 2017): $ rpm -q perl spamassassin perl-5.8.8-38.el5.x86_64 spamassassin-3.3.2-2.el5.rfx.x86_64
Re: Better phish detection
On 16/03/12 02:21, sporkman wrote:
Ned Slider wrote:
On 12/03/12 17:02, David B Funk wrote:
On Mon, 12 Mar 2012, Paul Russell wrote:
On 3/10/2012 16:43, Ned Slider wrote:
This one is easy enough - if the latter is the only valid url that
should ever appear in an email, create a meta rule that looks for a
url containing bway.net (or even just bway or webmail or login etc),
but isn't https://webmail.bway.net/.
Create meta rules for the common words you have identified. Link
these with a rule such as __HAS_ANY_URI or some of your webmail based
URI rules above.
What other rules commonly hit - are they sent from freemail accounts?
Do they hit any DNSBL's?
It's not that simple. If it were, the problem would not have been
ongoing for at least 4 years.
Technically what Ned said is correct "This one is easy enough".
Yes THIS ONE (emphasis mine) is easy enough, but for some of us these
kind of spear-phishing attacks are an ever mutating problem and some
are damned clever.
Exactly, if you only provide one snippet of an example you don't give us
much to work with so the best we can do is suggest a rule that will
catch that one very narrow example :-/
Give us a tarball of (preferably unredacted) examples to work with - you
must have hundreds collected over the last 4 years.
Here's a collection from our support folks:
http://home.bway.net/spork/phish/
Some patterns of interest (including some of the excellent suggestions
upthread):
-envelope-from is not from our domain, From: line in the message is, being
able to clobber that pattern would be quite helpful by itself.
-I'm not going to go into details, but the messages quite often have a from
or envelope-from that we simply don't use when sending email to customers or
replying to them. In fact, all of these samples have that wrong.
-Most the messages contain a URL that includes our domain in it after the
host part of the URL. The ones that don't still include "bway" somewhere in
the URL.
If I could slap together a rule that detects those three anomalies together,
I'd be pretty happy.
Good - you have something to work with now.
Here's a rule that will catch bway.net after the host part of the URL:
uri LOCAL_URI_BWAY_NET m{https?://[^/]+/.{0,40}bway\.net}
describeLOCAL_URI_BWAY_NET URL contains bway.net in the path
score LOCAL_URI_BWAY_NET 1
And many of your examples link to bway.htm so lets catch those too:
uri LOCAL_URI_BWAY_HTM m{bway\.html?$}
describeLOCAL_URI_BWAY_HTM Contains link to bway.htm
score LOCAL_URI_BWAY_HTM 1
Then I'd create some rules to make meta rules from. For example, not
from our domain + has URI + URI contains "bway".
Generally, the more rules you can create the better. Lots of small
scoring rules will invariably outperform a few high scoring rules unless
you are able to identify that one killer feature that identifies it as a
phish to you.
Re: Blocking frequent botnet pattern
On 15/03/12 00:39, Alex wrote: One clue: "X-Originating-IP: [41.189.207.189]" Check the various RBL hits on that address. ;) Are there existing plugins for this? Is there a way to check a range to see if it's part of a known blacklisted botnet? Or if you don't expect to receive email from Africa then just score this quite heavily. There are already non-scoring meta rules in place: That's helpful, thanks. Are there RBLs that maintain a list of botnet originating IPs, in the same way as IPs are checked by RCVD_IN_BRBL_LASTEXT and others? Am I right in thinking that the only time this originating IP is useful is just before the client host that is relaying the botnet mail is listed on an RBL, which may be the client host itself? Thanks, Alex Depending on how they are configured in SA, some RBLs are only used to check the last external IP address (i.e, the server that relays the message to your server) whereas others will check all IPs in the chain, including those in the Received headers as well as X-Originating-IP as you have above. This is called deep parsing. Have a look at the stock rules to see how it works. Some RBLs, such as the Spamhaus PBL, are *only* designed to be checked for the last external IP address and doing otherwise will lead to many FPs. However, some others are useful to check against all IPs. For example, Spamhaus XBL contains many IPs belonging to botnets. Although I already block all mail hitting Spamhaus XBL (as part of Spamhaus zen) that the MTA level, I still find it useful to query XBL and SBL against all IPs, and in this case we see 41.189.207.189 is listed in the XBL. What it's telling you is that although the server that relayed the spam to you is clean (yeah, no freemail servers generally get listed), this mail originated from an infected PC or passed through an otherwise less than clean server somewhere along it's route. To me that's certainly worth a few points given the reputation of Spamhaus. But be warned - play this game with the wrong DNSBL lists and it will bite you with FPs (e.g, Spamhaus PBL as stated above). You need to understand the type of IPs that get listed on any given list. The types of FPs that deep parsing against SBL-XBL causes are hitting against legitimate mail sent from Uncle Bob whose PC just happens to also be infected and spewing spam, which is not such a bad thing as now you can go help him clean up his PC. Also be aware that deep parsing IP addresses against DNSBLs will create a large increase in DNS queries against those lists as you are no longer querying just the one last external IP but are now querying all IPs in the chain, which could be quite a few.
Re: Blocking frequent botnet pattern
On 14/03/12 03:09, David B Funk wrote:
On Tue, 13 Mar 2012, Alex wrote:
Hi,
http://pastebin.com/raw.php?i=iquXBnH0
While I could create a rule to block this specific domain, or submit
it to a RBL, I'd appreciate any ideas how to more generally block
them, rather than by one characteristic in the message.
We need more examples.
That just occurred to me that it would help. Here are a few similar
ones, but these hit bayes99:
http://pastebin.com/raw.php?i=Axgx8qSP
http://pastebin.com/raw.php?i=7iU2MnP7
These two URI formats have been around for ages and we developed a rule
on this list a while back to catch these. Here's what I'm currently using:
# Generic template to match any domain DDFirstLastname:
# URIs matching http://some.domain.com/breakingnews/12FirstLastname/
uri LOCAL_URI_DDFIRSTLAST
m{https?://[^/]+/[^/]+/\d\d[A-Z][a-z]{1,20}[A-Z][a-z]{1,20}/$}
describeLOCAL_URI_DDFIRSTLAST Spammy URI template with
DDFirstnameLastname
Here is an example more closely relating to the first one. Hit only
bayes50, no subject, freemail, very similar short body:
http://pastebin.com/raw.php?i=juvD9yzS
Thanks,
Alex
Note that URL, yet another p0ned WordPress website (the
"/wp-content/plugins/" stuff). Now you get a hint of why I hate
"install-and-forget" websites.
When ever I run into p0ned websites their domain name goes into my
private URIBL list. They don't get spam past me again.
One clue: "X-Originating-IP: [41.189.207.189]"
Check the various RBL hits on that address. ;)
Re: Blocking frequent botnet pattern
On 14/03/12 03:09, David B Funk wrote:
On Tue, 13 Mar 2012, Alex wrote:
Hi,
http://pastebin.com/raw.php?i=iquXBnH0
While I could create a rule to block this specific domain, or submit
it to a RBL, I'd appreciate any ideas how to more generally block
them, rather than by one characteristic in the message.
We need more examples.
That just occurred to me that it would help. Here are a few similar
ones, but these hit bayes99:
http://pastebin.com/raw.php?i=Axgx8qSP
http://pastebin.com/raw.php?i=7iU2MnP7
Here is an example more closely relating to the first one. Hit only
bayes50, no subject, freemail, very similar short body:
http://pastebin.com/raw.php?i=juvD9yzS
Thanks,
Alex
Note that URL, yet another p0ned WordPress website (the
"/wp-content/plugins/" stuff). Now you get a hint of why I hate
"install-and-forget" websites.
When ever I run into p0ned websites their domain name goes into my
private URIBL list. They don't get spam past me again.
Here's a rule to catch that:
# URIs matching http://some.domain.com/wp-content/plugins/
uri LOCAL_URI_WPCONTENT m{https?://[^/]+/wp-content/plugins/}
describeLOCAL_URI_WPCONTENT Spammy URI with /wp-content/plugins/
I have similar rules for other hacked site templates and they work quite
well.
One clue: "X-Originating-IP: [41.189.207.189]"
Check the various RBL hits on that address. ;)
Or if you don't expect to receive email from Africa then just score this
quite heavily. There are already non-scoring meta rules in place:
$ grep FROM_41 *.cf
72_active.cf: meta __FROM_41_FREEMAIL
(__NSL_ORIG_FROM_41 || __NSL_RCVD_FROM_41) && (FREEMAIL_FROM ||
FREEMAIL_REPLYTO) && !__THREADED
72_active.cf: describe __FROM_41_FREEMAIL Sent from Africa
+ freemail provider
72_active.cf:header __NSL_ORIG_FROM_41X-Originating-IP
=~ /^(?:.+\[)?41\./
72_active.cf:describe__NSL_ORIG_FROM_41Originates from
41.0.0.0/8
72_active.cf:header __NSL_RCVD_FROM_41
X-Spam-Relays-External =~ / ip=41\./
72_active.cf:describe__NSL_RCVD_FROM_41Received from
41.0.0.0/8
Re: Blocking frequent botnet pattern
On 14/03/12 02:36, Alex wrote: Hi, http://pastebin.com/raw.php?i=iquXBnH0 While I could create a rule to block this specific domain, or submit it to a RBL, I'd appreciate any ideas how to more generally block them, rather than by one characteristic in the message. We need more examples. That just occurred to me that it would help. Here are a few similar ones, but these hit bayes99: http://pastebin.com/raw.php?i=Axgx8qSP http://pastebin.com/raw.php?i=7iU2MnP7 Here is an example more closely relating to the first one. Hit only bayes50, no subject, freemail, very similar short body: http://pastebin.com/raw.php?i=juvD9yzS Thanks, Alex I've had some success with various meta rules working with FREEMAIL_FROM, MISSING_SUBJECT, __HAS_ANY_URI, __MANY_RECIPS and __LOCAL_SUBJ_BLANK. __LOCAL_SUBJ_BLANK is my own simple meta: # Blank Subject lines header __LOCAL_SUBJ_CONTENTSubject =~ /./ header __LOCAL_SUBJ_RE Subject =~ /^re:?$/i meta__LOCAL_SUBJ_BLANK (__LOCAL_SUBJ_RE || !__LOCAL_SUBJ_CONTENT) I have had some FPs though, as it seems it's not that uncommon for freemail users to send emails to half their address book with a URI and forget to add a subject! Still, for my small volume server bayes and whitelisting tend to catch those.
Re: Better phish detection
On 12/03/12 17:02, David B Funk wrote: On Mon, 12 Mar 2012, Paul Russell wrote: On 3/10/2012 16:43, Ned Slider wrote: This one is easy enough - if the latter is the only valid url that should ever appear in an email, create a meta rule that looks for a url containing bway.net (or even just bway or webmail or login etc), but isn't https://webmail.bway.net/. Create meta rules for the common words you have identified. Link these with a rule such as __HAS_ANY_URI or some of your webmail based URI rules above. What other rules commonly hit - are they sent from freemail accounts? Do they hit any DNSBL's? It's not that simple. If it were, the problem would not have been ongoing for at least 4 years. Technically what Ned said is correct "This one is easy enough". Yes THIS ONE (emphasis mine) is easy enough, but for some of us these kind of spear-phishing attacks are an ever mutating problem and some are damned clever. Exactly, if you only provide one snippet of an example you don't give us much to work with so the best we can do is suggest a rule that will catch that one very narrow example :-/ Give us a tarball of (preferably unredacted) examples to work with - you must have hundreds collected over the last 4 years.
Re: Better phish detection
On 10/03/12 20:27, sporkman wrote: Generally it is easier to offer suggestions if examples are provided (on pastebin) Here's the latest example: http://broomesol.com/upgrade.webmail.bway.net/main_login.htm Compare to our actual webmail login: https://webmail.bway.net/ This one is easy enough - if the latter is the only valid url that should ever appear in an email, create a meta rule that looks for a url containing bway.net (or even just bway or webmail or login etc), but isn't https://webmail.bway.net/. Create meta rules for the common words you have identified. Link these with a rule such as __HAS_ANY_URI or some of your webmail based URI rules above. What other rules commonly hit - are they sent from freemail accounts? Do they hit any DNSBL's? Thanks, Charles
Re: sa-update / perl error again
On 05/01/12 01:28, email builder wrote: What is the Net::DNS version, are you pure ipv6 and are you 64-bit? perl-Net-DNS-0.63-1.el5.rf You are in no man's land there - the distro uses perl-Net-DNS-0.59-3.el5 and the latest rpmforge package is perl-Net-DNS-0.66-1.el5.rfx. If you're going to use rpmforge packages, keep them up to date (you'll need to enable the rpmforge-extras repo). Hrm, not sure how that could happen, since I don't have rpmforge disabled. But you do have rpmforge-extras disabled... How could yum not be seeing the newer package? and the newer version is in rpmforge-extras, as denoted by the .rfx tab. That is why yum was not seeing the updated version. cat /etc/yum.repos.d/rpmforge.repo ### Name: RPMforge RPM Repository for RHEL 5 - dag ### URL: http://rpmforge.net/ [rpmforge] name = RHEL $releasever - RPMforge.net - dag baseurl = http://apt.sw.be/redhat/el5/en/$basearch/rpmforge mirrorlist = http://apt.sw.be/redhat/el5/en/mirrors-rpmforge #mirrorlist = file:///etc/yum.repos.d/mirrors-rpmforge enabled = 1 protect = 0 gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-rpmforge-dag gpgcheck = 1 [rpmforge-extras] name = RHEL $releasever - RPMforge.net - extras baseurl = http://apt.sw.be/redhat/el5/en/$basearch/extras mirrorlist = http://apt.sw.be/redhat/el5/en/mirrors-rpmforge-extras #mirrorlist = file:///etc/yum.repos.d/mirrors-rpmforge-extras enabled = 0 protect = 0 gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-rpmforge-dag gpgcheck = 1 [rpmforge-testing] name = RHEL $releasever - RPMforge.net - testing baseurl = http://apt.sw.be/redhat/el5/en/$basearch/testing mirrorlist = http://apt.sw.be/redhat/el5/en/mirrors-rpmforge-testing #mirrorlist = file:///etc/yum.repos.d/mirrors-rpmforge-testing enabled = 0 protect = 0 gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-rpmforge-dag gpgcheck = 1
Re: sa-update / perl error again
On 04/01/12 21:41, email builder wrote: I have spamassassin-3.3.2-2.el5 installed from rpmforge on el5 - that package, besides being more up to date than the distro version also does not require perl-IO-Socket-INET6. I suspect your version does not really require perl-IO-Socket-INET6 either. You may be right. It was suggested that there may be a conflict due to mixing of packages between CentOS and other (rpmforge?) repos, so maybe it'd sure be nice if CentOS had more up to date software and there was no need to resort to using other repos (rpmforge doesn't have such up to date packages either in some cases I've noted). Do you actually need perl-IO-Socket-INET6? I assume not as you tried to remove it. I did not try to remove it, only saw others suggesting to remove it on the 'net. What does the following show: rpm -q --requires perl-IO-Socket-INET6 perl(:MODULE_COMPAT_5.8.8) perl(Carp) perl(Errno) perl(Exporter) perl(IO::Socket) perl(Socket) perl(Socket6) perl(strict) rpmlib(CompressedFileNames)<= 3.0.4-1 rpmlib(PayloadFilesHavePrefix)<= 4.0-1 rpmlib(VersionedDependencies)<= 3.0.3-1 I would update SpamAssassin from rpmforge and then remove perl-IO-Socket-INET6. Would I lose my current configuration in the process? it's a bit elaborate. IIRC, I'd have to temporarily disable CentOS repo to get the rpmforge one? Your opinion is that there's no fixing it without dumping my current SA package? You have two choices: 1. Update SA to the rpmforge package that doesn't require perl-IO-Socket-INET6 and then uninstall it. 2. Force uninstall perl-IO-Socket-INET6 with your current SA package: rpm -e --force perl-IO-Socket-INET6 and deal with yum/rpm complaining about missing deps in future.
Re: sa-update / perl error again
On 04/01/12 21:33, email builder wrote: What is the Net::DNS version, are you pure ipv6 and are you 64-bit? perl-Net-DNS-0.63-1.el5.rf You are in no man's land there - the distro uses perl-Net-DNS-0.59-3.el5 and the latest rpmforge package is perl-Net-DNS-0.66-1.el5.rfx. If you're going to use rpmforge packages, keep them up to date (you'll need to enable the rpmforge-extras repo).
Re: sa-update / perl error again
On 04/01/12 15:30, Ned Slider wrote: What does the following show: rpm -q --requires perl-IO-Socket-INET6 Oops, I meant: rpm -q --whatrequires perl-IO-Socket-INET6
Re: sa-update / perl error again
On 04/01/12 07:35, email builder wrote: Anyone have any other insights? Thanks! I have spamassassin-3.3.2-2.el5 installed from rpmforge on el5 - that package, besides being more up to date than the distro version also does not require perl-IO-Socket-INET6. I suspect your version does not really require perl-IO-Socket-INET6 either. Do you actually need perl-IO-Socket-INET6? I assume not as you tried to remove it. What does the following show: rpm -q --requires perl-IO-Socket-INET6 I would update SpamAssassin from rpmforge and then remove perl-IO-Socket-INET6. For reference, here are the perl packages I have installed on my system running SA/Amavisd-new/Clam (.el5 = distro, .rf = rpmforge, .rfx = rpmforge-extras) # rpm -q spamassassin amavisd-new clamav spamassassin-3.3.2-2.el5.rfx.x86_64 amavisd-new-2.6.6-1.el5.rf.x86_64 clamav-0.97.3-1.el5.rf.x86_64 # rpm -qa perl* | sort perl-5.8.8-32.el5_7.6.x86_64 perl-Archive-Tar-1.39.1-1.el5_5.2.noarch perl-Archive-Zip-1.16-1.2.1.noarch perl-Authen-SASL-2.15-1.el5.rf.noarch perl-BerkeleyDB-0.43-1.el5.rf.x86_64 perl-Bit-Vector-6.4-2.2.2.1.x86_64 perl-Carp-Clan-5.3-1.2.1.noarch perl-Compress-Raw-Bzip2-2.037-1.el5.rf.x86_64 perl-Compress-Raw-Zlib-2.037-1.el5.rf.x86_64 perl-Convert-ASN1-0.20-1.1.noarch perl-Convert-BinHex-1.119-2.2.el5.rf.noarch perl-Convert-TNEF-0.17-3.2.el5.rf.noarch perl-Convert-UUlib-1.34-1.el5.rf.x86_64 perl-Crypt-OpenSSL-Random-0.04-1.el5.rf.x86_64 perl-Crypt-OpenSSL-RSA-0.26-1.el5.rf.x86_64 perl-Date-Calc-5.4-1.2.2.1.x86_64 perl-DBI-1.52-2.el5.x86_64 perl-Digest-HMAC-1.01-15.noarch perl-Digest-SHA1-2.11-1.2.1.x86_64 perl-Digest-SHA-5.50-1.el5.rf.x86_64 perl-Email-Date-Format-1.002-1.el5.rf.noarch perl-Encode-Detect-1.01-1.el5.rf.x86_64 perl-Error-0.17016-1.el5.rf.noarch perl-Geography-Countries-2009041301-1.el5.rf.noarch perl-Git-1.7.6.4-1.el5.rf.x86_64 perl-HTML-Parser-3.59-1.el5.x86_64 perl-HTML-Tagset-3.10-2.1.1.noarch perl-IO-Compress-2.037-1.el5.rfx.noarch perl-IO-Multiplex-1.10-1.el5.rf.noarch perl-IO-Socket-SSL-1.01-1.fc6.noarch perl-IO-stringy-2.110-1.2.el5.rf.noarch perl-IO-Zlib-1.04-4.2.1.noarch perl-IP-Country-2.27-1.el5.rf.noarch perl-libwww-perl-5.805-1.1.1.noarch perl-Mail-DKIM-0.39-1.el5.rf.noarch perl-Mail-SPF-2.006-1.el5.rf.noarch perl-MailTools-2.08-1.el5.rf.noarch perl-MIME-Lite-3.027-1.el5.rf.noarch perl-MIME-tools-5.420-2.el5.rf.noarch perl-NetAddr-IP-4.044-1.el5.rf.x86_64 perl-Net-DNS-0.59-3.el5.x86_64 perl-Net-Ident-1.23-1.el5.rf.noarch perl-Net-IP-1.25-2.fc6.noarch perl-Net-Server-0.99-1.el5.rf.noarch perl-Net-SMTP-SSL-1.01-1.el5.rf.noarch perl-Net-SSLeay-1.30-4.fc6.x86_64 perl-Parse-Syslog-1.10-1.el5.rf.noarch perl-Pod-Escapes-1.04-1.2.el5.rf.noarch perl-Pod-Simple-3.16-1.el5.rf.noarch perl-Razor-Agent-2.84-1.el5.rf.x86_64 perl-SGMLSpm-1.03ii-16.2.1.noarch perl-Socket6-0.19-3.fc6.x86_64 perl-String-CRC32-1.4-2.fc6.x86_64 perl-Test-Pod-1.45-1.el5.rf.noarch perl-TimeDate-1.16-5.el5.noarch perl-Unix-Syslog-1.1-1.el5.rf.x86_64 perl-URI-1.35-3.noarch perl-version-0.91-1.el5.rf.x86_64
Re: Help tagging URL spam
On 02/01/12 06:03, Alex wrote: Hi, http://pastebin.com/raw.php?i=1Y5QCkfh http://pastebin.com/raw.php?i=KdmZXM0d give dkim invalid positive score if it was not pass on recieved ? add sbcglobal.net to freemail_domains add sanjit.in to local.cf url rule Thanks for your help. I should have been more clear about what I've already done to prevent these. I've already created some simple rules, including a local URIBL, that blocks these once they have been identified. They used to contain a unique pattern in the HTML component of the email, but they're only including a text content type now. What I haven't been able to figure out is a more generalized pattern from these, such as something in the header that is inconsistent with non-spam or contains some type of invalid header data, such as the mismatch between having originated at yahoo but being sent as sbcglobal? Shouldn't have bayes picked this up after learning a dozen or more of these? IMHO, yes. Are you sure you are training bayes correctly. Are you using the same user to train bayes as the user that is running SA? Work through some of the advice already given regarding bayes. From running your examples locally, I note: sanjit.in is now listed in a couple URIBLs (URIBL_PH_SURBL & URIBL_HOSTKARMA_BL) - don't know if it was listed at the time you received them. They hit some local meta rules I have combining FREEMAIL_FROM with __HAS_ANY_URI, __MANY_RECIPS, and various missing/blank subject rules. For me these are relatively good indicators of FREEMAIL spam. Also, as I see very little legit mail from AOL I also nail from @aol.com with 6 pts and then whitelist any legit senders with an AOL email address. YMMV.
URIBL_DBL_REDIR
Hi List,
I noticed the recent addition of URIBL_DBL_REDIR hitting on a few spams:
25_uribl.cf:urirhssub URIBL_DBL_REDIR dbl.spamhaus.org. A
127.0.1.3
25_uribl.cf:bodyURIBL_DBL_REDIR
eval:check_uridnsbl('URIBL_DBL_REDIRECTOR')
25_uribl.cf:describeURIBL_DBL_REDIR Contains a URL listed in
the DBL as a spammed redirector domain
Nice.
I just wondered if someone would like to look at a meta rule with
FREEMAIL_FROM as that's hitting a lot of spam here, although I don't see
too much (non-ML) legit freemail so I don't know how much ham it might
hit for others (hence the request).
Re: DNSWL will be disabled by default as of tomorrow
On 12/12/11 19:50, Ted Mittelstaedt wrote: I concur 100%. Daniel is wrong. The problem isn't dnswl.org the problem is the person who made the decision in SpamAssassin to have the default for the dnswl plugin ENABLED by default. That decision has been recognized to have been a mistake which is why SA is making an update that will turn it off by default. This is not a "blame the user for stupid configuration mistakes" problem this is a "blame the software developer for a stupid configuration mistake" And the software developer has acknowledged it was a mistake. So why people are calling SA users "abusive" is beyond me. Anyone who produces software understands that the users of the software are not as knowledgeable about whatever it is the software does - in a word, they are ignorant. That is why there are software developers and software users. If the users knew as much as the developer did they could write their own software and they wouldn't need the developers. Thus, the developer has an obligation to be somewhat responsible in not putting in defaults that shoot people in the foot. If those users want to enable things that shoot themselves in the foot that is their affair. The SA developers understand this, I don't see why it's so difficult a concept for others to grasp. Ted So does this mean SA should disable ALL network based tests by default as they all have the same potential to return false positives/negatives to get the attention of (abusive) sysadmins? About the only difference is dnswl.org got to hit folks with a -5 score whereas most others would have significantly less scoring impact available, but the potential threat is the same. I can understand the decision if dnswl.org have requested SA disable lookups by default, but otherwise it's a last resort attempt to get the attention of someone after all other reasonable efforts to communicate the issue have failed. I personally don't think it unreasonable. Either way, I appreciate the heads up here so we (SA users) may make the decision whether or not to re-enable dnswl.org on our own setups.
Re: URIBL_PH_SURBL
On 01/12/11 08:29, Tom Kinghorn wrote:
Good morning list.
could someone possibly explain how the scoring for ph.surbl.org works?
I see the following in my spam logs
spam-1DSMgl4+-YFV.gz: TO_NO_BRKTS_HTML_ONLY=1.258, URIBL_PH_SURBL=0.001]
spam-1DSMgl4+-YFV.gz: * 0.0 URIBL_PH_SURBL Contains an URL listed in the PH
SURBL blocklist
Why does the ph.surbl.org score so low?
I see the rule is defined as
urirhssub URIBL_PH_SURBL multi.surbl.org. A 8
body URIBL_PH_SURBL eval:check_uridnsbl('URIBL_PH_SURBL')
describe URIBL_PH_SURBL Contains an URL listed in the PH SURBL blocklist
tflags URIBL_PH_SURBL net
reuse URIBL_PH_SURBL
how does this work?
Thanks
Tom
and the score is defined in 50_scores.cf:
score URIBL_PH_SURBL 0 0.001 0 0.610 # n=0 n=2
These 4 scores are defined as local, net, with bayes, with bayes+net.
Net means you have network tests enabled, local means you don't have
network tests enabled.
So because you are showing a score of 0.001, you appear to be using the
"net" score set - network tests enabled but no bayes. If you were using
"net" and bayes, then this rule would have scored 0.610.
You can over ride scores locally in local.cf if you want.
The scores are automatically generated based on nightly masschecks
http://wiki.apache.org/spamassassin/NightlyMassCheck
This is obviously dependent upon people contributing data for their spam
and ham.
Re: Trends in spam
On 26/11/11 21:36, Karsten Bräckelmann wrote:
On Sat, 2011-11-26 at 19:46 +, Ned Slider wrote:
# URIs matching http://some.domain.com/profile/12FirstLastname/
uri LOCAL_URI_PROFILE
m{https?://.{1,40}/profile/\d\d[A-Z][a-z]{1,20}[A-Z][a-z]{1,20}/}
^^^
Using [^/]+ for the domain part instead should be slightly more
efficient, since it will never result in backtracking. Also stricter,
since it anchors the /profile directory at the domain.
Now you made me go away and read the perl RE manual!
Nice - [^/] meaning to match any character that is not a "/", one or
more (+) times. I'd not come across negated character classes before so
thanks for the tip!
And anchoring the /profile directory at the domain definitely reduces
the likelihood of FPs.
Trends in spam
Hi list,
One of the main strengths of SpamAssassin is the ability to allow the
user to write their own custom rules. However, writing good rules is not
always easy and one of the hardest parts is identifying trends in spam
worth targeting.
So what trends have you noticed this week?
Here's one I spotted whilst reviewing the weeks spam. I've seen a bunch
of URIs with the following pattern:
http://some.example.domain.com/profile/12BobSmith/
There's even an example sent the LKML here:
https://lkml.org/lkml/2011/11/26/55
and have written a rule to match the /profile/12FirstnameLastname/ part.
So in the spirit of sharing, enjoy my rule:
# URIs matching http://some.domain.com/profile/12FirstLastname/
uri LOCAL_URI_PROFILE
m{https?://.{1,40}/profile/\d\d[A-Z][a-z]{1,20}[A-Z][a-z]{1,20}/}
describeLOCAL_URI_PROFILE Spammy URI with profile
score LOCAL_URI_PROFILE 1
Score as you see fit.
So far these URIs have been redirectors to "jobs" sites. Clearly there
is some malware that is compromising web hosts using this same template
so this type of rule nullifies them all in one go - no need to collect
the domains and feed them into the black lists (although this is still a
good idea).
So have you spotted any decent trends worth sharing this week?
Re: Hacked webmail accounts (BTInternet/Yahoo)
On 26/11/11 01:21, Karsten Bräckelmann wrote:
On Fri, 2011-11-25 at 20:27 +, Ned Slider wrote:
header __L_BT_YAHOO_WEBMAIL01 Received =~ /from
\[86\.1[2-9][0-9]\.\d{1,3}\.\d{1,3}] by
web\d{4,6}\.mail\.\w{3}\.yahoo\.com via HTTP/i
but it would be far easier if I could somehow do a rDNS lookup on the
IP, see if it matches btcentralplus.com and score those that don't.
No, it would not be easier. It would require writing a plugin, rather
than the IP-range catching Received header rule. ;)
Alas that's where I was hoping you would say there was a plugin that
would do this type of thing that I wasn't aware of. Or maybe some geoip
type plugin to establish if the County of origin was the UK or not
(assuming that most hacked attempts probably originate outside of the UK).
Since you mentioned rDNS, you probably had the RDNS_NONE and friends
rules in mind. SA does not do these rDNS lookups, but depends on the MTA
to do them and note it in the Received headers.
There *might* be one alternative. The ASN plugin. I once had a similar
problem with a really spammy ASN [1], continuing to send out specific
German junk that for some reason managed to fly low and definitely under
the radar of Bayes. Alas, the ASN metadata was not available for rules.
IIRC there are some changes in trunk, that might fix this, and actually
make the ASN metadata also available for rules (and Bayes).
Without this option, I ended up writing a few X-Spam-Relays-External
rules with RE-encoded IP-ranges.
Looks like I'll be doing similar here. I just need to collect the BT ranges.
Thanks for the ideas though - much appreciated.
Hacked webmail accounts (BTInternet/Yahoo)
Hi,
I'm looking at try to write some rules to detect these. Specifically,
I'd like to target btinternet.com accounts (one of the largest UK
telecom companies) who have recently outsourced their email to Yahoo.
An example (spam) crossed my path today that only hit bayes_99. Looking
at the mail it is hard to see anything much to go on (sorry, I don't
really want to post the example as it contains quite a few email
addresses in the To: field). The body contains a URL which wasn't
detected at the time but which I've since submitted to URIBL and is now
detected.
My only thought is to try to score the mail based on the IP address
submitting it to the Yahoo webmail service. For example, the first
received header in my spam example is:
Received: from [88.178.198.52] by web87104.mail.ird.yahoo.com via HTTP;
Fri, 25 Nov 2011 16:37:29 GMT
whereas some legitimate mails look like this:
Received: from [86.155.137.177] by web86507.mail.ird.yahoo.com via HTTP;
Fri, 25 Nov 2011 18:13:12 GMT
Received: from [86.128.88.173] by web87309.mail.ird.yahoo.com via HTTP;
Sun, 06 Nov 2011 10:31:58 GMT
So my spam example is clearly originating from France:
$ host 88.178.198.52
52.198.178.88.in-addr.arpa domain name pointer
bop62-1-88-178-198-52.fbx.proxad.net.
whereas my legitimate examples (mostly) originate on BT's own ranges:
$ host 86.128.88.173
173.88.128.86.in-addr.arpa domain name pointer
host86-128-88-173.range86-128.btcentralplus.com.
I *know* it's not fool-proof (people use webmail when travelling), but
I'm thinking it's possibly worth a point or two (or maybe useful in a
further meta rule) to score webmail submissions From: btinternet.com
which don't originate from a BT IP range.
So any suggestions on the best way to construct the rule? I could try to
catch each BT IP address range like so:
header __LOCAL_FROM_BT_COM From:addr =~ /\@btinternet\.com$/i
header __LOCAL_MAILER_YAHOOX-Mailer =~ /YahooMailWebService/
header __L_BT_YAHOO_WEBMAIL01 Received =~ /from
\[86\.1[2-9][0-9]\.\d{1,3}\.\d{1,3}] by
web\d{4,6}\.mail\.\w{3}\.yahoo\.com via HTTP/i
meta LOCAL_BT_YAHOO_WEBMAIL (__LOCAL_FROM_BT_COM &&
__LOCAL_MAILER_YAHOO && !__L_BT_YAHOO_WEBMAIL01)
describe LOCAL_BT_YAHOO_WEBMAIL Submitted to BT/Yahoo Webmail from
foreign IP
score LOCAL_BT_YAHOO_WEBMAIL 0.001
but it would be far easier if I could somehow do a rDNS lookup on the
IP, see if it matches btcentralplus.com and score those that don't.
Any thoughts or ideas?
Re: Custom rawbody rule in local.cf not triggered
In addition to other replies... On 23/11/11 14:13, Simon Loewenthal wrote: I have spam that hits on these rules. X-Spam-Report: * 1.7 URIBL_BLACK Contains an URL listed in the URIBL blacklist * [URIs: europjobs.eu] * 1.2 URIBL_JP_SURBL Contains an URL listed in the JP SURBL blocklist * [URIs: europjobs.eu] * 0.0 UNPARSEABLE_RELAY Informational: message has unparseable relay lines * 0.8 BAYES_50 BODY: Bayes spam probability is 40 to 60% * [score: 0.5000] * 1.1 DCC_CHECK Listed in DCC (http://rhyolite.com/anti-spam/dcc/) * 1.4 PYZOR_CHECK Listed in Pyzor (http://pyzor.sf.net/) * 0.3 DIGEST_MULTIPLE Message hits more than one network digest check * 0.8 RDNS_NONE Delivered to internal network by a host with no rDNS What I fail to understand is why it did not hit on this local.cf rule: describe RBODY_JOB_DOMAINS1 English language job opportunity1 rawbody RBODY_JOB_DOMAINS1 /\@(?:axeabout|career-lists|careers-consult|eur-exlusive|europe-career|europ-exlusive|it-jobsearch\.com|uk-exlusive|tech-newposition|new-joboffers|joblists|web-newcarer|world-jobsearch|gb-totaljob|simple-jobneed|sprytex-it|europjobs.eu|businesinsiders.com)\./ scoreRBODY_JOB_DOMAINS1 4.5 You can do that with a URI rule which will also hit on email addresses in the body of the email - it doesn't *need* to be a rawbody rule. A very simple truncated example based on your rule above: URI L_JOB_DOMAINS /\b(?:axeabout|career-lists|careers-consult|eur-exlusive)\b/i describe L_JOB_DOMAINS Unwanted Job Domains score L_JOB_DOMAINS 4.5 Feel free to make it stricter by adding TLDs at the end. Personally I would just increase the scores of URIBL_BLACK, URIBL_JP_SURBL et al., as I've found them to be pretty safe lists and excellent spam indicators. Then your custom rule only needs to cover those domains that are not yet listed.
Re: Exit spamassassin when reaching a value
On 05/11/11 20:52, John Hardin wrote: On Sat, 5 Nov 2011, Cherubini Enrico wrote: Good day, I would like to remove rbl check from postfix and using them in spamassassin, this because for some users rbl are too aggressive while for others they aren't enough. I would like to know if it is possible to set rbl check as the first check in spamassassin and, if the remote IP is blacklisted then stop other test and exit, so keeping server load as low as possible. Search the mailing list archives and SA documentation for "shortcircuit". You should probably also look at Postscreen, available in the latest version of Postfix: http://www.postfix.org/POSTSCREEN_README.html which I believe also allows RBLs to be scored if you don't trust to reject on a single RBL.
Re: Whitelisting with DKIM
On 31/10/11 19:54, Alex wrote: I'd rather not whitelist all of auth.ccsend.com, but only as it relates to bertolini-sales.com, just as I wouldn't want to whitelist all of constantcontact.com, or am I misunderstanding? Thanks again, Alex I'm not sure why you feel the need to whitelist these at all given your posted example already scored -4.555 points. It's not really whitelisting, it's just adding a *further* negative score for a matched DKIM signature. I tend to use DKIM (and/or SPF) whitelisting for ham that is hitting other rules causing a FP or near FP situation, or in situations where I want to penalise (score) mail pretending to be from a commonly abused domain whilst whitelisting genuine mail from that domain (e.g, bank phish etc).
Re: Disable a Rule
On 30/10/11 20:45, Jeremy McSpadden wrote: Thanks for the help Benny. .. Anyone besides this guy have anything to say ? -- See here: http://www.dnswl.org/news/archives/24-Abusive-use-of-dnswl.org-infrastructure-enforcing-limits.html and also the thread on this list from the archives dated 17th Oct 2011 with subject: DNSWL.org enforcement of free usage limits. Benny is correct - using your providers DNS servers results in exceeding the limit at DNSWL which results in all queries hitting RCVD_IN_DNSWL_HI - that's generally how they get your attention. Now they have your attention, the solution if you want to continue using DNSWL is to deploy your own local DNS caching server assuming you can stay under the free usage terms, or buy a data feed, or disable the DNSWL rules in SA by scoring them at zero: score RCVD_IN_DNSWL_HI 0 score RCVD_IN_DNSWL_MED 0 score RCVD_IN_DNSWL_LOW 0 score RCVD_IN_DNSWL_NONE 0 all of which has previously been stated. Hope that helps.
Re: real world spamassassin experiences re: processing on servers emailing from .info domains
On 27/10/11 18:36, Jenny Lee wrote: From: list...@abbacomm.net To: users@spamassassin.apache.org Subject: real world spamassassin experiences re: processing on servers emailing from .info domains Date: Thu, 27 Oct 2011 09:15:13 -0700 greetings SA users there sure seems to be a lot of from .info server spamming wierd temp registered .info domains spamming eh? for those of you with volume, large or small, care to share an SA tips on how you deal with .info domains? i would imagine there is a very small percentage of valid emails coming from .info domains should we just pull the plug and reject all .info from touching the smtp server or carefully craft SA rules? real close to doing so and just reject them all, unless there is a list of valids out there somewhere thank you in advance - rh In 14 years, we never received any single legit mail from .info. It costs $1 per year to register an info domain, and if the people I do business cannot afford $10 a year for their domain, they probably will not give me business to start with. We reject all .info on sendmail during transaction stage. Half of my rejected connections are .info (rest are same-sender/same-recipient). You have to assess your own situation. Jenny I haven't had to go as far as rejecting all .info domains yet, but I did spot a trend a while back where snowshoe spammers where using i...@example.info so I block those at the MTA with a simple PCRE: /^info@[a-z0-9]+\.info$/REJECT Looks like snowshoe They seem to have moved on now though as I currently see very little .info spam make it as far as SA. Screwfix in the UK (a large online hardware [screws and nails type] supplier) currently send out their mailings from em...@screwfix.info even though their main site is at screwfix.com, so there are some legit senders. YMMV
Re: all spam emails from mailengine1.com servers
On 22/10/11 03:03, Chip M. wrote: R - elists wrote: does anyone get legit emails that come from the mailengine1.com email marketing servers? Yes, I've seen a trickle of ham, so did some data mining for you... The IP ranges I have for them are: 66.59.0.0 - 66.59.31.255 72.19.192.0 - 72.19.255.255 Does anyone have any others? I found one other hostname: mailengine3.com and would infer there would be more, for you RE gurus. :) and I see a few instances of mailengine2 from the same IP address ranges. The vast majority of what I've seen is spam, however there are enough "human shields" that you may want to score cautiously, depending on your environment. From here they look like an ESP that seems to have some customers operating with non-COI lists. There will certainly be some legit mail flowing from their ranges, even from those customers with less than ideal operating policies. I guess the key is how they respond to abuse reports and if they are seriously interested in terminating customers with bad habits.
Re: Why doesn't anything at all get these botnet spammers?
On 17/10/11 19:07, Jenny Lee wrote: Every 2nd of my emails to this list from hotmail is returning as a nondeliverable. Hotmail does not give any info as to what failed but I am assuming it is the SPAM filters of the mailing list. Well done! Then stop posting spam to the list. You can see what rules you're hitting in the headers of your posts: X-ASF-Spam-Status: No, hits=9.8 required=10.0 tests=FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS,URIBL_BLACK,URIBL_SBL If you want to post examples, please make them available on a pastebin and provide the link - that is the etiquette here on this list.
Re: Recieving email from aol or yahoo or hotmail, that is not addressed to me personally.
On 16/10/11 19:28, Martin Gregorie wrote: On Sun, 2011-10-16 at 20:02 +0200, Benny Pedersen wrote: I may never know they sent me an email. Unless I spend time going over my logs. ah you have logs ? :=) A possible way out is to process the logs overnight, possibly as part of logwatch, and build a datastore of addresses that mail has been sent to via your MTA. You can use this as the basis for an automatic whitelisting SA plugin which whitelists people who have received mail from your domain. Messages from everybody else have to take their chances with the standard SA rules and any locally defined ones. This works well here: the only difference is that I archive all incoming and outgoing mail and use my archive as the sender-whitelisting plugin's data source. That's pretty much what I've been doing too - hotmail, yahoo and aol score 6pts by default and legit/known senders get a whitelist scoring to counterbalance that. Works brilliantly. Catches all new spam runs from the big freemail providers and, worst case, very occasionally the first correspondence from a legit sender gets stuck in quarantine for a few hours. I see more spam that ham out of these big freemail providers so have long since lost any trust in them. Funny, google manages to keep spam to a minimum (at least from the flow I see here) so one can only conclude the other big freemail providers simply don't care.
Re: antiphishing
On 10/12/2011 07:01 PM, Christian Grunfeld wrote: Hi, I have an idea that I want to discuss with users and developers. Many phishing mails exploit the bad knowledge of the difference between real url and link anchor text by simple users. So they show atractive link text that points to hiden, unrecognized and evil urls. eg: exe files hiden by photo names, etc. My idea is to have a rewrite engine in spamassassin that can rewrite real url in place of the link anchor text or at least to write it near the anchor text without removing it. In that way people can check if both agree or if the url is known or unknown. It would be another step before the "inevitable click" :p The link functionality is not broken in any case (good or evil link) so genuine links can be followed and evil links can be warned ! In sumary...replace text between and by the href or add the href next to the text with an ascii arrow (-->) or something like that. Cheers ! Rather than tampering with the original mail, surely the solution is to clearly detect the mail as spam in the first place so it hopefully never reaches the user. History has taught me that if there's a link, someone *will* click on it regardless of how obvious it might be to you or I that the link is malicious.
Re: Blacklisting based on SPF
On 07/10/11 13:27, Daniel McDonald wrote:
Something like this Unverified Yahoo rule I shameless stole from Mark
Martinec:
I have some similar rules...
header __L_FROM_Y1 From:addr =~ m{[@.]yahoo\.com$}i
header __L_FROM_Y2 From:addr =~ m{\@yahoo\.com\.(ar|br|cn|hk|my|sg)$}i
header __L_FROM_Y3 From:addr =~ m{\@yahoo\.co\.(id|in|jp|nz|uk)$}i
header __L_FROM_Y4 From:addr =~
m{\@yahoo\.(ca|de|dk|es|fr|gr|ie|it|pl|se)$}i
and thought I'd share my updated list of Yahoo TLDs as you're missing a few:
header __LOCAL_FROM_YAHOO1 From:addr =~
/\@yahoo\.com\.(ar|br|cn|hk|mx|my|ph|sg)$/i
header __LOCAL_FROM_YAHOO2 From:addr =~
/\@yahoo\.co\.(id|in|jp|nz|th|uk)$/i
header __LOCAL_FROM_YAHOO3 From:addr =~
/\@yahoo\.(ca|cn|de|dk|es|fr|gr|ie|in|it|pl|ru|se)$/i
Re: Help with tagging hotmail spam
On 04/10/11 05:50, Alex wrote:
Hi,
I have a fedora15 box with v3.3.2 and I have some hotmail spam that I
can't figure out how to catch:
http://pastebin.com/kkUUvYQp
It's hitting BAYES_00 and no blacklists or other significant spam
rules and not sure how to tag it. The user has reported receiving this
spam several times before, each with a different URL in the body but
otherwise the same.
It's still not listed in a URIBL.
X-Originating-IP hits Spamhaus XBL list. I would deep parse headers against
SBL-XBL. This does have the potential for FPs on legitimate mail sent from
infected computers also spewing botnet spam so take that into account in
your scoring.
Okay, I see that it is (now) listed in the XBL, but I have zen being
checked at the smtp level with postfix and it didn't catch it. I guess
it's possible I received it before it was listed, but I also have zen
in SA, and although it appears to hit on zen, it isn't reflected in
the score.
This makes me think there's a different problem I'm having. This means
it hit zen, correct?
Oct 4 00:30:21.417 [12281] dbg: dns: hit
127.0.0.4
I've uploaded a new pastebin with full debugging, and hoped someone
could help me to investigate.
I don't think you quite understand. By default, SA queries zen (which is
a combination of pbl, sbl and xbl) for all IP addresses in the Received
headers but only the last external IP address, which in this case was
the Hotmail server delivering to you - [65.55.90.240], is scored for
hits against the pbl and xbl. So in this case the DNS lookup was done
but the result wasn't used.
It's highly unlikely that a Hotmail server will ever be (intentionally)
listed in Spamhaus.
What I'm talking about above is setting up some new rules to score IP
addresses further down the delivery chain, including in this case the
originating IP. This is sometimes known as deep parsing.
However, it is NOT safe to deep parse received headers against
zen.spamhaus as the zen list includes the pbl and this will generate a
huge number of false positives. However, it is fairly safe to deep parse
against the remaining sbl and xbl lists (SA actually already does this
for the sbl).
Here is the rule I use to query all IPs against the sbl and xbl:
# SBL-XBL is the Spamhaus Block List: http://www.spamhaus.org/
# SBL returns 127.0.0.2, XBL returns 127.0.0.4-8
header RCVD_IN_SBLXBL eval:check_rbl_sub('zen', '127.0.0.[245678]')
describe RCVD_IN_SBLXBL Received via a relay in Spamhaus SBL-XBL
tflags RCVD_IN_SBLXBL net
Then score it as you see fit:
score RCVD_IN_SBLXBL 3
As I said above, this has the potential to cause some false positive
hits against legitimate mail sent from infected spambot hosts, but
generally if mail originated from a spammy host I'd rather know about it.
I've got the score to increase a bit by training it in bayes, but I'm
not even sure that's the right thing to do. How do this affect
legitimate mail received from hotmail?
Bayes is extremely clever and when properly trained with sufficient
examples so I wouldn't worry about it - just make sure you feed examples
of both ham and spam from hotmail.
Re: Help with tagging hotmail spam
On 03/10/11 01:31, Alex wrote: Hi all, I have a fedora15 box with v3.3.2 and I have some hotmail spam that I can't figure out how to catch: http://pastebin.com/kkUUvYQp It's hitting BAYES_00 and no blacklists or other significant spam rules and not sure how to tag it. The user has reported receiving this spam several times before, each with a different URL in the body but otherwise the same. It's still not listed in a URIBL. Any ideas greatly appreciated. Thanks, Alex X-Originating-IP hits Spamhaus XBL list. I would deep parse headers against SBL-XBL. This does have the potential for FPs on legitimate mail sent from infected computers also spewing botnet spam so take that into account in your scoring. As always, manually train Bayes with such examples.
Re: "Your mailbox has exceeded..."
On 30/09/11 18:04, John Hardin wrote: On Fri, 30 Sep 2011, Ned Slider wrote: On 30/09/11 01:41, jida...@jidanni.org wrote: Sure a lot of "Your mailbox has exceeded" spam these days. I'll use body J_MAILBOX_FULL /^Your mailbox has exceeded/ score J_MAILBOX_FULL ... myself for now. I've seen a few of these, but probably not enough examples to have Bayes reliably catch them yet - the first few sneaked straight through uncaught. If we could organise a working group or something, and/or collect some examples, I'd happily help with writing some rules specifically for these. I've been collecting phishes for a while now - in several languages - and I've been intending to start working on some rules, perhaps evolved along the lines of the ADVANCE FEE rules. Anyone who wants to send me .tar.gz archives of phish samples is welcome to do so. John, I have a rather small set (277) of purely UK bank-related phish I started collecting back in 2009 (happy to share them with you if you feel such a small/old set would be useful). Similarly, my plan was to try to write a set of rules to catch these. In the end I took a somewhat different (and simpler) approach of scoring all mail that claims to be FROM a banking domain, and then whitelist legitimate senders through whitelist_from_dkim/spf. # Claiming to be from a bank or other financial institution # We can whitelist legitimate mail using SPF and/or DKIM header LOCAL_FROM_BANK From:addr =~ /\@(abbey|abbeyinternational|abbeynational|allianceleicester|alliance-leicester|bankofamerica|barclaycard|barclays|cahoot|cbonline|citibank|cooperativebank|co-operativebank|cooperative-bank|egg|eggconnect|firstdirect|halifax|halifax-online|hbos|hsbc|hsbcgroup|lloydstsb|mbna|natwest|nationwide|newegg|new\.egg|northernbank|northernrock|nwolb|rbs|santander|santandercards|smile|woolwich|ybonline|zenithbank)\.(com|co\.uk)/i score LOCAL_FROM_BANK 6 describeLOCAL_FROM_BANK From a bank domain I have more rules along this line scoring senders from .org/net domains etc that banks are unlikely to ever use - I'm sure you get the idea. This approach stemmed from the fact that for my small set of banking phish, I noticed nearly all claim to be FROM the primary domain (e.g. @bank.com), wherein in reality (again from the very small set of legitimate banking mails I have) banks typically send from a subdomain, something like @email.bank.com or @e.bank.com etc, so this approach seems to produce very few FPs. Not the most sophisticated approach but it works really well for me - it just needs a different mindset, moving from the default view that one should accept all email and then try to filter out the spam to one where we deny all bank-related email by default and then whitelist the known good senders. Ultimately 99% of email I see claiming to be from a bank was phish. If one were to adopt such an approach, it would be useful to share information of which domains have working spf records and/or dkim signatures as this information isn't always easy to collate, and such domains could either be incorporated into SA's USER_IN_DEF_*_WHITELIST's or shipped as separate whitelist_from_dkim/spf rule sets that users could enable/disable at will. If banks were to clearly publish the (sub)domains they intend to send mail from, and publish accurate spf records/dkim signatures for those domains then this would be an almost trivial issue to solve. I'm really not sure of the best approach to tackling the wider phish problem, nor the specific problem of mailbox phish of this thread.
Re: "Your mailbox has exceeded..."
On 30/09/11 01:41, jida...@jidanni.org wrote: Sure a lot of "Your mailbox has exceeded" spam these days. I'll use body J_MAILBOX_FULL /^Your mailbox has exceeded/ score J_MAILBOX_FULL ... myself for now. I've seen a few of these, but probably not enough examples to have Bayes reliably catch them yet - the first few sneaked straight through uncaught. If we could organise a working group or something, and/or collect some examples, I'd happily help with writing some rules specifically for these.
Re: Increasing score based on membership to commercial whitelist
On 26/09/11 19:00, David F. Skoll wrote: On Mon, 26 Sep 2011 13:49:36 -0400 dar...@chaosreigns.com wrote: On 09/24, David Bennett wrote: It occurred to me that a sender that is paying their way into my inbox is almost certainly sending me junk mail. A little research in my inbox and it turns out to be right on the money. All stuff that I didn't want. Disclaimer: I'm a dnswl.org admin, although haven't been active lately. Also, dnswl.org (provider of the data used by RCVD_IN_DNSWL_* rules) doesn't charge anybody for being listed. They only charge very high volume users of the data for use of the data, like Spamhaus and some other major blacklist providers. As someone listed on dnswl.org, I can confirm this. We did not pay to get our domain (roaringpenguin.com) or IP addresses listed. And I assume that if we spam, we will be delisted in a hurry. So please don't automatically assume that we're spammers just because we're on dnswl.org. :) Regards, David. Same here. I'm not generally a big fan of whitelists (at least not someone else's whitelists), but the fact I bothered to sign up serves as a good indicator of the level of trust I have in dnswl.org - and that's trust that has been earned based on what I see within my own mail flow. Each to their own of course, but I have no issue with dnswl.org nor it's default scoring in SA which seems very reasonable to me.
Re: Why does this hit __HAS_ANY_URI
On 22/08/11 21:46, John Hardin wrote:
On Mon, 22 Aug 2011, Ned Slider wrote:
uri __REALLY_HAS_ANY_URI m{https?://.}
and if we want to test for email addresses:
uri __HAS_ANY_URI_EMAIL /@/
and make __HAS_ANY_URI (and __DOS_HAS_ANY_URI) a meta of the above two
rules.
Given they're the vast majority, but https? and mailto are not the only
protocols supported in URIs. Such a meta would ignore them.
I could see some subrules that try to distinguish between different URI
protocols. What are you trying to accomplish?
I was using __HAS_ANY_URI in a local meta rule and was just surprised
when my meta rule hit on a spam that didn't contain a "URI" (as per my
original post). Of course I now realise it matched the plain text email
address in the body. I was thinking purely in terms of https? rather
than other (less common) protocols.
Re: Why does this hit __HAS_ANY_URI
On 22/08/11 20:37, Adam Katz wrote:
On 08/14/2011 02:17 PM, Ned Slider wrote:
Hi all,
The following email hits __HAS_ANY_URI and I'm not sure why:
http://pastebin.com/jvFrFhA4
When I run the message through SpamAssassin in debug mode I see:
dbg: rules: __DOS_HAS_ANY_URI merged duplicates: __HAS_ANY_URI
dbg: rules: ran uri rule __DOS_HAS_ANY_URI ==> got hit: "r"
SpamAssassin version 3.3.2
Any clues? I'm guessing it has something to do with the email address in
the message body?
The letter "r" is the beginning of an implicit mailto: URI in the body
(ram12...@live.com).
Thanks, you are right.
I didn't realise URI rules also matched email addresses but a simple
test confirms that any plain text email address in the message body
causes __HAS_ANY_URI to fire.
I guess that's just not what I had expected. Here's my revised version
that behaves more as I had expected:
uri __REALLY_HAS_ANY_URIm{https?://.}
and if we want to test for email addresses:
uri __HAS_ANY_URI_EMAIL /@/
and make __HAS_ANY_URI (and __DOS_HAS_ANY_URI) a meta of the above two
rules.
Why does this hit __HAS_ANY_URI
Hi all, The following email hits __HAS_ANY_URI and I'm not sure why: http://pastebin.com/jvFrFhA4 When I run the message through SpamAssassin in debug mode I see: dbg: rules: __DOS_HAS_ANY_URI merged duplicates: __HAS_ANY_URI dbg: rules: ran uri rule __DOS_HAS_ANY_URI ==> got hit: "r" SpamAssassin version 3.3.2 Any clues? I'm guessing it has something to do with the email address in the message body? Thanks.
Re: RFE: high, med and low whitelist_from_dkim and whitelist_from_spf
On 01/08/11 12:23, Mark Martinec wrote: Ned, On the same basis that some DNSWLs have high, medium and low scores depending upon the level of trust, I'm wondering if it would be useful to have user defined high, medium and low scores available to rules such as whitelist_from_dkim and whitelist_from_spf so mail admins can score depending on the level of trust. Would it be feasible and/or desirable to implement such a feature? E.g, whitelist_from_dkim_low whitelist_from_dkim_med whitelist_from_dkim_high full DKIM_VALID_Y eval:check_dkim_valid(yahoogroups.com, .yahoo.com, .yahoo.ca, .yahoo.de, .yahoo.fr, .yahoo.in, .yahoo.co.in, .yahoo.co.jp, .yahoo.co.nz, .yahoo.co.uk, .yahoo.com.hk, .yahoo.com.ph, .yahoo.com.vn) full DKIM_VALID_G eval:check_dkim_valid(gmail.com, googlemail.com, googlegroups.com) full DKIM_VALID_WEGAME eval:check_dkim_valid(email.wegame.com) score DKIM_VALID_Y -0.5 score DKIM_VALID_G -0.8 score DKIM_VALID_WEGAME -8 Mark Brilliant - Thanks Mark!
RFE: high, med and low whitelist_from_dkim and whitelist_from_spf
Hi all, On the same basis that some DNSWLs have high, medium and low scores depending upon the level of trust, I'm wondering if it would be useful to have user defined high, medium and low scores available to rules such as whitelist_from_dkim and whitelist_from_spf so mail admins can score depending on the *level* of trust. Would it be feasible and/or desirable to implement such a feature? E.g, whitelist_from_dkim_low whitelist_from_dkim_med whitelist_from_dkim_high etc.
Re: RP_MATCHES_RCVD
On 28/07/11 15:28, RW wrote: There seems to be a consensus that SPF and DKIM passes aren't worth significant scores. So how is it that RP_MATCHES_RCVD, scores -1.2 when it just a circumstantial version of what SPF does explicitly. For me it's hitting more spam that ham, and what's worse, it's mostly hitting low-scoring freemail spam. Is it just me that's seeing this, or is there maybe some kind of bias the test corpora? Yes, I've noticed this too recently and had knocked the score down to 0.001 for information only about a week ago. I've found it hitting on spam and didn't find it useful on ham (i.e, I don't generally suffer from ham being mis-classified as spam).
JM_SOUGHT rules hitting Facebook again
Hi, JM_SOUGHT_3 is hitting Facebook notifications again. The offending rule appears to be: body __SEEK_2GW7AI /\. If you don\'t want to receive these emails from Facebook in the future or have your email address used for friend suggestions, you can unsubscribe\. / Channel update was: Jul 28 11:38:52.060 [16462] dbg: dns: 1.3.3.sought.rules.yerp.org => 3301151446, parsed as 3301151446
Re: sa-update failing
On 18/07/11 02:58, Warren Togami Jr. wrote: On 7/17/2011 7:55 AM, Axb wrote: On 2011-07-17 18:32, Warren Togami Jr. wrote: On 7/16/2011 4:54 AM, dar...@chaosreigns.com wrote: On 07/15, ssapp80 wrote: Running spamassassin-3.3.2 on CentOS 5.5 perl-Net-DNS ver 0.59 installed When I run sa-update i receive the following failures on the Net::DNS module "name2labels" is not exported by the Net::DNS module My guess is Net::DNS version 0.59 is too old. In which case it would be nice if spamassassin specified that in its use statement so it would give a more useful error. If upgrading that perl module fixes that problem, it might be worth opening a bug to improve that error. I'm using Net::DNS v0.65. EL-5 has perl-Net-DNS 0.59. I've been using it in production for years without issue, even with 3.3.2. This is not the droid we're looking for. Move along. Warren unless RH has patched perl-Net-DNS 0.59, iirc, the original had some issues in RR.pm (going back to 2007) including some security thingie. I'd recomend using 0.63 or higher. http://pkgs.repoforge.org/perl-Net-DNS/ offers a good choice. (I'm using 0.66 and happy) Looking at the changelog, it appears they did patch several bugs and a security issue. All I know is I have had no issues using EL-5's 0.59 for years now. Warren Likewise, no problems running SpamAssassin-3.3.1 with perl-Net-DNS-0.59-3 dating from 2007 on EL-5.
Re: Lowering spam threshold
On 06/07/11 09:17, Lars Jørgensen wrote: I think many people run with tag at 5.0 and discard at 10.0 I should have mentioned that we are running amavisd-new. I thought that was the de facto way of integrating spamassassin into a mail gateway, but reading this list reveals that most people probably doesn't do that. Makes me wonder if I am doing the wrong thing? Amavisd-new has further settings as to thresholds, and these are the ones I put in as of today (after reading other peoples tips here, thank you everybody): $sa_tag_level_deflt = -10; # add spam info headers if at, or above that level $sa_tag2_level_deflt = 5.2; # add 'spam detected' headers at that level $sa_kill_level_deflt = 6.2; # triggers spam evasive actions (e.g. blocks mail) $sa_dsn_cutoff_level = 7.4; # spam level beyond which a DSN is not sent Does above scores make sense? Yes, makes perfect sense to other amavisd-new users. I currently tag at 5.0 (the default SA score) and quarantine at 6.0. I also set the DSN cut-off level to be the same as quarantine as I don't want to send DSNs. If you are finding spam is getting through untagged with the default SA score of 5.0 then I would look to write some additional rules to target those spam that are getting through rather than lowering the score below the SA default of 5.0. This list can help you with that if you provide examples. Additionally, I have very carefully hand trained bayes with only confirmed spam/ham and tweaked the scores to be more representative of the faith I have in my bayes data. I find many cases where bayes alone will identify spam and have scored bayes_99 accordingly. The main "problem" I see with SA is that I reject all the easy spam (>90%) at the smtp level so SA only really gets to see the more difficult and less obvious stuff. If SA saw all spam then the detection rates out of the box would be extremely high, but with only the more difficult samples to chew on detection rates inevitably drop and are artificially lowered. As a result it can appear that a lot of spam is getting through when in reality the overall percentage is still really small. That last 1% is just hard to catch without increasing the risk of false positives.
Re: X-Spam-Relays-External
On 29/06/11 12:50, Henrik K wrote: On Wed, Jun 29, 2011 at 01:28:48PM +0300, Henrik K wrote: On Wed, Jun 29, 2011 at 11:02:13AM +0100, Ned Slider wrote: Hi List, I see the useful X-Spam-Relays-External pseudo header but what I'd really like to be able to specifically check is the Last External header as DNSBL rules are able to do with -lastexternal. Is there a X-Spam-Relays-Last-External option that I'm missing, and if not would it be possible to implement such a feature or perhaps someone can suggest a workaround method? For example, I'd like to be able to do something like this against only the last external Received header: header __RCVD_FROM_SOMEISP X-Spam-Relays-Last-External =~ /someisp\.com/i Thanks Example from 20_dynrdns.cf # Note the '^[^\]]+ ' stanza: this ensures that we only match spamware # connecting to a internal relay; if a mail came from a dynamic addr but # was relayed through their smarthost, that's fine. ... header __LAST_EXTERNAL_RELAY_NO_AUTH X-Spam-Relays-External =~ /^[^\]]+ auth= / To prevent further questions.. header __RCVD_FROM_SOMEISP X-Spam-Relays-External =~ /^[^\]]+ rdns=[^ ]+\.someisp\.com / As you see, all the relays are enclosed in [ ..relay1.. ] [ ..relay2.. ] ... Thus the stanza will not look further than first ]. Brilliant - thank you very much. Works perfectly.
Re: X-Spam-Relays-External
On 29/06/11 11:24, Benny Pedersen wrote: On Wed, 29 Jun 2011 11:02:13 +0100, Ned Slider wrote: header __RCVD_FROM_SOMEISP X-Spam-Relays-Last-External =~ /someisp\.com/i bad rule, hostnames can have more then one ip, would you trust every ip now ? Who said anything about trusting the IP ? I simply want to verify that the email was relayed to me from a particular ISP as part of a meta rule. The very fact that the hostname(s) do have many IPs is the reason for matching that rather than trying to match multiple subnet ranges. better would be to extend ASN plugin to have whitelist specific ASN or blacklist
