[Declude.JunkMail] Deleting emails based solely on Sniffer?
Can someone please explain to me why, if an email is flagged as spam by Sniffer, I shouldn't just delete it outright? Are there instances where Sniffer is wrong? Or is this the way you all use it already? Reason I ask is that I have Sniffer setup with a weight of 10...and I hold messages with a weight of 10-14. This morning I got a Nigerian-type scam that sniffer flagged, but it only scored a total weight of 5. I'll have to check through my global.cfg when I get back from my 9am meeting, but something added a weight of -5 somewhere, meaning the email got through. If I had deleted all Sniffer-found spam outright, this would not have happened. Thoughts? _ Joey Proulx SAU #21 Technology Support Staff 2 Alumni Drive Hampton, NH 03842 (603) 926-8992, ext 115 [EMAIL PROTECTED] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] Deleting emails based solely on Sniffer?
If you delete, you should delete based on achieving a minimum weight accumulated. Sniffer on occasion may detect something as a false positive. For example, it may misinterpret a legitimate e-mail as Spam with an attachment based on conversion of the attachment to characters and a series triggering something in Sniffer rules. I have seen this on occasion. In our scenario, we hold on a certain weight range for review, and higher weight range we auto-delete. We also will hold if failing Sniffer alone and no other tests. HTH's -Don -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Joey Proulx Sent: Thursday, April 14, 2005 8:50 AM To: Declude.JunkMail@declude.com Subject: [Declude.JunkMail] Deleting emails based solely on Sniffer? Can someone please explain to me why, if an email is flagged as spam by Sniffer, I shouldn't just delete it outright? Are there instances where Sniffer is wrong? Or is this the way you all use it already? Reason I ask is that I have Sniffer setup with a weight of 10...and I hold messages with a weight of 10-14. This morning I got a Nigerian-type scam that sniffer flagged, but it only scored a total weight of 5. I'll have to check through my global.cfg when I get back from my 9am meeting, but something added a weight of -5 somewhere, meaning the email got through. If I had deleted all Sniffer-found spam outright, this would not have happened. Thoughts? _ Joey Proulx SAU #21 Technology Support Staff 2 Alumni Drive Hampton, NH 03842 (603) 926-8992, ext 115 [EMAIL PROTECTED] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. -- CompBiz.Net scanned for Virus' --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] Deleting emails based solely on Sniffer?
On 14 Apr 2005 at 8:50, Joey Proulx wrote: Hi Joey, Can someone please explain to me why, if an email is flagged as spam by Sniffer, I shouldn't just delete it outright? Are there instances where Sniffer is wrong? Or is this the way you all use it already? Well from my perspective the beauty of Declude is you can use multiple tests to fasil an email - as I'm sure you are aware. No doubt an email that fails sniffer needs to be punished however to delete on that one test may cause some good email to be deleted.. For example I do get false positives on newsletters and some lists I belong to. So I generally wack an email 70% [varies depending of return code] of my hold weight and look for other failures to push it over the threshold My .02 ... :) -Nick --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] Deleting emails based solely on Sniffer?
- Original Message - From: Joey Proulx [EMAIL PROTECTED] Can someone please explain to me why, if an email is flagged as spam by Sniffer, I shouldn't just delete it outright? Are there instances where Sniffer is wrong? Or is this the way you all use it already? Reason I ask is that I have Sniffer setup with a weight of 10...and I hold messages with a weight of 10-14. This morning I got a Nigerian-type scam that sniffer flagged, but it only scored a total weight of 5. I'll have to check through my global.cfg when I get back from my 9am meeting, but something added a weight of -5 somewhere, meaning the email got through. If I had deleted all Sniffer-found spam outright, this would not have happened. Thoughts? I wouldn't recommend doing that, since I typically submit a few false-positives each week to the Sniffer false@ address. The better thing to do, as you said, is determine what test(s) is/are reducing the weight and adjust it. Bill --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] Deleting emails based solely on Sniffer?
Joey Proulx writes: Can someone please explain to me why, if an email is flagged as spam by Sniffer, I shouldn't just delete it outright? Are there instances where Sniffer is wrong? Or is this the way you all use it already? A couple of things Sniffer is very effective but not perfect close. There are false positives. The common rule is that no message should not be delivered because of one test. Now on my system Sniffer is right under the hold weight which means a second test is required to push it over. Darrell -- Try invURIBL - an advanced URI filtering test that will block more than 85% of all SPAM with the default configuration? Try it for free http://www.invariantsystems.com/invuribl/default.htm --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] Deleting emails based solely on Sniffer?
On Thursday, April 14, 2005, 8:50:12 AM, Joey wrote: JP Can someone please explain to me why, if an email is flagged as spam by JP Sniffer, I shouldn't just delete it outright? Are there instances where JP Sniffer is wrong? Or is this the way you all use it already? JP Reason I ask is that I have Sniffer setup with a weight of 10...and I hold JP messages with a weight of 10-14. This morning I got a Nigerian-type scam JP that sniffer flagged, but it only scored a total weight of 5. I'll have to JP check through my global.cfg when I get back from my 9am meeting, but JP something added a weight of -5 somewhere, meaning the email got JP through. If I had deleted all Sniffer-found spam outright, this would not JP have happened. JP Thoughts? ... Just adding to the thread... First, I agree with Nick Don ... As much as we try to make SNF perfect, the definition of it's design, and the fact of any spam test dictate that there will be some error rate. For example, our false positive handling process is based on our best guess about the consensus of all of our customers Do most of the people we serve agree with this rule? Is that agreement worth the risk of a false positive? These questions are answered primarily by statistics... The point is that there is a gray area where some folks will always find a false positive (and we generally will adjust their rulebase accordingly). That somebody could be you :-) So it is safest NOT to delete on SNF, or for that matter any single test - even if that will lead to some spam getting through. This is one of the key benefits of Declude is it's weighting system. That said, the best practice (as I observe it) is to always hold on SNF and to delete on a specific weight that is high enough to include at least two other tests. Using this strategy, any FP generated by SNF will still be around to be noticed if it is discovered - either by review or by a customer asking why some message appears to be missing. The message can then be recovered, a false positive report made, and appropriate adjustments implemented. In your scenario you might want to set the weight of SNF higher so that the -5 might still keep the message in your hold range. This might force you to adjust your upper limit on the hold weight, but it's a decent compromise I think. In the end only you can know for sure what is the best strategy for your system. All of this is a balance of resources and risks. There are many happy systems out there that do regularly delete messages on a single test - for example IMGate which has been debated widely. While I would not recommend deleting a message solely on SNF as a general practice, clearly there is room for this strategy on some systems. Hope this helps, _M --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] Something new with v 2.0.6
Title: Message The Space was the issue. Added the "-" and all is well. - Original Message - From: Matt To: Declude.JunkMail@declude.com Sent: Wednesday, April 13, 2005 4:37 PM Subject: Re: [Declude.JunkMail] Something new with v 2.0.6 Fred,Those are all legit. Andy has keen eyes and I suspect that he may have identified the trigger, though it would be a bug in Declude to behave in this way, but a minor one.The examples that you gave all have no spaces prior to the first colon, and that is compliant. The one that Andy gave was clearly not, and it is the one that is also causing you problems.MattFrederick Samarelli wrote: Good Thought but I have these others without problem. Thanks. XINHEADER X-Note: Total spam weight of this E-mail is %WEIGHT%.XINHEADERX-RBL-Warning: Total weight: %WEIGHT%XINHEADERX-Note: This E-mail was scanned filtered by TCB [%VERSION%] for SPAM virus.XINHEADERX-Note: Sent from: %MAILFROM%XINHEADERX-Note: Sent from Reverse DNS: %REVDNS% ([%REMOTEIP%])XINHEADERX-Note: Recipient(s): %REALRECIPS%- Original Message - From: Andy Schmidt To: Declude.JunkMail@declude.com Sent: Wednesday, April 13, 2005 4:02 PM Subject: RE: [Declude.JunkMail] Something new with v 2.0.6 Hi Frederick: I don't know if this has been asked/suggested already and I don't have time to go back to the RFCs to see if embedded spaces are permitted in the header name. But have you ever tried eliminating that space: XINHEADERX-Spam-Tests-Failed Weight: %TESTSFAILEDWITHWEIGHTS% replace with: XINHEADERX-Spam-Tests-Failed-Weight: %TESTSFAILEDWITHWEIGHTS% May be the problem is that there is a CR/LF followed by a line that contains no header name(due to the embedded space) following by another CR/LF. May be those two CR/LF without valid header information inbetween are interpreted as "start of message body" by some entities? Best RegardsAndy SchmidtPhone: +1 201 934-3414 x20 (Business)Fax: +1 201 934-9206 -Original Message-From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of Frederick SamarelliSent: Wednesday, April 13, 2005 03:42 PMTo: Declude.JunkMail@declude.comSubject: Re: [Declude.JunkMail] Something new with v 2.0.6 Mike/Matt (thanks for your help) You should be able to duplicated by just forwarding an email to an outside account using the problem line at the bottom. As not to confuse things I simplified the process. Send an email from [EMAIL PROTECTED]= [EMAIL PROTECTED](forwarded to) = [EMAIL PROTECTED] This run through only one server on my network. Header from My AOL account. Return-Path: [EMAIL PROTECTED]Received: from rly-xm04.mx.aol.com (rly-xm04.mail.aol.com [172.20.83.105]) by air-xm03.mail.aol.com (v105.26) with ESMTP id MAILINXM31-606425d743d132; Wed, 13 Apr 2005 15:34:25 -0400Received: from bks.tcbinc.com (bks.tcbinc.com [64.124.117.196]) by rly-xm04.mx.aol.com (v105.26) with ESMTP id MAILRELAYINXM42-606425d743d132; Wed, 13 Apr 2005 15:34:21 -0400Received: from SMTP32-FWD by bks.tcbinc.com (SMTP32) id A741100040470EC67; Wed, 13 Apr 2005 15:33:42 Received: from web51806.mail.yahoo.com [206.190.38.237] by bks.tcbinc.com (SMTPD32-8.15) id A41140470; Wed, 13 Apr 2005 15:33:37 -0400Received: (qmail 50369 invoked by uid 60001); 13 Apr 2005 19:34:12 -Comment: DomainKeys? See http://antispam.yahoo.com/domainkeysDomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; b=M12dWrk8x99pS4FhLTVJbfbgIc60YrjkjS/Vi2yiCoare5X2fk5F+zDzAA2XuOKAyAuKoj3EEGBHc6gPlwybZ/TMSShXoJtIypUpKUZZrm7SoU0rx30hedmPe9IecDArBynamRJFf8HjmCsGFKGIwJhKUjwV4wNnw1wLdarF7SE= ;Message-ID: [EMAIL PROTECTED]Received: from [64.124.117.139] by web51806.mail.yahoo.com via HTTP; Wed, 13 Apr 2005 12:34:12 PDTDate: Wed, 13 Apr 2005 12:34:12 -0700 (PDT)From: Frederick Samarelli [EMAIL PROTECTED]Subject: test10To: [EMAIL PROTECTED]MIME-Version: 1.0Content-Type: text/plain; charset=us-asciiX-RBL-Warning: SNIFFERZERO: Message failed SNIFFERZERO: 0.X-Declude-Sender: [EMAIL PROTECTED] [206.190.38.237]X-Declude-Spoolname: D741100040470EC67.SMDX-Note: Total spam weight of this E-mail is 0.X-RBL-Warning: Total weight: 0X-Note: This E-mail was scanned filtered by TCB [2.0.6] for SPAM virus.X-Spam-Tests-Failed: SNIFFERZERO Message
Re: [Declude.JunkMail] Deleting emails based solely on Sniffer?
I certainly wouldn't change my Sniffer weighting based on a 419 scam. The 419/Lotteries tend to be some of the more difficult spams to catch. Many of them come from legitate mail servers so they won't be on any blacklists and they won't score on technical tests. In your case I'd bet the -5 came from a combination of IPNOTINMX and NOLEGITCONTENT which will tend to trigger on 419 emails. - Original Message - From: Joey Proulx [EMAIL PROTECTED] To: Declude.JunkMail@declude.com Sent: Thursday, April 14, 2005 7:50 AM Subject: [Declude.JunkMail] Deleting emails based solely on Sniffer? Can someone please explain to me why, if an email is flagged as spam by Sniffer, I shouldn't just delete it outright? Are there instances where Sniffer is wrong? Or is this the way you all use it already? Reason I ask is that I have Sniffer setup with a weight of 10...and I hold messages with a weight of 10-14. This morning I got a Nigerian-type scam that sniffer flagged, but it only scored a total weight of 5. I'll have to check through my global.cfg when I get back from my 9am meeting, but something added a weight of -5 somewhere, meaning the email got through. If I had deleted all Sniffer-found spam outright, this would not have happened. Thoughts? _ Joey Proulx SAU #21 Technology Support Staff 2 Alumni Drive Hampton, NH 03842 (603) 926-8992, ext 115 [EMAIL PROTECTED] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
[Declude.JunkMail] Negative weighting filters to reduce false positives
We just started something I've been thinking about
for a while: Negative weight tests to offset specific test failures for
well-known domains. For example, a large
number of false positives we see are from Earthlink, Mindspring, Sprint,
Verizon, etc.
Now you may be thinking, of course, these are large
providers with dial-up user bases, so you would expect a large percentage
offalse positives tobe from them...but hold on a minute. Many
of these large domains are being penalized in our system for routing or not
having abuse@ or postmaster@ addresses. Almost all of these would not have
ended up in the hold queue if they had not been so penalized...thus the idea to
figure out a manageable way to NOT penalize them for these technical RFC
violations.
So, what we've done is to start filters to
counteract the weights for major tests that a few of these domains fail.
By doing it specifically for a particular domain, we reduce false positives but
avoid losing theeffectiveness of the test on other domains.
Anyway,attached zip are the filter
files. As I mentioned, they have just been started, so there are just a
few domains in them at present. At the top of the filter file are
suggested guidelines on how to use them. There are probably better ways to
handle this, so I welcome comments/feedback.
Darin.
#
# Filter Name: n_ROUTING
# Author: Darin Cox
# Date: 4/14/2005
# Description: Used to negate the weight of the ROUTING test for select domains
#
# Notes:Recommended config is to negative weight this test to
# exactly offset the ROUTING test. For example, if you have
#
# ROUTING spamrouting x
x 125 0
#
# Then this test is recommended to be added as
#
# n_ROUTING filter C:\{MAILSERVER}\Declude\n_ROUTING.txt
x -1250
#
#
# Don't run the test unless the email failed the ROUTING test
TESTSFAILED END NOTCONTAINS ROUTING
MAILFROMEND ENDSWITHmail.sprint.com
#
# Filter Name: n_NOABUSE
# Author: Darin Cox
# Date: 4/14/2005
# Description: Used to negate the weight of the NOABUSE test for poorly
# administered domains from which we still need to accept mail
#
# Notes:Recommended config is to negative weight this test to
# exactly offset the NOABUSE test. For example, if you have
#
# NOABUSE rhsbl abuse.rfc-ignorant.org
127.0.0.4 20 0
#
# Then this test is recommended to be added as
#
# n_NOABUSE filter C:\{MAILSERVER}\Declude\n_NOABUSE.txt
x -20 0
#
#
# Don't run the test unless the email failed the NOABUSE test
TESTSFAILED END NOTCONTAINS NOABUSE
MAILFROMEND ENDSWITHearthlink.net
MAILFROMEND ENDSWITHmail.sprint.com
MAILFROMEND ENDSWITHmindspring.com
MAILFROMEND ENDSWITHverizon.com
#
# Filter Name: n_NOPOSTMASTER
# Author: Darin Cox
# Date: 4/14/2005
# Description: Used to negate the weight of the NOPOSTMASTER test for poorly
# administered domains from which we still need to accept mail
#
# Notes:Recommended config is to negative weight this test to
# exactly offset the NOPOSTMASTER test. For example, if you have
#
# NOPOSTMASTERrhsbl abuse.rfc-ignorant.org
127.0.0.3 25 0
#
# Then this test is recommended to be added as
#
# n_NOPOSTMASTER filter
C:\{MAILSERVER}\Declude\n_NOPOSTMASTER.txt x -25 0
#
#
# Don't run the test unless the email failed the NOPOSTMASTER test
TESTSFAILED END NOTCONTAINS NOPOSTMASTER
MAILFROMEND ENDSWITHverizon.com
RE: [Declude.JunkMail] Negative weighting filters to reduce false positives
This is pretty interesting, but one question What is your hold weight set to? It seems that you are assigning a huge negative value for the first test, and much smaller for the other two, nay insight as to how you came up with these values? We are running into some of the same problems here, and this is an interesting idea for a way around it. - greg From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Darin Cox Sent: Thursday, April 14, 2005 5:41 PM To: Declude.JunkMail@declude.com Subject: [Declude.JunkMail] Negative weighting filters to reduce false positives We just started something I've been thinking about for a while: Negative weight tests to offset specific test failures for well-known domains. For example, a large number of false positives we see are from Earthlink, Mindspring, Sprint, Verizon, etc. Now you may be thinking, of course, these are large providers with dial-up user bases, so you would expect a large percentage offalse positives tobe from them...but hold on a minute. Many of these large domains are being penalized in our system for routing or not having abuse@ or postmaster@ addresses. Almost all of these would not have ended up in the hold queue if they had not been so penalized...thus the idea to figure out a manageable way to NOT penalize them for these technical RFC violations. So, what we've done is to start filters to counteract the weights for major tests that a few of these domains fail. By doing it specifically for a particular domain, we reduce false positives but avoid losing theeffectiveness of the test on other domains. Anyway,attached zip are the filter files. As I mentioned, they have just been started, so there are just a few domains in them at present. At the top of the filter file are suggested guidelines on how to use them. There are probably better ways to handle this, so I welcome comments/feedback. Darin.
Re: [Declude.JunkMail] Negative weighting filters to reduce false positives
I myself have pondered why I am even running the RFCI-noabuse and RFCI-nopostmaster test. The NoAbuse misfired 23.6% of the time. The NoPostmaster misfired 12.7% of the time. Due to the underperformance, I weight each test 5 (hold at 200). Thetest failures are who's who of ISP / webmail providers. The vast majority of the results are from e-mails with faked mailfrom addresses... Zombie spammers. I wonder if you'd be better of using the REVDNS instead of the Mailfrom. - Original Message - From: Darin Cox To: Declude.JunkMail@declude.com Sent: Thursday, April 14, 2005 4:41 PM Subject: [Declude.JunkMail] Negative weighting filters to reduce false positives We just started something I've been thinking about for a while: Negative weight tests to offset specific test failures for well-known domains. For example, a large number of false positives we see are from Earthlink, Mindspring, Sprint, Verizon, etc. Now you may be thinking, of course, these are large providers with dial-up user bases, so you would expect a large percentage offalse positives tobe from them...but hold on a minute. Many of these large domains are being penalized in our system for routing or not having abuse@ or postmaster@ addresses. Almost all of these would not have ended up in the hold queue if they had not been so penalized...thus the idea to figure out a manageable way to NOT penalize them for these technical RFC violations. So, what we've done is to start filters to counteract the weights for major tests that a few of these domains fail. By doing it specifically for a particular domain, we reduce false positives but avoid losing theeffectiveness of the test on other domains. Anyway,attached zip are the filter files. As I mentioned, they have just been started, so there are just a few domains in them at present. At the top of the filter file are suggested guidelines on how to use them. There are probably better ways to handle this, so I welcome comments/feedback. Darin.
Re: [Declude.JunkMail] Negative weighting filters to reduce false positives
Darin Cox wrote: Hi Darin - We just started something I've been thinking about for a while: Negative weight tests to offset specific test failures for well- known domains. For example, a large number of false positives we see are from Earthlink, Mindspring, Sprint, Verizon, etc. Well here is one way - If you have tests you like but do not want them to punish certain entities you can do something like this in a filter file: # prequalifying tests that have to fail TESTSFAILED END NOTCONTAINS testname TESTSFAILED END NOTCONTAINS filtername etc - # who not to punish MAILFROM ENDCONTAINSEarthlink etc - # if it gets to here wack the email REMOTEIP0 CONTAINS. Or go the other route and credit back [negative values] as need be- -Nick --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] Negative weighting filters to reduce false positives
Sorry...should've mentioned our weighting scale. We hold at 100 and delete at 300. Those were just examples, however. The point is to weight them exactly opposite of your current weight for those tests. Darin. - Original Message - From: Greg Birdsall To: Declude.JunkMail@declude.com Sent: Thursday, April 14, 2005 5:54 PM Subject: RE: [Declude.JunkMail] Negative weighting filters to reduce false positives This is pretty interesting, but one question What is your hold weight set to? It seems that you are assigning a huge negative value for the first test, and much smaller for the other two, nay insight as to how you came up with these values? We are running into some of the same problems here, and this is an interesting idea for a way around it. - greg From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Darin CoxSent: Thursday, April 14, 2005 5:41 PMTo: Declude.JunkMail@declude.comSubject: [Declude.JunkMail] Negative weighting filters to reduce false positives We just started something I've been thinking about for a while: Negative weight tests to offset specific test failures for well-known domains. For example, a large number of false positives we see are from Earthlink, Mindspring, Sprint, Verizon, etc. Now you may be thinking, of course, these are large providers with dial-up user bases, so you would expect a large percentage offalse positives tobe from them...but hold on a minute. Many of these large domains are being penalized in our system for routing or not having abuse@ or postmaster@ addresses. Almost all of these would not have ended up in the hold queue if they had not been so penalized...thus the idea to figure out a manageable way to NOT penalize them for these technical RFC violations. So, what we've done is to start filters to counteract the weights for major tests that a few of these domains fail. By doing it specifically for a particular domain, we reduce false positives but avoid losing theeffectiveness of the test on other domains. Anyway,attached zip are the filter files. As I mentioned, they have just been started, so there are just a few domains in them at present. At the top of the filter file are suggested guidelines on how to use them. There are probably better ways to handle this, so I welcome comments/feedback. Darin.
Re: [Declude.JunkMail] Negative weighting filters to reduce false positives
Well you make a good point on the value of NOABUSE and NOPOSTMASTER. NOABUSE hits on about 18% of incoming mail, while NOPOSTMASTER hits on about 10%. Most of the false positives we see from them can be eliminated with these filters, extending their usefulnessso there's still value in them for us. Certainly REVDNS would avoid forging issues... we just went with MAILFROM initially since many of the false positives we saw were coming from various mailservers, some in the domain, some notbut the filter files could mix MAILFROM and REVDNS as desired, depending on the patterns seen. Darin. - Original Message - From: Scott Fisher To: Declude.JunkMail@declude.com Sent: Thursday, April 14, 2005 6:03 PM Subject: Re: [Declude.JunkMail] Negative weighting filters to reduce false positives I myself have pondered why I am even running the RFCI-noabuse and RFCI-nopostmaster test. The NoAbuse misfired 23.6% of the time. The NoPostmaster misfired 12.7% of the time. Due to the underperformance, I weight each test 5 (hold at 200). Thetest failures are who's who of ISP / webmail providers. The vast majority of the results are from e-mails with faked mailfrom addresses... Zombie spammers. I wonder if you'd be better of using the REVDNS instead of the Mailfrom. - Original Message - From: Darin Cox To: Declude.JunkMail@declude.com Sent: Thursday, April 14, 2005 4:41 PM Subject: [Declude.JunkMail] Negative weighting filters to reduce false positives We just started something I've been thinking about for a while: Negative weight tests to offset specific test failures for well-known domains. For example, a large number of false positives we see are from Earthlink, Mindspring, Sprint, Verizon, etc. Now you may be thinking, of course, these are large providers with dial-up user bases, so you would expect a large percentage offalse positives tobe from them...but hold on a minute. Many of these large domains are being penalized in our system for routing or not having abuse@ or postmaster@ addresses. Almost all of these would not have ended up in the hold queue if they had not been so penalized...thus the idea to figure out a manageable way to NOT penalize them for these technical RFC violations. So, what we've done is to start filters to counteract the weights for major tests that a few of these domains fail. By doing it specifically for a particular domain, we reduce false positives but avoid losing theeffectiveness of the test on other domains. Anyway,attached zip are the filter files. As I mentioned, they have just been started, so there are just a few domains in them at present. At the top of the filter file are suggested guidelines on how to use them. There are probably better ways to handle this, so I welcome comments/feedback. Darin.
