Re: Shortcircuit Question
Clayton Keller wrote: I have been reading throught the Shortcircuit manpage as well as some articles within the Wiki, and the manner in which I see it performing within our install does not seem to coincide with how I am reading and presumably understanding it to work. First off, we are using SpamAssassin 3.2.4 provided by the rpmforge batch of rpm's. I have about a dozen priorities specified mainly handling URIBL, SURBL, as well as DCC, Razor, Pyzor and Bayes. What I am seeing is that although the first shortcircuit rule hits and scores appropriately. Subsequent short circuit rules will continue to fire. The scores themselves are then totaled along with the original scores for the rules. My understanding of how the shortcircuit should work is that once a shortcircuit is triggered any subsequent rules should be bypassed and the message wither classified as spam/ham or if set to on, it would use the current score specified for the rule. As a for instance: I have the following: priority URIBL_BLACK-500 priority URIBL_JP_SURBL -498 priority URIBL_SC_SURBL -488 priority URIBL_OB_SURBL -487 priority SC_URIBL_SURBL -480 priority SC_URIBL_SBL -479 priority RAZOR2_CHECK -450 priority DCC_CHECK -449 priority PYZOR_CHECK-448 priority SC_URIBL_HASH -440 meta SC_URIBL_SURBL(URIBL_BLACK (URIBL_SC_SURBL || URIBL_JP_SURBL || URIBL_OB_SURBL)) meta SC_URIBL_SBL((URIBL_BLACK || URIBL_SC_SURBL || URIBL_JP_SURBL || URIBL_OB_SURBL) URIBL_SBL) meta SC_URIBL_HASH((URIBL_BLACK || URIBL_SC_SURBL || URIBL_JP_SURBL || URIBL_OB_SURBL) (RAZOR2_CHECK || DCC_CHECK || PYZOR_CHECK)) meta SC_URIBL_SBL ((URIBL_BLACK || URIBL_SC_SURBL || URIBL_JP_SURBL || URIBL_OB_SURBL) URIBL_SBL) shortcircuit SC_URIBL_SURBL spam shortcircuit SC_URIBL_SBL spam shortcircuit SC_URIBL_HASH spam score SC_URIBL_SURBL100.00 score SC_URIBL_HASH 100.00 score SC_URIBL_SBL 100.00 I do not have a recent debug to show but I can say that from the debug I do see the SC_URIBL_SURBL trigger, after the earlier priority rules are ran. However, the remaining priorities are then ran, and if meeting critera, the RAZOR, DCC, and PYZOR rules run and then the SC_URIBL_HASH rule would trigger. Thus giving a total score of 200 + the scores for the URIBL/SURBL scores that hit and if included the Razor, DCC, and Pyzor scores as well. I was thinking after the SC_URIBL_SURBL was triggered remaining rules would not run, and the spam classification would take precendence. Am I overlooking the obvious, have I misunderstood how the SC should work, is it something with the rpm that was released by rpmforge? Any thoughts or insight would be appreciated. SA is, rather fortunately, circumventing what you're trying to do because of how DNS is handled internally. DO NOT try to split up the priority of DNS based tests. Priority and shortcircuiting is intended to be used on *fast* rules, not slow ones. If you were successful, you would make the performance of SpamAssassin absurdly slow by serializing DNS queries. *OUCH*. SA normally runs these in parallel, and running them in serial would very seriously impact performance. Currently, all DNS based tests run at their priority, but that only launches the DNS queries. All the results are gathered together at HARVEST_DNSBL_PRIORITY, which is currently set to 500. None of the rules will actually trigger until this point.
Re: possible idea for backscatter problem
.rp wrote: One of the users (actually the boss) had the email address harvested and we got clobbered by backscatter. Looking at the emails of the various 'unable to deliver' type messages, I saw what these could be filtered on, but don't know how to write up and implement the rule outside of procmail. I don't want to use procmail for this since it I think it would be an expensive routine for procmail to run. In the body of the 'unable to deliver' message, the original message is quoted. One of the lines quoted is the Message-ID: header from the original. The format of this line is always wrong as it does not contain the FQDN that our server appends to the end of the hash number , following the '@' symbol . So, need a rule that would parse the Message-ID: in the body (or attachment) and not header, and look for the @FQDN Is this rule already out in the wild? (note: your To: was the bogofilter list, but this appeared on spamassassin-users as well.. It looks like you bcc'ed the SA list. Anyway, I'm answering on the SA list because that's where I picked up the message from) Not that I know of, but it would be fairly quick as a spamassassin rule. You'd likely need a meta of some sort. Theoretically, something like this should work. I'm leveraging some of the stock ruleset here, by reusing BOUNCE_MESSAGE to detect if the message really is a bounce, make sure it is in your ruleset. -- body __BOUNCE_HASMSGID /Message-ID:/ body __BOUNCE_MINE /Message-ID:.{1,[EMAIL PROTECTED]/ meta BOUNCE_NOTMINE (BOUNCE_MESSAGE __BOUNCE_HASMSGID !__BOUNCE_MINE) score BOUNCE_NOTMINE 0.1 describe BOUNCE_NOTMINE Appears to be a bounce with a message without a valid message ID. -- Modify the example.com part to suit your needs. Be sure to run spamassassin --lint after adding it. Note: I've intentionally set to score to 0.1, as this rule isn't tested. It theoretically should do its job, but make sure it works properly before you increase the score.
Multiple X-Envelope-From and SPF
At the MTA( postfix) I am inserting X-Envelope-From: If The mail had already a X-Envelope-From before landing at my MTA then There would be multiple lines of these Then SA refuses to do SPF for these messages , and I can see in my debug logs - [18469] dbg: message: X-Envelope-From header found after 1 or more Received lines, cannot trust envelope-from [18469] dbg: spf: relayed through one or more trusted relays, cannot use header-based Envelope-From, skipping -- How do I avoid this situation. Thanks Ram
Re: Experimental - use my server for your high fake MX record
Kevin W. Gagel writes: - Original Message - Marc Perkel wrote: Looking for a few volunteers who want to reduce their spambot spam and at the same time help me track spambots for my black list. This is free and mutual benefit. I (junkemailfilter.com) want to be your highest numbered fake MX record. Here's how you would configure your domain: A generous offer and an admirable effort. But if you think I or my clients are going to route mail to your servers you are mistaken. Even if I knew you personally, I don't think ethics or common sense would allow me to do so. DAve Personally I use the honeypot project. I recomend it. See: http://www.projecthoneypot.org For info. btw, if you have spare spamtrap *domains* -- not just /etc/aliases forwards -- we'd love to get a couple pointed at the SpamAssassin spamtraps... --j.
Re: possible idea for backscatter problem
Matt Kettler writes: .rp wrote: One of the users (actually the boss) had the email address harvested and we got clobbered by backscatter. Looking at the emails of the various 'unable to deliver' type messages, I saw what these could be filtered on, but don't know how to write up and implement the rule outside of procmail. I don't want to use procmail for this since it I think it would be an expensive routine for procmail to run. In the body of the 'unable to deliver' message, the original message is quoted. One of the lines quoted is the Message-ID: header from the original. The format of this line is always wrong as it does not contain the FQDN that our server appends to the end of the hash number , following the '@' symbol . So, need a rule that would parse the Message-ID: in the body (or attachment) and not header, and look for the @FQDN Is this rule already out in the wild? (note: your To: was the bogofilter list, but this appeared on spamassassin-users as well.. It looks like you bcc'ed the SA list. Anyway, I'm answering on the SA list because that's where I picked up the message from) Not that I know of, but it would be fairly quick as a spamassassin rule. You'd likely need a meta of some sort. Theoretically, something like this should work. I'm leveraging some of the stock ruleset here, by reusing BOUNCE_MESSAGE to detect if the message really is a bounce, make sure it is in your ruleset. Actually, that's overkill -- BOUNCE_MESSAGE _already_ does this. the VBounce plugin is intended to catch backscatter -- bounces in response to mail you didn't send -- so it'll ignore bounces in response to mail you _did_ send, by parsing the bounced message's Received: headers and looking for the mailserver's name in there. See the FAQ for more info... --j.
Re: Shortcircuit Question
Matt Kettler writes: Clayton Keller wrote: I have been reading throught the Shortcircuit manpage as well as some articles within the Wiki, and the manner in which I see it performing within our install does not seem to coincide with how I am reading and presumably understanding it to work. First off, we are using SpamAssassin 3.2.4 provided by the rpmforge batch of rpm's. I have about a dozen priorities specified mainly handling URIBL, SURBL, as well as DCC, Razor, Pyzor and Bayes. What I am seeing is that although the first shortcircuit rule hits and scores appropriately. Subsequent short circuit rules will continue to fire. The scores themselves are then totaled along with the original scores for the rules. My understanding of how the shortcircuit should work is that once a shortcircuit is triggered any subsequent rules should be bypassed and the message wither classified as spam/ham or if set to on, it would use the current score specified for the rule. As a for instance: I have the following: priority URIBL_BLACK-500 priority URIBL_JP_SURBL -498 priority URIBL_SC_SURBL -488 priority URIBL_OB_SURBL -487 priority SC_URIBL_SURBL -480 priority SC_URIBL_SBL -479 priority RAZOR2_CHECK -450 priority DCC_CHECK -449 priority PYZOR_CHECK-448 priority SC_URIBL_HASH -440 meta SC_URIBL_SURBL(URIBL_BLACK (URIBL_SC_SURBL || URIBL_JP_SURBL || URIBL_OB_SURBL)) meta SC_URIBL_SBL((URIBL_BLACK || URIBL_SC_SURBL || URIBL_JP_SURBL || URIBL_OB_SURBL) URIBL_SBL) meta SC_URIBL_HASH((URIBL_BLACK || URIBL_SC_SURBL || URIBL_JP_SURBL || URIBL_OB_SURBL) (RAZOR2_CHECK || DCC_CHECK || PYZOR_CHECK)) meta SC_URIBL_SBL ((URIBL_BLACK || URIBL_SC_SURBL || URIBL_JP_SURBL || URIBL_OB_SURBL) URIBL_SBL) shortcircuit SC_URIBL_SURBL spam shortcircuit SC_URIBL_SBL spam shortcircuit SC_URIBL_HASH spam score SC_URIBL_SURBL100.00 score SC_URIBL_HASH 100.00 score SC_URIBL_SBL 100.00 I do not have a recent debug to show but I can say that from the debug I do see the SC_URIBL_SURBL trigger, after the earlier priority rules are ran. However, the remaining priorities are then ran, and if meeting critera, the RAZOR, DCC, and PYZOR rules run and then the SC_URIBL_HASH rule would trigger. Thus giving a total score of 200 + the scores for the URIBL/SURBL scores that hit and if included the Razor, DCC, and Pyzor scores as well. I was thinking after the SC_URIBL_SURBL was triggered remaining rules would not run, and the spam classification would take precendence. Am I overlooking the obvious, have I misunderstood how the SC should work, is it something with the rpm that was released by rpmforge? Any thoughts or insight would be appreciated. SA is, rather fortunately, circumventing what you're trying to do because of how DNS is handled internally. DO NOT try to split up the priority of DNS based tests. Priority and shortcircuiting is intended to be used on *fast* rules, not slow ones. If you were successful, you would make the performance of SpamAssassin absurdly slow by serializing DNS queries. *OUCH*. SA normally runs these in parallel, and running them in serial would very seriously impact performance. Currently, all DNS based tests run at their priority, but that only launches the DNS queries. All the results are gathered together at HARVEST_DNSBL_PRIORITY, which is currently set to 500. None of the rules will actually trigger until this point. actually Matt, you're wrong ;) if some of the network rules are at a higher priority than others, and are used in shortcircuit rules, SpamAssassin 3.2.x will indeed sleep until the results of those rules arrive. The idea is that, if you have the memory to support that degree of concurrency, you can make a local policy decision to do that, instead of doing the lookups at the MTA level which does effectively the same thing. This wait is logged, so you can spot it with --debug on. --j.
Re: possible idea for backscatter problem
Henrik Krohns writes: On Thu, May 08, 2008 at 09:35:31AM +0100, Justin Mason wrote: the VBounce plugin is intended to catch backscatter -- bounces in response to mail you didn't send -- so it'll ignore bounces in response to mail you _did_ send, by parsing the bounced message's Received: headers and looking for the mailserver's name in there. I've been trying it for myself. One thing I don't like is that all null-sender mail is assumed to be bounces. It creates many FPs. Yes. What kind of null-sender mail from machines that are not in whitelist_bounce_relays do you get? Also check https://issues.apache.org/SpamAssassin/show_bug.cgi?id=5900 saw that... --j.
Re: possible idea for backscatter problem
Matt Kettler writes: Justin Mason wrote: Matt Kettler writes: .rp wrote: One of the users (actually the boss) had the email address harvested and we got clobbered by backscatter. Looking at the emails of the various 'unable to deliver' type messages, I saw what these could be filtered on, but don't know how to write up and implement the rule outside of procmail. I don't want to use procmail for this since it I think it would be an expensive routine for procmail to run. In the body of the 'unable to deliver' message, the original message is quoted. One of the lines quoted is the Message-ID: header from the original. The format of this line is always wrong as it does not contain the FQDN that our server appends to the end of the hash number , following the '@' symbol . So, need a rule that would parse the Message-ID: in the body (or attachment) and not header, and look for the @FQDN Is this rule already out in the wild? (note: your To: was the bogofilter list, but this appeared on spamassassin-users as well.. It looks like you bcc'ed the SA list. Anyway, I'm answering on the SA list because that's where I picked up the message from) Not that I know of, but it would be fairly quick as a spamassassin rule. You'd likely need a meta of some sort. Theoretically, something like this should work. I'm leveraging some of the stock ruleset here, by reusing BOUNCE_MESSAGE to detect if the message really is a bounce, make sure it is in your ruleset. Actually, that's overkill -- BOUNCE_MESSAGE _already_ does this. Whoops.. good point. I didn't read the code, I just saw the name and assumed it did just what it says, and nothing more. So, really all .rp needs to do is enable the vbounce plugin (which is loaded by default ) Yep. to enable it, just set whitelist_bounce_relays in the configuration or user prefs. --j.
Re: possible idea for backscatter problem
Justin Mason wrote: Matt Kettler writes: .rp wrote: One of the users (actually the boss) had the email address harvested and we got clobbered by backscatter. Looking at the emails of the various 'unable to deliver' type messages, I saw what these could be filtered on, but don't know how to write up and implement the rule outside of procmail. I don't want to use procmail for this since it I think it would be an expensive routine for procmail to run. In the body of the 'unable to deliver' message, the original message is quoted. One of the lines quoted is the Message-ID: header from the original. The format of this line is always wrong as it does not contain the FQDN that our server appends to the end of the hash number , following the '@' symbol . So, need a rule that would parse the Message-ID: in the body (or attachment) and not header, and look for the @FQDN Is this rule already out in the wild? (note: your To: was the bogofilter list, but this appeared on spamassassin-users as well.. It looks like you bcc'ed the SA list. Anyway, I'm answering on the SA list because that's where I picked up the message from) Not that I know of, but it would be fairly quick as a spamassassin rule. You'd likely need a meta of some sort. Theoretically, something like this should work. I'm leveraging some of the stock ruleset here, by reusing BOUNCE_MESSAGE to detect if the message really is a bounce, make sure it is in your ruleset. Actually, that's overkill -- BOUNCE_MESSAGE _already_ does this. Whoops.. good point. I didn't read the code, I just saw the name and assumed it did just what it says, and nothing more. So, really all .rp needs to do is enable the vbounce plugin (which is loaded by default )
Re: possible idea for backscatter problem
On Thu, May 08, 2008 at 10:03:28AM +0100, Justin Mason wrote: Henrik Krohns writes: On Thu, May 08, 2008 at 09:35:31AM +0100, Justin Mason wrote: the VBounce plugin is intended to catch backscatter -- bounces in response to mail you didn't send -- so it'll ignore bounces in response to mail you _did_ send, by parsing the bounced message's Received: headers and looking for the mailserver's name in there. I've been trying it for myself. One thing I don't like is that all null-sender mail is assumed to be bounces. It creates many FPs. Yes. What kind of null-sender mail from machines that are not in whitelist_bounce_relays do you get? I would assume this is common knowledge. :-) - Mass mailings of all sorts, mailing lists, news - Order confirmations - Some very legimate mails from people using lotus notes, hotmail - etc, etc.. You can just imagine a system/administrator thinking that it's wise to send anything that doesn't require a reply as null. It's very common. Do you want a bug opened?
Re: possible idea for backscatter problem
On Thu, May 08, 2008 at 11:35:30AM +0100, Justin Mason wrote: Not in my experience! I haven't seen anything that isn't a bounce message, an out-of-office notification, auto-replies, or other stuff targeted by the VBounce ruleset. certainly not transactional mail. as far as I can tell, it's *not* that common. I can give you dozen sample emails from last week right now. Including handwritten mails from people, that come from strange systems (Lotus, Live Mail, Horde/IMP..) sending as null. There's a lot more, I don't have time to check every mail. It *is* common. And this server only passes 15k mails a day. You are only talking about your own mailfeed, right? I can't believe you would say this otherwise. Do you want a bug opened? Nah, that's ok ;) Well, frankly I have too much on my plate to start working on this too. So I'll let it sleep for now. :)
Re: Experimental - use my server for your high fake MX record
IOn Wed, 2008-05-07 at 08:50 -0700, Marc Perkel wrote: Looking for a few volunteers who want to reduce their spambot spam and at the same time help me track spambots for my black list. This is free and mutual benefit. I (junkemailfilter.com) want to be your highest numbered fake MX record. Here's how you would configure your domain: mail.yourdomain.com MX 10 tarbaby.junkemailfilter.com MX 20 I will never actually receive your email. The recipient all always get a 451 error just after the DATA command. So if your servers are down you won't lose anything. A 451 error is a I'm not ready, come back later error. This will help you reduce your spambot spam generally by half. ... I use fake MX as well. But even if my lower MXes are perfectly available. I have seen quiet a lot of legitimate traffic coming on my fake MX and get turned down with a tempfail. So If you are populating blacklists based on this data , better be careful. (I'm sure you would have seen that yourself) Anyway I think moving an MX record to a third party with no bussiness contact would not be possible for anyone Thanks Ram
Re: Experimental - use my server for your high fake MX record
On Thu, 2008-05-08 at 09:33 +0100, Justin Mason wrote: Kevin W. Gagel writes: - Original Message - Marc Perkel wrote: Looking for a few volunteers who want to reduce their spambot spam and at the same time help me track spambots for my black list. This is free and mutual benefit. I (junkemailfilter.com) want to be your highest numbered fake MX record. Here's how you would configure your domain: A generous offer and an admirable effort. But if you think I or my clients are going to route mail to your servers you are mistaken. Even if I knew you personally, I don't think ethics or common sense would allow me to do so. DAve Personally I use the honeypot project. I recomend it. See: http://www.projecthoneypot.org For info. btw, if you have spare spamtrap *domains* -- not just /etc/aliases forwards -- we'd love to get a couple pointed at the SpamAssassin spamtraps... What should the MX'es be pointed to ? Also what are tricks of getting mails on your spamtrap ? --j.
Re: possible idea for backscatter problem
Henrik K schrieb: On Thu, May 08, 2008 at 11:35:30AM +0100, Justin Mason wrote: Not in my experience! I haven't seen anything that isn't a bounce message, an out-of-office notification, auto-replies, or other stuff targeted by the VBounce ruleset. certainly not transactional mail. as far as I can tell, it's *not* that common. I can give you dozen sample emails from last week right now. Including handwritten mails from people, that come from strange systems (Lotus, Live Mail, Horde/IMP..) sending as null. There's a lot more, I don't have time to check every mail. It *is* common. And this server only passes 15k mails a day. You are only talking about your own mailfeed, right? I can't believe you would say this otherwise. Do you want a bug opened? Nah, that's ok ;) Well, frankly I have too much on my plate to start working on this too. So I'll let it sleep for now. :) After my fix in the 20_vbounce.cf I had also several FPs, the most significant the SA-bugzilla account confirmation mail. :-| Reason for this was the simple match on 'daemon' in the From: -line regexp. Is somebody already working on an enhancement of the ruleset? Otherwise, if you can provide some of your FP's (and maybe some real backscatter) I can imagine to put them together with mine and try to improve the rules a little bit in the next few weeks. BTW: Also for me 'null senders' are not common - never had problems with this, except UBE. Correctly configured servers/clients should not produce such mails, IMHO. Robert
Re: possible idea for backscatter problem
On Thu, 8 May 2008, Justin Mason wrote: Matt Kettler writes: .rp wrote: So, need a rule that would parse the Message-ID: in the body (or attachment) and not header, and look for the @FQDN Is this rule already out in the wild? You'd likely need a meta of some sort. Theoretically, something like this should work. I'm leveraging some of the stock ruleset here, by reusing BOUNCE_MESSAGE to detect if the message really is a bounce, make sure it is in your ruleset. Actually, that's overkill -- BOUNCE_MESSAGE _already_ does this. the VBounce plugin is intended to catch backscatter -- bounces in response to mail you didn't send -- so it'll ignore bounces in response to mail you _did_ send, by parsing the bounced message's Received: headers and looking for the mailserver's name in there. Pardon my ignorance if I'm just not understanding this right, but my impression is that there's a possibility that messages marked the BOUNCE_MESSAGE could be legitimate bounces (just not bounces generated by a whitelisted server). Certainly I've read plenty of people on this list advise against raising the vbounce rule scores as a way to combat this new wave of seemingly intentional bounce spam, but rather to filter all the bounces off to a separate folder. But, doesn't the existence of a Message ID in the text of the bounce that has a bogus or malformed email address provide a stronger indication that this is not a valid bounce? In which case, such email could be scored higher, rather than just sent to a bounce folder, which is really what many of us would rather be doing with these messages? -- Public key #7BBC68D9 at| Shane Williams http://pgp.mit.edu/| System Admin - UT iSchool =--+--- All syllogisms contain three lines | [EMAIL PROTECTED] Therefore this is not a syllogism | www.ischool.utexas.edu/~shanew
Re: possible idea for backscatter problem
On Thu, May 08, 2008 at 03:11:59PM +0200, Robert Müller wrote: BTW: Also for me 'null senders' are not common - never had problems with this, except UBE. Have you even looked at your traffic archives, if you keep one? How do you know there isn't any problems if someone doesn't realize to report it? I have concrete evidence. Correctly configured servers/clients should not produce such mails, IMHO. That's just absurd. I could just as well say: Correctly configured servers don't create backscatter.. Yet we have the problem. In case of VBounce, chances of FPs are even less acceptable. You are supposed to reject or discard backscatter, I see no point in just tagging it. So discarding is bad, but rejecting most likely means that sending party doesn't get any notification of failure either. Currently VBounce is only useful as add little score and hope URIBL and other checks match the returned body. Hopefully someone has time to fix the too generic rules. We should only match sure bounces.
triplets.txt
Hi, could someone kindly tell me what the file triplets.txt is used for, and if I need to have it in my rules directory or not? Cheers, Jeremy
Re: Shortcircuit Question
Justin Mason wrote: Matt Kettler writes: Clayton Keller wrote: I have been reading throught the Shortcircuit manpage as well as some articles within the Wiki, and the manner in which I see it performing within our install does not seem to coincide with how I am reading and presumably understanding it to work. First off, we are using SpamAssassin 3.2.4 provided by the rpmforge batch of rpm's. I have about a dozen priorities specified mainly handling URIBL, SURBL, as well as DCC, Razor, Pyzor and Bayes. What I am seeing is that although the first shortcircuit rule hits and scores appropriately. Subsequent short circuit rules will continue to fire. The scores themselves are then totaled along with the original scores for the rules. My understanding of how the shortcircuit should work is that once a shortcircuit is triggered any subsequent rules should be bypassed and the message wither classified as spam/ham or if set to on, it would use the current score specified for the rule. As a for instance: I have the following: priority URIBL_BLACK-500 priority URIBL_JP_SURBL -498 priority URIBL_SC_SURBL -488 priority URIBL_OB_SURBL -487 priority SC_URIBL_SURBL -480 priority SC_URIBL_SBL -479 priority RAZOR2_CHECK -450 priority DCC_CHECK -449 priority PYZOR_CHECK-448 priority SC_URIBL_HASH -440 meta SC_URIBL_SURBL(URIBL_BLACK (URIBL_SC_SURBL || URIBL_JP_SURBL || URIBL_OB_SURBL)) meta SC_URIBL_SBL((URIBL_BLACK || URIBL_SC_SURBL || URIBL_JP_SURBL || URIBL_OB_SURBL) URIBL_SBL) meta SC_URIBL_HASH((URIBL_BLACK || URIBL_SC_SURBL || URIBL_JP_SURBL || URIBL_OB_SURBL) (RAZOR2_CHECK || DCC_CHECK || PYZOR_CHECK)) meta SC_URIBL_SBL ((URIBL_BLACK || URIBL_SC_SURBL || URIBL_JP_SURBL || URIBL_OB_SURBL) URIBL_SBL) shortcircuit SC_URIBL_SURBL spam shortcircuit SC_URIBL_SBL spam shortcircuit SC_URIBL_HASH spam score SC_URIBL_SURBL100.00 score SC_URIBL_HASH 100.00 score SC_URIBL_SBL 100.00 I do not have a recent debug to show but I can say that from the debug I do see the SC_URIBL_SURBL trigger, after the earlier priority rules are ran. However, the remaining priorities are then ran, and if meeting critera, the RAZOR, DCC, and PYZOR rules run and then the SC_URIBL_HASH rule would trigger. Thus giving a total score of 200 + the scores for the URIBL/SURBL scores that hit and if included the Razor, DCC, and Pyzor scores as well. I was thinking after the SC_URIBL_SURBL was triggered remaining rules would not run, and the spam classification would take precendence. Am I overlooking the obvious, have I misunderstood how the SC should work, is it something with the rpm that was released by rpmforge? Any thoughts or insight would be appreciated. SA is, rather fortunately, circumventing what you're trying to do because of how DNS is handled internally. DO NOT try to split up the priority of DNS based tests. Priority and shortcircuiting is intended to be used on *fast* rules, not slow ones. If you were successful, you would make the performance of SpamAssassin absurdly slow by serializing DNS queries. *OUCH*. SA normally runs these in parallel, and running them in serial would very seriously impact performance. Currently, all DNS based tests run at their priority, but that only launches the DNS queries. All the results are gathered together at HARVEST_DNSBL_PRIORITY, which is currently set to 500. None of the rules will actually trigger until this point. actually Matt, you're wrong ;) if some of the network rules are at a higher priority than others, and are used in shortcircuit rules, SpamAssassin 3.2.x will indeed sleep until the results of those rules arrive. The idea is that, if you have the memory to support that degree of concurrency, you can make a local policy decision to do that, instead of doing the lookups at the MTA level which does effectively the same thing. This wait is logged, so you can spot it with --debug on. --j. With that said Justin, is the behavior I am seeing correct? Even though the first prioritized shortcircuit rule hits, and I see that in the debug log, shouldn't it be bypassing the remaining rules rather than continuing to process until all the shortcircuit priorities have ran? From reading the initial bug when this was originally featured, along with the man page, as well as a wiki post with an example by you, that is how I understood it to function. Clay
Re: Shortcircuit Question
Clayton Keller writes: Justin Mason wrote: Matt Kettler writes: Clayton Keller wrote: I have been reading throught the Shortcircuit manpage as well as some articles within the Wiki, and the manner in which I see it performing within our install does not seem to coincide with how I am reading and presumably understanding it to work. First off, we are using SpamAssassin 3.2.4 provided by the rpmforge batch of rpm's. I have about a dozen priorities specified mainly handling URIBL, SURBL, as well as DCC, Razor, Pyzor and Bayes. What I am seeing is that although the first shortcircuit rule hits and scores appropriately. Subsequent short circuit rules will continue to fire. The scores themselves are then totaled along with the original scores for the rules. My understanding of how the shortcircuit should work is that once a shortcircuit is triggered any subsequent rules should be bypassed and the message wither classified as spam/ham or if set to on, it would use the current score specified for the rule. As a for instance: I have the following: priority URIBL_BLACK-500 priority URIBL_JP_SURBL -498 priority URIBL_SC_SURBL -488 priority URIBL_OB_SURBL -487 priority SC_URIBL_SURBL -480 priority SC_URIBL_SBL -479 priority RAZOR2_CHECK -450 priority DCC_CHECK -449 priority PYZOR_CHECK-448 priority SC_URIBL_HASH -440 meta SC_URIBL_SURBL(URIBL_BLACK (URIBL_SC_SURBL || URIBL_JP_SURBL || URIBL_OB_SURBL)) meta SC_URIBL_SBL((URIBL_BLACK || URIBL_SC_SURBL || URIBL_JP_SURBL || URIBL_OB_SURBL) URIBL_SBL) meta SC_URIBL_HASH((URIBL_BLACK || URIBL_SC_SURBL || URIBL_JP_SURBL || URIBL_OB_SURBL) (RAZOR2_CHECK || DCC_CHECK || PYZOR_CHECK)) meta SC_URIBL_SBL ((URIBL_BLACK || URIBL_SC_SURBL || URIBL_JP_SURBL || URIBL_OB_SURBL) URIBL_SBL) shortcircuit SC_URIBL_SURBL spam shortcircuit SC_URIBL_SBL spam shortcircuit SC_URIBL_HASH spam score SC_URIBL_SURBL100.00 score SC_URIBL_HASH 100.00 score SC_URIBL_SBL 100.00 I do not have a recent debug to show but I can say that from the debug I do see the SC_URIBL_SURBL trigger, after the earlier priority rules are ran. However, the remaining priorities are then ran, and if meeting critera, the RAZOR, DCC, and PYZOR rules run and then the SC_URIBL_HASH rule would trigger. Thus giving a total score of 200 + the scores for the URIBL/SURBL scores that hit and if included the Razor, DCC, and Pyzor scores as well. I was thinking after the SC_URIBL_SURBL was triggered remaining rules would not run, and the spam classification would take precendence. Am I overlooking the obvious, have I misunderstood how the SC should work, is it something with the rpm that was released by rpmforge? Any thoughts or insight would be appreciated. SA is, rather fortunately, circumventing what you're trying to do because of how DNS is handled internally. DO NOT try to split up the priority of DNS based tests. Priority and shortcircuiting is intended to be used on *fast* rules, not slow ones. If you were successful, you would make the performance of SpamAssassin absurdly slow by serializing DNS queries. *OUCH*. SA normally runs these in parallel, and running them in serial would very seriously impact performance. Currently, all DNS based tests run at their priority, but that only launches the DNS queries. All the results are gathered together at HARVEST_DNSBL_PRIORITY, which is currently set to 500. None of the rules will actually trigger until this point. actually Matt, you're wrong ;) if some of the network rules are at a higher priority than others, and are used in shortcircuit rules, SpamAssassin 3.2.x will indeed sleep until the results of those rules arrive. The idea is that, if you have the memory to support that degree of concurrency, you can make a local policy decision to do that, instead of doing the lookups at the MTA level which does effectively the same thing. This wait is logged, so you can spot it with --debug on. --j. With that said Justin, is the behavior I am seeing correct? Even though the first prioritized shortcircuit rule hits, and I see that in the debug log, shouldn't it be bypassing the remaining rules rather than continuing to process until all the shortcircuit priorities have ran? From reading the initial bug when this was originally featured, along with the man page, as well as a wiki post with an example by you, that is how I understood it to function. actually, no, it sounds like a bug.could you open a
Re: possible idea for backscatter problem
Henrik K schrieb: On Thu, May 08, 2008 at 03:11:59PM +0200, Robert Müller wrote: BTW: Also for me 'null senders' are not common - never had problems with this, except UBE. Have you even looked at your traffic archives, if you keep one? How do you know there isn't any problems if someone doesn't realize to report it? I would have realized, if I had this problem, for sure. I have concrete evidence. I believe you, no problem. Correctly configured servers/clients should not produce such mails, IMHO. That's just absurd. I could just as well say: Correctly configured servers don't create backscatter.. Yet we have the problem. I think you misunderstood me. Such mails are - in my experience - not the majority, and therefore this issue is also for me not common and not known as a big problem. Nothing more I wanted to say. Now I hear different from you - fine, something learned. In case of VBounce, chances of FPs are even less acceptable. You are supposed to reject or discard backscatter, I see no point in just tagging it. So discarding is bad, but rejecting most likely means that sending party doesn't get any notification of failure either. Currently VBounce is only useful as add little score and hope URIBL and other checks match the returned body. ACK. Because this doesn't work in most cases, and the currently (for me) unknown risk of FPs, I actually I sort them in a different folder. Hopefully someone has time to fix the too generic rules. We should only match sure bounces. Sure is as always relative, but that was the goal, yes.
Re: possible idea for backscatter problem
In case of VBounce, chances of FPs are even less acceptable. You are supposed to reject or discard backscatter who says? It seems perfectly fine to me to tag vbounce-filtered mail. In mail filtering, there will always be FPs. --j.
Re: IE Parse bug olso in SpamAssassin ?
On Thu, 8 May 2008, Benny Pedersen wrote: On Thu, May 8, 2008 05:00, Joseph Brennan wrote: Should we just call http://{; bad, and not bother checking the uri? i belive there is parts in sa that parse the same way as ie and that could be used by spammers to hide there domains in multilvel obfu Why worry about where the URI is trying to point if it's so obviously obfuscated? -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ [EMAIL PROTECTED]FALaholic #11174 pgpk -a [EMAIL PROTECTED] key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- News flash: Lowest Common Denominator down 50 points --- Today: the 63rd anniversary of VE day
Re: possible idea for backscatter problem
On Thu, May 08, 2008 at 04:20:42PM +0100, Justin Mason wrote: In case of VBounce, chances of FPs are even less acceptable. You are supposed to reject or discard backscatter who says? It seems perfectly fine to me to tag vbounce-filtered mail. In mail filtering, there will always be FPs. Come on, we all know ISPs are not supposed to drop anything and that there are users that insist on receiving all mail to be sure. This has nothing to do with VBounce (which is constantly touted as being backscatter stopping tool). In not above scenarios, why would anyone want to receive backscatter? VBounce is currently a FP-prone match or not match tool for that purpose. Either you want backscatter or you don't. For other things it's perfectly fine to tag maybe spam and discard sure spam. Now you can just hope that VBounce gets it over the discard limit.
Re: IE Parse bug olso in SpamAssassin ?
On Thu, May 8, 2008 17:29, John Hardin wrote: Why worry about where the URI is trying to point if it's so obviously obfuscated? to get more data to bayes Benny Pedersen Need more webspace ? http://www.servage.net/?coupon=cust37098
Re: Experimental - use my server for your high fake MX record
ram wrote: IOn Wed, 2008-05-07 at 08:50 -0700, Marc Perkel wrote: Looking for a few volunteers who want to reduce their spambot spam and at the same time help me track spambots for my black list. This is free and mutual benefit. I (junkemailfilter.com) want to be your highest numbered fake MX record. Here's how you would configure your domain: mail.yourdomain.com MX 10 tarbaby.junkemailfilter.com MX 20 I will never actually receive your email. The recipient all always get a 451 error just after the DATA command. So if your servers are down you won't lose anything. A 451 error is a I'm not ready, come back later error. This will help you reduce your spambot spam generally by half. ... I use fake MX as well. But even if my lower MXes are perfectly available. I have seen quiet a lot of legitimate traffic coming on my fake MX and get turned down with a tempfail. So If you are populating blacklists based on this data , better be careful. (I'm sure you would have seen that yourself) Anyway I think moving an MX record to a third party with no bussiness contact would not be possible for anyone Thanks Ram Hi Ram, Being a high numbered MX in itself doesn't get you listed on this new server I set up. It's just a prequalifier of what I want to look at. In order to get listed they also have to fail to send a QUIT after the 451 error and they have to commit some other significant sins. I'm looking at a number of things in the helo, the sender, the recipient, rDNS, etc. What I'm doing isn't going to catch as high of a percentage as I would if I were the official spam filtering host for the domain because I'm not running all my tests on it. I'm cutting them off before the data is sent. I'm not even seeing the message headers. However, I do think that I'll catch a lot of what I'm looking for and that's virus infected spambots. That's the only think I'm targeting here and I think I can distinguish them well enough that I can catch most all the spambot traffic with no false positives on legit email. I'm hoping for 50% accuracy of catching spambots on the first attempt. To participate all you have to do is set your highest numbered MX to point to: tarbaby.junkemailfilter.com Several people have asked me how I'm doing this and can they have my code to do it themselves. My situation is unique enough that it just won't work very easilly any place else and it's definitely not clean enough for just anyone to install. But I'll try to describe it here. First to do what I'm doing you have to be using EXIM. If you aren't running exim then you just can't do it. In fact, with all due respect, I can't see how anyone can do spam filtering and not use exim as their MTA. Exim has a feature where you can execute code based on how the connection is closed. It have a NOTQUIT acl and you can look at if the connection timed out and a number of other things that caused the connection to close without issuing a quit. Before the 451 error I store information in variables that I can retrieve in the notquit acl and based on that information I can send messages to another server that accumulating information from all my servers. This server is basically running stats on a one minute cycle to determine what data goes into my various white/black/yellow lists and that feeds my 4 rbldnsd servers which are updated every minute. Blacklist data is stored for 5 days and then it expired. Every 6 hours the oldest log file is deleted and everything is moved down a slot and a new log file created. Thus if someone fixed the virus then they will eventually be cleaned off the list. Users also have a web form where they can get themselves removed if there is a false positive. The list isn't perfect but it is my goal to have no false positives. Unlike some lists who think that some sloppy admins deserve to be blacklisted my attitude is if the listing is wrong it's my fault and I want to fix it. And unlike many other blacklisating services I focuse more on my white listing and yellow listing and use that information to reduce the chance of false positives in my blacklists. I also see the value of being as cooperative with others because although I'm good at coming up with new ideas, other are better at taking the ideas and doing it right. So many times I'll put an idea out there and someone else will do it better and I get to run their better version. I am of the opinion that 100% of spambot spam can be stopped because I'm doing it.I want to try to expand on that and get data from other sources and see if I can't help others make some progress too. Hope this is helpful.
Re: IE Parse bug olso in SpamAssassin ?
On Thu, 8 May 2008, Benny Pedersen wrote: On Thu, May 8, 2008 17:29, John Hardin wrote: Why worry about where the URI is trying to point if it's so obviously obfuscated? to get more data to bayes Bayes isn't going to parse a URI as a URI anyway, is it? It just tokenizes the message. Bayes will pick up the domain name string if it's delimited by {} as readily as it will if it's delimited by //. To clarify: why bother trying to deobfuscate the URI and figure out what domain it's really pointing at, so that domain can be checked against URIBL lists, if the form of the obfuscation is obvious and not seen in legitimate emails? Why not just give that obfuscation four or five points and be done with it? If that formatting *was* seen in legitimate emails, then I would say that it's important the URI parsers be aware of it. Can you provide any pointers to documentation of that formatting? I didn't find any in a quick gargle. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ [EMAIL PROTECTED]FALaholic #11174 pgpk -a [EMAIL PROTECTED] key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- The real opiate of the masses isn't religion; it's the belief that somewhere there is a benefit that can be delivered without a corresponding cost. -- Tom of Radio Free NJ --- Today: the 63rd anniversary of VE day
Re: Experimental - use my server for your high fake MX record
On Thu, 8 May 2008, Marc Perkel wrote: To participate all you have to do is set your highest numbered MX to point to: tarbaby.junkemailfilter.com Several people have asked me how I'm doing this and can they have my code to do it themselves. My situation is unique enough that it just won't work very easilly any place else and it's definitely not clean enough for just anyone to install. You should make an effort to clean it up so that others *can* install it as a standalone daemon, as I suggested. Why? How long will it be before the spambots explicitly refuse to contact your honeypot if it is listed as an MX for the domain they're attacking? -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ [EMAIL PROTECTED]FALaholic #11174 pgpk -a [EMAIL PROTECTED] key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- The real opiate of the masses isn't religion; it's the belief that somewhere there is a benefit that can be delivered without a corresponding cost. -- Tom of Radio Free NJ --- Today: the 63rd anniversary of VE day
Re: Experimental - use my server for your high fake MX record
John Hardin wrote: On Thu, 8 May 2008, Marc Perkel wrote: To participate all you have to do is set your highest numbered MX to point to: tarbaby.junkemailfilter.com Several people have asked me how I'm doing this and can they have my code to do it themselves. My situation is unique enough that it just won't work very easilly any place else and it's definitely not clean enough for just anyone to install. You should make an effort to clean it up so that others *can* install it as a standalone daemon, as I suggested. Why? How long will it be before the spambots explicitly refuse to contact your honeypot if it is listed as an MX for the domain they're attacking? Good point. I suppose that if this grows I can point to my traps using other hostnames. I can also set up traps on virtual server under OpenVZ so spammers won't know what IP ranges to avoid.
Re: Shortcircuit Question
Justin Mason wrote: Clayton Keller writes: Justin Mason wrote: Matt Kettler writes: Clayton Keller wrote: I have been reading throught the Shortcircuit manpage as well as some articles within the Wiki, and the manner in which I see it performing within our install does not seem to coincide with how I am reading and presumably understanding it to work. First off, we are using SpamAssassin 3.2.4 provided by the rpmforge batch of rpm's. I have about a dozen priorities specified mainly handling URIBL, SURBL, as well as DCC, Razor, Pyzor and Bayes. What I am seeing is that although the first shortcircuit rule hits and scores appropriately. Subsequent short circuit rules will continue to fire. The scores themselves are then totaled along with the original scores for the rules. My understanding of how the shortcircuit should work is that once a shortcircuit is triggered any subsequent rules should be bypassed and the message wither classified as spam/ham or if set to on, it would use the current score specified for the rule. As a for instance: I have the following: priority URIBL_BLACK-500 priority URIBL_JP_SURBL -498 priority URIBL_SC_SURBL -488 priority URIBL_OB_SURBL -487 priority SC_URIBL_SURBL -480 priority SC_URIBL_SBL -479 priority RAZOR2_CHECK -450 priority DCC_CHECK -449 priority PYZOR_CHECK-448 priority SC_URIBL_HASH -440 meta SC_URIBL_SURBL(URIBL_BLACK (URIBL_SC_SURBL || URIBL_JP_SURBL || URIBL_OB_SURBL)) meta SC_URIBL_SBL((URIBL_BLACK || URIBL_SC_SURBL || URIBL_JP_SURBL || URIBL_OB_SURBL) URIBL_SBL) meta SC_URIBL_HASH((URIBL_BLACK || URIBL_SC_SURBL || URIBL_JP_SURBL || URIBL_OB_SURBL) (RAZOR2_CHECK || DCC_CHECK || PYZOR_CHECK)) meta SC_URIBL_SBL ((URIBL_BLACK || URIBL_SC_SURBL || URIBL_JP_SURBL || URIBL_OB_SURBL) URIBL_SBL) shortcircuit SC_URIBL_SURBL spam shortcircuit SC_URIBL_SBL spam shortcircuit SC_URIBL_HASH spam score SC_URIBL_SURBL100.00 score SC_URIBL_HASH 100.00 score SC_URIBL_SBL 100.00 I do not have a recent debug to show but I can say that from the debug I do see the SC_URIBL_SURBL trigger, after the earlier priority rules are ran. However, the remaining priorities are then ran, and if meeting critera, the RAZOR, DCC, and PYZOR rules run and then the SC_URIBL_HASH rule would trigger. Thus giving a total score of 200 + the scores for the URIBL/SURBL scores that hit and if included the Razor, DCC, and Pyzor scores as well. I was thinking after the SC_URIBL_SURBL was triggered remaining rules would not run, and the spam classification would take precendence. Am I overlooking the obvious, have I misunderstood how the SC should work, is it something with the rpm that was released by rpmforge? Any thoughts or insight would be appreciated. SA is, rather fortunately, circumventing what you're trying to do because of how DNS is handled internally. DO NOT try to split up the priority of DNS based tests. Priority and shortcircuiting is intended to be used on *fast* rules, not slow ones. If you were successful, you would make the performance of SpamAssassin absurdly slow by serializing DNS queries. *OUCH*. SA normally runs these in parallel, and running them in serial would very seriously impact performance. Currently, all DNS based tests run at their priority, but that only launches the DNS queries. All the results are gathered together at HARVEST_DNSBL_PRIORITY, which is currently set to 500. None of the rules will actually trigger until this point. actually Matt, you're wrong ;) if some of the network rules are at a higher priority than others, and are used in shortcircuit rules, SpamAssassin 3.2.x will indeed sleep until the results of those rules arrive. The idea is that, if you have the memory to support that degree of concurrency, you can make a local policy decision to do that, instead of doing the lookups at the MTA level which does effectively the same thing. This wait is logged, so you can spot it with --debug on. --j. With that said Justin, is the behavior I am seeing correct? Even though the first prioritized shortcircuit rule hits, and I see that in the debug log, shouldn't it be bypassing the remaining rules rather than continuing to process until all the shortcircuit priorities have ran? From reading the initial bug when this was originally featured, along with the man page, as well as a wiki post with an example by you, that is how I understood it to function. actually, no, it sounds like a bug.could you open a bugzilla with a demonstration config/test message? --j. Just to add, with my previous debug, I can confirm the waiting of the tests to finish as you mentioned from the
Re: Experimental - use my server for your high fake MX record
Well now, if a spambot actually does start recognizing and avoiding his system, doesn't that mean he wins and the spammer loses? John Hardin [EMAIL PROTECTED] 05/08/08 12:11 PM On Thu, 8 May 2008, Marc Perkel wrote: To participate all you have to do is set your highest numbered MX to point to: tarbaby.junkemailfilter.com Several people have asked me how I'm doing this and can they have my code to do it themselves. My situation is unique enough that it just won't work very easilly any place else and it's definitely not clean enough for just anyone to install. You should make an effort to clean it up so that others *can* install it as a standalone daemon, as I suggested. Why? How long will it be before the spambots explicitly refuse to contact your honeypot if it is listed as an MX for the domain they're attacking?
Re: Experimental - use my server for your high fake MX record
Kevin Parris wrote: Well now, if a spambot actually does start recognizing and avoiding his system, doesn't that mean he wins and the spammer loses? I would say YES! You should make an effort to clean it up so that others *can* install it as a standalone daemon, as I suggested. Why? How long will it be before the spambots explicitly refuse to contact your honeypot if it is listed as an MX for the domain they're attacking? I don't see that happening. If the spammers were that sharp they would send quit and close the connection properly and defeat the meathod rather than defeating just me. But it would cost them some bandwidth and speed to do that. Especially if I added some delays before doing the rejection which would cause the spammer to have to keep the connection open longer which they aren't going to do. I'm going to think about the delay thing. You inspired possibly another good idea.
RE: Experimental - use my server for your high fake MX record
Or, The spammers will find his host and don't use the highest MX record. Or just remove his host from all the results. My best solution would be: Marc, - Clean up the code - Write a manual howto install so every admin can install it - Write an extra bit of code which will send you all the information WITHOUT the information below. - Everybody who wants it can use your great software and we all win* I have contracts with my customers that I will not use their email for other business then to deliver it to its destination. Some of my customers will get into problems if other people know their contacts. So I can give you all information about an email message without - The from - The to - The body But with all the IP addresses and with the QUIT after 451 status. * we all know you wouldn't use it as a selling point to spammers or do something else with it but can/will you write that into a contract with all other admins. And pay a large sum of money if some data is found on the internet. And do we want that type of silly contracts. No we want to stop spam and not kill every other spamkiller (application or person) met vriendelijke groet, Maurice Lucas TAOS-IT Paulus Buijsstraat 191 2613 HR Delft www.taos-it.nlhttp://www.taos-it.nl/ KvK Haaglanden nr. 27254410 From: Marc Perkel [mailto:[EMAIL PROTECTED] Sent: donderdag 8 mei 2008 19:07 To: Kevin Parris Cc: users@spamassassin.apache.org Subject: Re: Experimental - use my server for your high fake MX record Kevin Parris wrote: Well now, if a spambot actually does start recognizing and avoiding his system, doesn't that mean he wins and the spammer loses? I would say YES! You should make an effort to clean it up so that others *can* install it as a standalone daemon, as I suggested. Why? How long will it be before the spambots explicitly refuse to contact your honeypot if it is listed as an MX for the domain they're attacking? I don't see that happening. If the spammers were that sharp they would send quit and close the connection properly and defeat the meathod rather than defeating just me. But it would cost them some bandwidth and speed to do that. Especially if I added some delays before doing the rejection which would cause the spammer to have to keep the connection open longer which they aren't going to do. I'm going to think about the delay thing. You inspired possibly another good idea.
Re: IE Parse bug olso in SpamAssassin ?
On Thu, May 8, 2008 18:07, John Hardin wrote: Bayes isn't going to parse a URI as a URI anyway, is it? i belived it did use that info olso It just tokenizes the message. hopefully with url that confirm to rfc olso, but i see hte point where url is obfu not to bother now when i think more about it Bayes will pick up the domain name string if it's delimited by {} as readily as it will if it's delimited by //. yes got it now, i was just a bit blind on the hidded url in redir.html To clarify: why bother trying to deobfuscate the URI and figure out what domain it's really pointing at, so that domain can be checked against URIBL lists, the hidded url could olso be a whitelisted domain if the form of the obfuscation is obvious and not seen in legitimate emails? obfu is genricly a spam sign Why not just give that obfuscation four or five points and be done with it? yep i will If that formatting *was* seen in legitimate emails, then I would say that it's important the URI parsers be aware of it. yes, my fault not thinking that long here :/ Can you provide any pointers to documentation of that formatting? I didn't find any in a quick gargle. if i know what to search for i could :/ i just started this thread to be sure IE parse bug is not in sa aswell since i could see domains not detecked in spam, but i got it now Benny Pedersen Need more webspace ? http://www.servage.net/?coupon=cust37098
Re: IE Parse bug olso in SpamAssassin ?
On Thu, 8 May 2008, Benny Pedersen wrote: i just started this thread to be sure IE parse bug is not in sa aswell since i could see domains not detecked in spam, but i got it now Do you have a reference for discussion of this IE Parsing bug that led you to mention this oddball URI annotation format in the first place? There might be references in that to the definition of the format. Thanks. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ [EMAIL PROTECTED]FALaholic #11174 pgpk -a [EMAIL PROTECTED] key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- USMC Rules of Gunfighting #9: Accuracy is relative: most combat shooting standards will be more dependent on pucker factor than the inherent accuracy of the gun. --- Today: the 63rd anniversary of VE day
trusted mailing list subscriber spam
Odd how mailing lists that don't obfuscate addresses don't see more trusted mailing list subscriber spam. All a spam program would have to do is say [EMAIL PROTECTED] posts lots to that list. His address must be a trusted subscriber. Well, here's one more post from him, muhahaha. OK, I suppose that would be caught by SPF rules etc., if bob likes SPF.
Re: trusted mailing list subscriber spam
On Thu, May 8, 2008 19:19, [EMAIL PROTECTED] wrote: OK, I suppose that would be caught by SPF rules etc., if bob likes SPF. what are you talking about ?, to score email addresses found on maillist a bit negative since it looks like none spammy human ? Benny Pedersen Need more webspace ? http://www.servage.net/?coupon=cust37098
Re: IE Parse bug olso in SpamAssassin ?
- Original Message - Do you have a reference for discussion of this IE Parsing bug that led you to mention this oddball URI annotation format in the first place? There might be references in that to the definition of the format. John, I'm not sure if this is the bug Benny refers to but here is a link for info on what I think he is referring to: http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2006-1185 -- Kevin W. Gagel Postmaster for College of New Caledonia (250) 562-2131 loc. 5448 [EMAIL PROTECTED] http://www.cnc.bc.ca Anti-Spam info at: http://gateway.cnc.bc.ca --- The College of New Caledonia, Visit us at http://www.cnc.bc.ca Virus scanning is done on all incoming and outgoing email. Anti-spam information for CNC can be found at http://gateway.cnc.bc.ca ---
Re: Multiple X-Envelope-From and SPF
ram wrote: At the MTA( postfix) I am inserting X-Envelope-From: If The mail had already a X-Envelope-From before landing at my MTA then There would be multiple lines of these configure postfix to replace previous ones /^(X\-Envelope\-From:.*)/ REPLACE X-$1 I am assuming you are not adding them before header_checks. Then SA refuses to do SPF for these messages , and I can see in my debug logs - [18469] dbg: message: X-Envelope-From header found after 1 or more Received lines, cannot trust envelope-from [18469] dbg: spf: relayed through one or more trusted relays, cannot use header-based Envelope-From, skipping -- How do I avoid this situation. Thanks Ram
Re: Multiple X-Envelope-From and SPF
On Thu, May 8, 2008 23:19, mouss wrote: configure postfix to replace previous ones /^(X\-Envelope\-From:.*)/ REPLACE X-$1 envelope from can here be forged better for postfix is to add envelope_sender_header Return-Path in local.cf Benny Pedersen Need more webspace ? http://www.servage.net/?coupon=cust37098
Re: Shortcircuit Question
Clayton Keller wrote: Justin Mason wrote: Clayton Keller writes: Justin Mason wrote: Matt Kettler writes: Clayton Keller wrote: I have been reading throught the Shortcircuit manpage as well as some articles within the Wiki, and the manner in which I see it performing within our install does not seem to coincide with how I am reading and presumably understanding it to work. First off, we are using SpamAssassin 3.2.4 provided by the rpmforge batch of rpm's. I have about a dozen priorities specified mainly handling URIBL, SURBL, as well as DCC, Razor, Pyzor and Bayes. What I am seeing is that although the first shortcircuit rule hits and scores appropriately. Subsequent short circuit rules will continue to fire. The scores themselves are then totaled along with the original scores for the rules. My understanding of how the shortcircuit should work is that once a shortcircuit is triggered any subsequent rules should be bypassed and the message wither classified as spam/ham or if set to on, it would use the current score specified for the rule. As a for instance: I have the following: priority URIBL_BLACK-500 priority URIBL_JP_SURBL -498 priority URIBL_SC_SURBL -488 priority URIBL_OB_SURBL -487 priority SC_URIBL_SURBL -480 priority SC_URIBL_SBL -479 priority RAZOR2_CHECK -450 priority DCC_CHECK -449 priority PYZOR_CHECK-448 priority SC_URIBL_HASH -440 meta SC_URIBL_SURBL(URIBL_BLACK (URIBL_SC_SURBL || URIBL_JP_SURBL || URIBL_OB_SURBL)) meta SC_URIBL_SBL((URIBL_BLACK || URIBL_SC_SURBL || URIBL_JP_SURBL || URIBL_OB_SURBL) URIBL_SBL) meta SC_URIBL_HASH((URIBL_BLACK || URIBL_SC_SURBL || URIBL_JP_SURBL || URIBL_OB_SURBL) (RAZOR2_CHECK || DCC_CHECK || PYZOR_CHECK)) meta SC_URIBL_SBL ((URIBL_BLACK || URIBL_SC_SURBL || URIBL_JP_SURBL || URIBL_OB_SURBL) URIBL_SBL) shortcircuit SC_URIBL_SURBL spam shortcircuit SC_URIBL_SBL spam shortcircuit SC_URIBL_HASH spam score SC_URIBL_SURBL100.00 score SC_URIBL_HASH 100.00 score SC_URIBL_SBL 100.00 I do not have a recent debug to show but I can say that from the debug I do see the SC_URIBL_SURBL trigger, after the earlier priority rules are ran. However, the remaining priorities are then ran, and if meeting critera, the RAZOR, DCC, and PYZOR rules run and then the SC_URIBL_HASH rule would trigger. Thus giving a total score of 200 + the scores for the URIBL/SURBL scores that hit and if included the Razor, DCC, and Pyzor scores as well. I was thinking after the SC_URIBL_SURBL was triggered remaining rules would not run, and the spam classification would take precendence. Am I overlooking the obvious, have I misunderstood how the SC should work, is it something with the rpm that was released by rpmforge? Any thoughts or insight would be appreciated. SA is, rather fortunately, circumventing what you're trying to do because of how DNS is handled internally. DO NOT try to split up the priority of DNS based tests. Priority and shortcircuiting is intended to be used on *fast* rules, not slow ones. If you were successful, you would make the performance of SpamAssassin absurdly slow by serializing DNS queries. *OUCH*. SA normally runs these in parallel, and running them in serial would very seriously impact performance. Currently, all DNS based tests run at their priority, but that only launches the DNS queries. All the results are gathered together at HARVEST_DNSBL_PRIORITY, which is currently set to 500. None of the rules will actually trigger until this point. actually Matt, you're wrong ;) if some of the network rules are at a higher priority than others, and are used in shortcircuit rules, SpamAssassin 3.2.x will indeed sleep until the results of those rules arrive. The idea is that, if you have the memory to support that degree of concurrency, you can make a local policy decision to do that, instead of doing the lookups at the MTA level which does effectively the same thing. This wait is logged, so you can spot it with --debug on. --j. With that said Justin, is the behavior I am seeing correct? Even though the first prioritized shortcircuit rule hits, and I see that in the debug log, shouldn't it be bypassing the remaining rules rather than continuing to process until all the shortcircuit priorities have ran? From reading the initial bug when this was originally featured, along with the man page, as well as a wiki post with an example by you, that is how I understood it to function. actually, no, it sounds like a bug.could you open a bugzilla with a demonstration config/test message? --j. Just to add, with my previous debug, I can confirm the waiting of the tests to finish as