Re: SARE and RulesDuJour still relevant
On Fri, 14 Jan 2011 11:04:49 -1000, Warren Togami Jr. wtog...@gmail.com wrote: Anyone else have effective local rules? Please let me know and I'll put them into the nightly masscheck for testing. On 14.01.11 23:19, Benny Pedersen wrote: meta SPF_NICE_PASS (SPF_HELO_PASS SPF_PASS) I don't see any benefit of this rule, do you? -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Nothing is fool-proof to a talented fool.
Re: SARE and RulesDuJour still relevant
On Mon, 17 Jan 2011 11:43:16 +0100, Matus UHLAR - fantomas uh...@fantomas.sk wrote: On 14.01.11 23:19, Benny Pedersen wrote: meta SPF_NICE_PASS (SPF_HELO_PASS SPF_PASS) I don't see any benefit of this rule, do you? it only hits ham here, never spam, so wanted to know if that same in public corpus, in my own testing its usefull with ham since most spammers can get SPF_PASS and not much spam have SPF_HELO_PASS at the same time
Re: SARE and RulesDuJour still relevant
On 15/01/11 00:19, Warren Togami Jr. wrote: On 01/14/2011 01:09 PM, Ned Slider wrote: On 14/01/11 21:04, Warren Togami Jr. wrote: Anyone else have effective local rules? Please let me know and I'll put them into the nightly masscheck for testing. Warren header NSL_RCVD_HELO_USER Received =~ /helo[= ]user\)/i describe NSL_RCVD_HELO_USER Received from HELO User Might want to combine into a meta rule with existing NSL_RCVD_FROM_USER rule: header NSL_RCVD_FROM_USER Received =~ /from User [\[\(]/ describe NSL_RCVD_FROM_USER Received from User The above are particularly effective (here) against 419 / bank phish type emails sent from compromised webmail accounts. Hit rate is not great, but the FP count is near zero. Regards, Ned Thanks Ned, Both of the above rules are already in trunk/rulesrc/sandbox/jhardin/20_misc_testing.cf. http://ruleqa.spamassassin.org/20110114-r1058896-n/NSL_RCVD_FROM_USER/detail 0.5% spam hit rate, and some ham hits, however they are all in the ancient enron corpus that we will soon be removing. http://ruleqa.spamassassin.org/20110114-r1058896-n/T_NSL_RCVD_HELO_USER/detail Very few spam hits, and a number of ham hits but all in DOS's corpus. Perhaps we should ask him if they really are ham? Could you please describe how these rules work, and why the combination of them would be useful? Ah sorry, I meant to OR them in a meta rule: The idea behind these rules originates from a discussion on the old SpamL list around a year ago. They hit against a webmail - smtp injection point typically seen in compromised webmail accounts. Because they are so specific, some speculated this must be unique to only a few webmail packages. So we are simply looking back at (typically) the first Received header for strings like: Received: from User ([85.153.20.122]) Received: from User (unknown [200.138.162.23]) Received: from User (unverified [77.250.43.54]) by mail.hotspace.com.au or Received: from unknown (HELO User) (124.124.1.228) Received: from [62.172.163.253] (account t...@kievnet.com.ua HELO User) Received: from [75.137.153.140] (helo=User) Received: from [71.82.50.143] ([71.82.50.143:4150] helo=User) In a year of running them locally I've never seen them hit on a ham message. They appear to hit quite well for me because I pre-filter 95%+ of my spam at the smtp level (greylisting, HELO checks, spamhaus etc) so SA only gets to see the difficult to catch stuff which might inflate the percentage hits. As I said, they typically hit against bank phish sent from compromised accounts on legit servers hence why they make it through greylisting and many DNSBLs. In my corpus of 3402 spam I see NSL_RCVD_FROM_USER hit 604 (17.8%) and NSL_RCVD_HELO_USER hit 181 (5.3%). As there is (virtually?) no overlap, that's a combined hit rate of ~23%, the vast majority of which I would bet is bank phish. That is why I say these rules perform well for me - once you take out the spam that's trivial to filter (spambot spam), the hit rate against the remaining spam goes up. NSL_RCVD_FROM_USER already has a score. It appears that the combination of the two rules will be zero masscheck FP's, but a maximum of 0.1% spam hits. I suppose this is worthwhile for a night of testing, but I suspect it will be too small? Warren
Re: SARE and RulesDuJour still relevant
On 15/01/11 01:54, John Hardin wrote: On Fri, 14 Jan 2011, Ned Slider wrote: header NSL_RCVD_HELO_USER Received =~ /helo[= ]user\)/i describe NSL_RCVD_HELO_USER Received from HELO User Might want to combine into a meta rule with existing NSL_RCVD_FROM_USER rule: header NSL_RCVD_FROM_USER Received =~ /from User [\[\(]/ describe NSL_RCVD_FROM_USER Received from User The above are particularly effective (here) against 419 / bank phish type emails sent from compromised webmail accounts. Hit rate is not great, but the FP count is near zero. Ned, I put those into my sandbox when you first suggested them and they are performing _quite_ well. Hi John, Yes, sorry - I had forgotten you tested these.
Re: SARE and RulesDuJour still relevant
On 01/15/2011 01:36 AM, Ned Slider wrote: In a year of running them locally I've never seen them hit on a ham message. They appear to hit quite well for me because I pre-filter 95%+ of my spam at the smtp level (greylisting, HELO checks, spamhaus etc) so SA only gets to see the difficult to catch stuff which might inflate the percentage hits. As I said, they typically hit against bank phish sent from compromised accounts on legit servers hence why they make it through greylisting and many DNSBLs. In my corpus of 3402 spam I see NSL_RCVD_FROM_USER hit 604 (17.8%) and NSL_RCVD_HELO_USER hit 181 (5.3%). As there is (virtually?) no overlap, that's a combined hit rate of ~23%, the vast majority of which I would bet is bank phish. That is why I say these rules perform well for me - once you take out the spam that's trivial to filter (spambot spam), the hit rate against the remaining spam goes up. It seems that NSL_RCVD_FROM_USER is indeed safe (no FP's except for trec_enron), but the spam hit rate may vary wildly on different targets. My servers without any pre-spamassassin filters are seeing ~0.5-1.5% hit rates. 72_scores.cf score NSL_RCVD_FROM_USER1.180 1.226 1.180 1.226 spamassassin-3.3.x already has NSL_RCVD_FROM_USER with a production score. I am confused as to how NSL_RCVD_FROM_USER got this score, because AFAICT NSL_RCVD_FROM_USER was not in the 3.3 masscheck. In any case, OR with NSL_RCVD_FROM_HELO isn't going to be helpful as you're only piling up more score. Assigning a score to the HELO rule might be a good idea if we are certain it is safe. OTOH, the masschecks indicate very little hits at all on that rule. Warren
Re: SARE and RulesDuJour still relevant
On 01/14/2011 01:28 PM, James Lay wrote: Hey All! Been a while since I did a full blown install of SpamAssassin, and as I'm looking at my old setup, I see a fair amount of changes. I have the SARE rules as well as RulesDuJour running, but noticed that on a fresh install of SA, after doing an sa-update, there are very few rules files (the bulk of which are in /var/lib/spamassassin/3.003001/). Have rules been optimized or something? Should I copy over all the SARE rules and setup RulesDuJour to update, or leave as is? Thanks for the input. James As far as I know SpamAssassin Rules Emporium alias SARE is depreciated and no longer maintained... M$-Internet Exploder est le cancer de l'Internet, voyez pourquoi ici: http://www.aful.org/ressources/documentations/msie-problemes-securite -- (°- Bernard Lheureux Gestionnaire des MailingLists ML, TechML, LinuxML //\ http://www.bbsoft4.org/Mailinglists.htm ** MailTo:r...@bbsoft4.org v_/_ http://www.bbsoft4.org/ * http://www.portalinux.org/
Re: SARE and RulesDuJour still relevant
This is getting asked about every week :-) Short answer: no, not relevant anymore, don't use. Kai -- Get your web at Conactive Internet Services: http://www.conactive.com
Re: SARE and RulesDuJour still relevant
On 2011/01/14 7:28 AM, James Lay wrote: Hey All! Been a while since I did a full blown install of SpamAssassin, and as I'm looking at my old setup, I see a fair amount of changes. I have the SARE rules as well as RulesDuJour running, but noticed that on a fresh install of SA, after doing an sa-update, there are very few rules files (the bulk of which are in /var/lib/spamassassin/3.003001/). Have rules been optimized or something? Should I copy over all the SARE rules and setup RulesDuJour to update, or leave as is? Thanks for the input. James Since SA 3.3, rules are no longer included in the tarball. You are expected to run sa-update after installation to get the latest ruleset, which you've obviously done. There is no need to copy anything after that point. RulesDuJour is deprecated in favor of sa-update and custom channels. Worthwhile SARE rules were pulled into stock SA and are no longer being updated. Be careful when looking for additional channels due to outdated info still lurking about. I use SOUGHT, others find KHOP useful and there may be a couple of others. The OpenProtect channels should not be used. Check the list archives for recent posts on the matter. -- /Jason smime.p7s Description: S/MIME Cryptographic Signature
Re: SARE and RulesDuJour still relevant
On 01/14/2011 01:28 PM, James Lay wrote: Been a while since I did a full blown install of SpamAssassin, and as I'm looking at my old setup, I see a fair amount of changes. I have the SARE rules as well as RulesDuJour running, but noticed that on a fresh install of SA, after doing an sa-update, there are very few rules files (the bulk of which are in /var/lib/spamassassin/3.003001/). Have rules been optimized or something? Should I copy over all the SARE rules and setup RulesDuJour to update, or leave as is? Thanks for the input. On 14.01.11 13:34, Bernard Lheureux wrote: As far as I know SpamAssassin Rules Emporium alias SARE is depreciated and no longer maintained... and RulesDuJour is deprecated even longer. Last recommendations were to use sa-update even for SARE rules. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. 42.7 percent of all statistics are made up on the spot.
Re: SARE and RulesDuJour still relevant
On 1/14/11 6:38 AM, Jason Bertoch ja...@i6ix.com wrote: On 2011/01/14 7:28 AM, James Lay wrote: Hey All! Been a while since I did a full blown install of SpamAssassin, and as I'm looking at my old setup, I see a fair amount of changes. I have the SARE rules as well as RulesDuJour running, but noticed that on a fresh install of SA, after doing an sa-update, there are very few rules files (the bulk of which are in /var/lib/spamassassin/3.003001/). Have rules been optimized or something? Should I copy over all the SARE rules and setup RulesDuJour to update, or leave as is? Thanks for the input. James Since SA 3.3, rules are no longer included in the tarball. You are expected to run sa-update after installation to get the latest ruleset, which you've obviously done. There is no need to copy anything after that point. RulesDuJour is deprecated in favor of sa-update and custom channels. Worthwhile SARE rules were pulled into stock SA and are no longer being updated. Be careful when looking for additional channels due to outdated info still lurking about. I use SOUGHT, others find KHOP useful and there may be a couple of others. The OpenProtect channels should not be used. Check the list archives for recent posts on the matter. -- /Jason Excellentthanks for the quick answers all...have to admit SpamAssassin seems a lot easier to set up then a few years ago :) James P.S. Glad I could carry the torch of this question for this week ;)
Re: SARE and RulesDuJour still relevant
On 1/14/2011 2:28 AM, James Lay wrote: Hey All! Been a while since I did a full blown install of SpamAssassin, and as I'm looking at my old setup, I see a fair amount of changes. I have the SARE rules as well as RulesDuJour running, but noticed that on a fresh install of SA, after doing an sa-update, there are very few rules files (the bulk of which are in /var/lib/spamassassin/3.003001/). Have rules been optimized or something? Should I copy over all the SARE rules and setup RulesDuJour to update, or leave as is? Thanks for the input. James http://www.spamtips.org/ See my blog for current recommendations of rules that are tested to be safe. I use nightly masscheck results at http://ruleqa.spamassassin.org/ in addition to local masschecks to verify that rules are safe before making recommendations. https://admin.fedoraproject.org/mailman/listinfo/spamassassin-news Spamassassin for Sysadmins Newsletter You have installed all the optional plugins right (pyzor, razor, dcc)? http://www.spamtips.org/2010/12/cacheredir-rule-prevent-google-cache.html CACHEREDIR here has proven to be completely safe, while effective against 1-4% of low scoring spam. http://wiki.apache.org/spamassassin/SoughtRules Use SOUGHT. It is good. Anyone else have effective local rules? Please let me know and I'll put them into the nightly masscheck for testing. Warren
Re: SARE and RulesDuJour still relevant
On Fri, 14 Jan 2011 11:04:49 -1000, Warren Togami Jr. wtog...@gmail.com wrote: Anyone else have effective local rules? Please let me know and I'll put them into the nightly masscheck for testing. meta SPF_NICE_PASS (SPF_HELO_PASS SPF_PASS)
Re: SARE and RulesDuJour still relevant
On 14/01/11 21:04, Warren Togami Jr. wrote: Anyone else have effective local rules? Please let me know and I'll put them into the nightly masscheck for testing. Warren header NSL_RCVD_HELO_USER Received =~ /helo[= ]user\)/i describeNSL_RCVD_HELO_USER Received from HELO User Might want to combine into a meta rule with existing NSL_RCVD_FROM_USER rule: header NSL_RCVD_FROM_USER Received =~ /from User [\[\(]/ describe NSL_RCVD_FROM_USER Received from User The above are particularly effective (here) against 419 / bank phish type emails sent from compromised webmail accounts. Hit rate is not great, but the FP count is near zero. Regards, Ned
Re: SARE and RulesDuJour still relevant
On 01/14/2011 01:09 PM, Ned Slider wrote: On 14/01/11 21:04, Warren Togami Jr. wrote: Anyone else have effective local rules? Please let me know and I'll put them into the nightly masscheck for testing. Warren header NSL_RCVD_HELO_USER Received =~ /helo[= ]user\)/i describe NSL_RCVD_HELO_USER Received from HELO User Might want to combine into a meta rule with existing NSL_RCVD_FROM_USER rule: header NSL_RCVD_FROM_USER Received =~ /from User [\[\(]/ describe NSL_RCVD_FROM_USER Received from User The above are particularly effective (here) against 419 / bank phish type emails sent from compromised webmail accounts. Hit rate is not great, but the FP count is near zero. Regards, Ned Thanks Ned, Both of the above rules are already in trunk/rulesrc/sandbox/jhardin/20_misc_testing.cf. http://ruleqa.spamassassin.org/20110114-r1058896-n/NSL_RCVD_FROM_USER/detail 0.5% spam hit rate, and some ham hits, however they are all in the ancient enron corpus that we will soon be removing. http://ruleqa.spamassassin.org/20110114-r1058896-n/T_NSL_RCVD_HELO_USER/detail Very few spam hits, and a number of ham hits but all in DOS's corpus. Perhaps we should ask him if they really are ham? Could you please describe how these rules work, and why the combination of them would be useful? NSL_RCVD_FROM_USER already has a score. It appears that the combination of the two rules will be zero masscheck FP's, but a maximum of 0.1% spam hits. I suppose this is worthwhile for a night of testing, but I suspect it will be too small? Warren
Re: SARE and RulesDuJour still relevant
On Fri, 14 Jan 2011, Warren Togami Jr. wrote: Anyone else have effective local rules? Please let me know and I'll put them into the nightly masscheck for testing. I need to put in my postcard rules... -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- Activist: Someone who gets involved. Unregistered Lobbyist: Someone who gets involved with something the MSM doesn't approve of. -- WizardPC --- 3 days until Benjamin Franklin's 305th Birthday
Re: SARE and RulesDuJour still relevant
On Fri, 14 Jan 2011, Ned Slider wrote: header NSL_RCVD_HELO_USER Received =~ /helo[= ]user\)/i describeNSL_RCVD_HELO_USER Received from HELO User Might want to combine into a meta rule with existing NSL_RCVD_FROM_USER rule: header NSL_RCVD_FROM_USER Received =~ /from User [\[\(]/ describe NSL_RCVD_FROM_USER Received from User The above are particularly effective (here) against 419 / bank phish type emails sent from compromised webmail accounts. Hit rate is not great, but the FP count is near zero. Ned, I put those into my sandbox when you first suggested them and they are performing _quite_ well. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- Activist: Someone who gets involved. Unregistered Lobbyist: Someone who gets involved with something the MSM doesn't approve of. -- WizardPC --- 3 days until Benjamin Franklin's 305th Birthday
