Re: ANTIDRUG rulesets
Hi, Jon Armitage wrote: -Original Message- From: Anthony Peacock [mailto:[EMAIL PROTECTED] Sent: 13 February 2007 15:56 To: SpamAssassin Users sa-update updates the stock rules that are distributed with SA. Rules Du Jour is used to update add-on rulesets like the SARE rules. If you are not running sa-update you are only updating a small section of your rules. I believe there a way to use sa-update to get the SARE rules from saupdates.openprotect.com. Yes there is. I switched to using this a while ago. I didn't want to cloud the situation, as we were progressing in very small steps in improving the scoring of the OPs SA. As he was already using RDJ for the SARE rules I thought the easiest first step would be to get sa-update set up for the default ruleset and then once the OP was happy with that worry about moving his existing mechanism if neccessary. -- Anthony Peacock CHIME, Royal Free University College Medical School WWW:http://www.chime.ucl.ac.uk/~rmhiajp/ If you have an apple and I have an apple and we exchange apples then you and I will still each have one apple. But if you have an idea and I have an idea and we exchange these ideas, then each of us will have two ideas. -- George Bernard Shaw
RE: ANTIDRUG rulesets
I didn't want to cloud the situation, as we were progressing in very small steps in improving the scoring of the OPs SA. As he was already using RDJ for the SARE rules I thought the easiest first step would be to get sa-update set up for the default ruleset and then once the OP was happy with that worry about moving his existing mechanism if neccessary. I agree with you.. rocsca
RE: ANTIDRUG rulesets
Put a full email (including all headers) on a web page somewhere. http://www.rocsca.it/it_by_confocal.out That's not a drug spam, that's a stock spam. It just happens to be for a pharmacutical company. Sorry! I'm not very experienced with the kinds of spam.. I'ld very to learn to classify the spam per content.. I need a few documentation.. Get the SARE stocks ruleset and you will have some better luck. Often these are GIF images, so ImageInfo and FuzzyOCR can both help a lot. OK. I will do.. Indeed I already use FuzzyOCR.. but it often miss to block this email.. I'm afraid that I use a bad dictonary (the default) and I'm looking for a better one.. rocsca
RE: ANTIDRUG rulesets
Enable network tests. You may have to set up several things correctly to get this to work, but just removing -L from the spamd startup line may be enough as a start. I don' understand.. If I have a message in mbox format, what I have to do so that I can see what score SA should assign to it? I have seen the sintax of spamd command but It doesn't accept any kind of message as input parameter.. Should I run it in demonized mode and send the message on the the listening port? Looking at this my Bayes scores it highly, but so does a rules from the SARE_STOCKS rule set. There are also a number of network tests which get this. And so? How do you justify this? What I miss? Add-on rulesets. In this case the SARE stocks ruleset. Thanks, rocsca
RE: ANTIDRUG rulesets
Can you so us which tests these emails hit on your system? Please tell me how I have to do.. rocsca
Re: ANTIDRUG rulesets
Rocco Scappatura wrote: Can you so us which tests these emails hit on your system? Please tell me how I have to do.. If you have the email saved in a text file called email.txt, run this command making sure that you are logged in as the user who spamd run as. spamassassin -t email.txt If you want a lot more information you can use the debug switch spamassassin -D -t email.txt -- Anthony Peacock CHIME, Royal Free University College Medical School WWW:http://www.chime.ucl.ac.uk/~rmhiajp/ If you have an apple and I have an apple and we exchange apples then you and I will still each have one apple. But if you have an idea and I have an idea and we exchange these ideas, then each of us will have two ideas. -- George Bernard Shaw
RE: ANTIDRUG rulesets
If you have the email saved in a text file called email.txt, run this command making sure that you are logged in as the user who spamd run as. spamassassin -t email.txt If you want a lot more information you can use the debug switch spamassassin -D -t email.txt Thanks. Here the output on my system.. Spam detection software, running on the system av5.stt.vir, has identified this incoming email as possible spam. The original message has been attached to this so you can view it (if it isn't spam) or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: BULLISH REPORT! Campaign for: MISJPrice: $0.17Target: $0.95Market: hellish! SOMEBODY KNOWS SOMETHING. [...] Content analysis details: (0.3 points, 5.0 required) pts rule name description -- -- 0.0 HTML_MESSAGE BODY: HTML included in message 0.3 HTML_FONT_BIG BODY: HTML tag for a big font size
Re: ANTIDRUG rulesets
Hi, Rocco Scappatura wrote: If you have the email saved in a text file called email.txt, run this command making sure that you are logged in as the user who spamd run as. spamassassin -t email.txt If you want a lot more information you can use the debug switch spamassassin -D -t email.txt Thanks. Here the output on my system.. Spam detection software, running on the system av5.stt.vir, has identified this incoming email as possible spam. The original message has been attached to this so you can view it (if it isn't spam) or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: BULLISH REPORT! Campaign for: MISJPrice: $0.17Target: $0.95Market: hellish! SOMEBODY KNOWS SOMETHING. [...] Content analysis details: (0.3 points, 5.0 required) pts rule name description -- -- 0.0 HTML_MESSAGE BODY: HTML included in message 0.3 HTML_FONT_BIG BODY: HTML tag for a big font size Wow! That is low... I think the next thing you need to do is run the command with the -D switch. It doesn't look like you are running any network tests, you are certainly not running any Bayes tests. Can you remind us what OS this is on, what version of spamasssassin, how you installed SA, how you call SA? -- Anthony Peacock CHIME, Royal Free University College Medical School WWW:http://www.chime.ucl.ac.uk/~rmhiajp/ If you have an apple and I have an apple and we exchange apples then you and I will still each have one apple. But if you have an idea and I have an idea and we exchange these ideas, then each of us will have two ideas. -- George Bernard Shaw
RE: ANTIDRUG rulesets
I think the next thing you need to do is run the command with the -D switch. The output is attached.. It doesn't look like you are running any network tests, you are certainly not running any Bayes tests. I have executed the command you've sayed me after lauching spamd.. Can you remind us what OS this is on, what version of spamasssassin, how you installed SA, how you call SA? I call SA via amavisd-new-2.4.4 # /usr/bin/spamassassin --version SpamAssassin version 3.1.7 running on Perl version 5.8.8 OS: SLES 10 Linux av5 2.6.16.21-0.8-bigsmp #1 SMP Mon Jul 3 18:25:39 UTC 2006 i686 i686 i386 GNU/Linux rocsca it_by_confocal.out.debug Description: it_by_confocal.out.debug
Re: ANTIDRUG rulesets
Hi, Keep replies on the list. Rocco Scappatura wrote: [30482] dbg: dns: is Net::DNS::Resolver available? no [30482] dbg: dns: is DNS available? 0 Ive installed Net::DNS::Resolver. DNS server is local. And have you run spamassassin -D to make sure it is picking it up correctly? [30482] dbg: bayes: using username: amavis [30482] dbg: dbiplugin: Creating uncached database handle to 'bayes:mysql2.sttspa.intranet_bayes_bayes_AutoCommit=0_PrintEr ror=0_Username=bayes' [30482] dbg: bayes: unable to connect to database: Access denied for user 'bayes'@'80.74.176.142' (using password: YES) [30482] dbg: config: score set 1 chosen. [30482] dbg: dbiplugin: Creating uncached database handle to 'bayes:mysql2.sttspa.intranet_bayes_bayes_AutoCommit=0_PrintEr ror=0_Username=bayes' [30482] dbg: bayes: unable to connect to database: Access denied for user 'bayes'@'80.74.176.142' (using password: YES) Thes lines indicate that you have configured your Bayes system to use a MySQL database, but the connection to the database has failed. This looks like permission problems on the MySQL server. I ve corrected this... Good, you are now using the Bayes rules. [30482] dbg: config: read file /etc/mail/spamassassin/antidrug.cf You have the antidrug.cf rules file in your local config directory. The rules in this file are now included in the standard rule set for SA 3.x. This could be 'downgrading' some tests, remove this file. /etc/mail/spamassassin/antidrug.cf deleted Good. You still don't appear to have the 70_sare_stocks.cf file installed. Get it from http://www.rulesemporium.com and install it in the local rules folder. I have updated the config file of rules du jour to dowload the SARE STOCKs ruleset too.. Good. I ve restarted amavisd-new.. This is the score of the previous message.. Spam detection software, running on the system av5.stt.vir, has identified this incoming email as possible spam. The original message has been attached to this so you can view it (if it isn't spam) or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: BULLISH REPORT! Campaign for: MISJPrice: $0.17Target: $0.95Market: hellish! SOMEBODY KNOWS SOMETHING. [...] Content analysis details: (1.8 points, 5.0 required) pts rule name description -- -- 0.1 FORGED_RCVD_HELO Received: contains a forged HELO 1.7 SARE_PROLOSTOCK_SYM3 BODY: Last week's hot stock scam 0.0 HTML_MESSAGE BODY: HTML included in message 0.0 BAYES_50 BODY: Bayesian spam probability is 40 to 60% [score: 0.5002] OK, getting there. You are know using the SARE STOCKS rules. Your Bayes system is working, although you now need to train it that these messages are SPAM. You can use the sa-learn utility to teach the Bayes system about Spam. sa-learn --ham email.txt http://spamassassin.apache.org/full/3.1.x/doc/sa-learn.html Your network tests are still not working. Run spamassassin -D again to make sure the Net::DNS installation is being used by SA. -- Anthony Peacock CHIME, Royal Free University College Medical School WWW:http://www.chime.ucl.ac.uk/~rmhiajp/ If you have an apple and I have an apple and we exchange apples then you and I will still each have one apple. But if you have an idea and I have an idea and we exchange these ideas, then each of us will have two ideas. -- George Bernard Shaw
Re: ANTIDRUG rulesets
Hi, Rocco Scappatura wrote: Your network tests are still not working. Run spamassassin -D again to make sure the Net::DNS installation is being used by SA. Infact, I'm tryng to install it cos it is not installed.. I have succeded.. Attached there is the output of spamassassin -D before and after instruct bayes db.. Success! That looks far more healthy now. You now need to feed as many of these messages into the Bayes system as possible. The Bayes system calculates its score on statistical probabilities feeding one message may not make a huge difference to the score, so you need to continue to feed messages into the system. If you can train your Bayes system to correctly mark these at 99% probability that will give you another 3.5 marks. -- Anthony Peacock CHIME, Royal Free University College Medical School WWW:http://www.chime.ucl.ac.uk/~rmhiajp/ If you have an apple and I have an apple and we exchange apples then you and I will still each have one apple. But if you have an idea and I have an idea and we exchange these ideas, then each of us will have two ideas. -- George Bernard Shaw
Re: ANTIDRUG rulesets
Rocco Scappatura wrote: Your network tests are still not working. Run spamassassin -D again to make sure the Net::DNS installation is being used by SA. Infact, I'm tryng to install it cos it is not installed.. I have succeded.. Attached there is the output of spamassassin -D before and after instruct bayes db.. The other thing to do is to run sa-update to make sure you are running the latest versions of the standard SA rules. http://spamassassin.apache.org/full/3.1.x/doc/sa-update.html -- Anthony Peacock CHIME, Royal Free University College Medical School WWW:http://www.chime.ucl.ac.uk/~rmhiajp/ If you have an apple and I have an apple and we exchange apples then you and I will still each have one apple. But if you have an idea and I have an idea and we exchange these ideas, then each of us will have two ideas. -- George Bernard Shaw
RE: ANTIDRUG rulesets
The other thing to do is to run sa-update to make sure you are running the latest versions of the standard SA rules. http://spamassassin.apache.org/full/3.1.x/doc/sa-update.html I already use rules_du_jour.. It's OK? Or I can obtain further improvement using sa-update? rocsca
Re: ANTIDRUG rulesets
Rocco Scappatura wrote: The other thing to do is to run sa-update to make sure you are running the latest versions of the standard SA rules. http://spamassassin.apache.org/full/3.1.x/doc/sa-update.html I already use rules_du_jour.. It's OK? Or I can obtain further improvement using sa-update? sa-update updates the stock rules that are distributed with SA. Rules Du Jour is used to update add-on rulesets like the SARE rules. If you are not running sa-update you are only updating a small section of your rules. -- Anthony Peacock CHIME, Royal Free University College Medical School WWW:http://www.chime.ucl.ac.uk/~rmhiajp/ If you have an apple and I have an apple and we exchange apples then you and I will still each have one apple. But if you have an idea and I have an idea and we exchange these ideas, then each of us will have two ideas. -- George Bernard Shaw
RE: ANTIDRUG rulesets
-Original Message- From: Anthony Peacock [mailto:[EMAIL PROTECTED] Sent: 13 February 2007 15:56 To: SpamAssassin Users sa-update updates the stock rules that are distributed with SA. Rules Du Jour is used to update add-on rulesets like the SARE rules. If you are not running sa-update you are only updating a small section of your rules. I believe there a way to use sa-update to get the SARE rules from saupdates.openprotect.com. Jon
RE: ANTIDRUG rulesets
[EMAIL PROTECTED] says... I believe there a way to use sa-update to get the SARE rules from saupdates.openprotect.com. There's a good guide here: http://daryl.dostech.ca/sa-update/sare/sare- sa-update-howto.txt HTH -- A.
Re: ANTIDRUG rulesets
Rocco Scappatura ha scritto: Hello, SA doesn't blocks emails cointaining spam email with pharmaceutical contents.. I think of missing some ruleset. I cant figure out what.. I think that the more appropriate is antidrug.cf but on SA site I have read that it is unnecessary.. But if I look into the dir of conf file of spamassassin I can't find it.. Is it normal? Or I have to install it? TIA, rocsca __ Informazione NOD32 2050 (20070210) __ Questo messaggio รจ stato controllato dal Sistema Antivirus NOD32 http://www.nod32.it Try using kam.cf. Bye, Simone ABATE begin:vcard fn:Simone ABATE n:ABATE;Simone org:Software Design S.P.A.;Information Tecnology email;internet:[EMAIL PROTECTED] title:System Manager tel;work:+390817896828 tel;fax:+390817896344 x-mozilla-html:TRUE version:2.1 end:vcard
Re: ANTIDRUG rulesets
Rocco Scappatura wrote: Hello, SA doesn't blocks emails cointaining spam email with pharmaceutical contents.. I think of missing some ruleset. I cant figure out what.. I think that the more appropriate is antidrug.cf but on SA site I have read that it is unnecessary.. But if I look into the dir of conf file of spamassassin I can't find it.. Is it normal? Or I have to install it? Antidrug has been mereged into 20_drugs.cf from the standard ruleset. If you read through the file, you'll find the antidrug rules. It's about halfway down. Some of the comments have been stripped, but this part of the head comments still exist in 20_drugs.cf, and you can look at it to find the start of the antidrug section. # Note: many of the drugs named in here are brand-names and are trademarked. # All trademarks are property of the respective owners. #current best char substitutions # i - [i1!|l\xEC-\xEF] # a - [EMAIL PROTECTED] # e - [e3\xE8-\xEB] # o - [o0\xF2-\xF6] # u - [u\xB5\xF9-\xFC] # v - (?:\\\/|V) # l - [l!|1]
RE: ANTIDRUG rulesets
Antidrug has been mereged into 20_drugs.cf from the standard ruleset. If you read through the file, you'll find the antidrug rules. It's about halfway down. OK. Now Its all clear!! I have an old 'antidrug.cf' file in SA config dir.. maybe this overcome 20_drugs.cf? I don't know.. but I have removed it as well and restarted Amavisd-new, as Docs state for SA3.0.1 (I have SA 3.1.7). But I note that some 'pharma message' still is not blocked.. Do I have to install some other ruleset? (If yes how I have to configure automatic update with rdj?) thanks, rocsca
Re: ANTIDRUG rulesets
Hi, Rocco Scappatura wrote: Antidrug has been mereged into 20_drugs.cf from the standard ruleset. If you read through the file, you'll find the antidrug rules. It's about halfway down. OK. Now Its all clear!! I have an old 'antidrug.cf' file in SA config dir.. maybe this overcome 20_drugs.cf? I don't know.. but I have removed it as well and restarted Amavisd-new, as Docs state for SA3.0.1 (I have SA 3.1.7). But I note that some 'pharma message' still is not blocked.. Do I have to install some other ruleset? (If yes how I have to configure automatic update with rdj?) I think you need to make available an example of the messages that aren't being stopped. Preferably with full headers. That way people here can run them through their systems and tell you which rules hit them. Put a full email (including all headers) on a web page somewhere. -- Anthony Peacock CHIME, Royal Free University College Medical School WWW:http://www.chime.ucl.ac.uk/~rmhiajp/ If you have an apple and I have an apple and we exchange apples then you and I will still each have one apple. But if you have an idea and I have an idea and we exchange these ideas, then each of us will have two ideas. -- George Bernard Shaw
Re: ANTIDRUG rulesets
Hi, Rocco Scappatura wrote: Put a full email (including all headers) on a web page somewhere. http://www.rocsca.it/it_by_confocal.out My scores: Content analysis details: (10.4 points, 5.0 required) pts rule name description -- -- 0.1 FORGED_RCVD_HELO Received: contains a forged HELO 0.0 DK_POLICY_SIGNSOME Domain Keys: policy says domain signs some mails 1.7 SARE_PROLOSTOCK_SYM3 BODY: Last week's hot stock scam 0.0 HTML_MESSAGE BODY: HTML included in message 3.5 BAYES_99 BODY: Bayesian spam probability is 99 to 100% [score: 1.] 0.5 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/) 1.5 RAZOR2_CF_RANGE_E4_51_100 Razor2 gives engine 4 confidence level above 50% [cf: 100] 0.5 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level above 50% [cf: 100] 1.0 RCVD_IN_JANET_DUL RBL: Relay in JANET MAPS RBL+ DUL [60.215.113.19 listed in rbl-plus.mail-abuse.ja.net] 1.6 RCVD_IN_BL_SPAMCOP_NET RBL: Received via a relay in bl.spamcop.net [Blocked - see http://www.spamcop.net/bl.shtml?60.215.113.19] Looking at this my Bayes scores it highly, but so does a rules from the SARE_STOCKS rule set. There are also a number of network tests which get this. -- Anthony Peacock CHIME, Royal Free University College Medical School WWW:http://www.chime.ucl.ac.uk/~rmhiajp/ If you have an apple and I have an apple and we exchange apples then you and I will still each have one apple. But if you have an idea and I have an idea and we exchange these ideas, then each of us will have two ideas. -- George Bernard Shaw
RE: ANTIDRUG rulesets
Put a full email (including all headers) on a web page somewhere. http://www.rocsca.it/it_by_confocal.out
RE: ANTIDRUG rulesets
My scores: Content analysis details: (10.4 points, 5.0 required) pts rule name description -- -- 0.1 FORGED_RCVD_HELO Received: contains a forged HELO 0.0 DK_POLICY_SIGNSOME Domain Keys: policy says domain signs some mails 1.7 SARE_PROLOSTOCK_SYM3 BODY: Last week's hot stock scam 0.0 HTML_MESSAGE BODY: HTML included in message 3.5 BAYES_99 BODY: Bayesian spam probability is 99 to 100% [score: 1.] 0.5 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/) 1.5 RAZOR2_CF_RANGE_E4_51_100 Razor2 gives engine 4 confidence level above 50% [cf: 100] 0.5 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level above 50% [cf: 100] 1.0 RCVD_IN_JANET_DUL RBL: Relay in JANET MAPS RBL+ DUL [60.215.113.19 listed in rbl-plus.mail-abuse.ja.net] 1.6 RCVD_IN_BL_SPAMCOP_NET RBL: Received via a relay in bl.spamcop.net [Blocked - see http://www.spamcop.net/bl.shtml?60.215.113.19] How I have to do to get the score for the same message on my platform? Looking at this my Bayes scores it highly, but so does a rules from the SARE_STOCKS rule set. There are also a number of network tests which get this. And so? How do you justify this? What I miss? Thanks, rocsca
Re: ANTIDRUG rulesets
Put a full email (including all headers) on a web page somewhere. http://www.rocsca.it/it_by_confocal.out That's not a drug spam, that's a stock spam. It just happens to be for a pharmacutical company. Get the SARE stocks ruleset and you will have some better luck. Often these are GIF images, so ImageInfo and FuzzyOCR can both help a lot. Loren
Re: ANTIDRUG rulesets
How I have to do to get the score for the same message on my platform? Enable network tests. You may have to set up several things correctly to get this to work, but just removing -L from the spamd startup line may be enough as a start. Looking at this my Bayes scores it highly, but so does a rules from the SARE_STOCKS rule set. There are also a number of network tests which get this. And so? How do you justify this? What I miss? Add-on rulesets. In this case the SARE stocks ruleset. Loren
Re: ANTIDRUG rulesets
Hi, Rocco Scappatura wrote: My scores: Content analysis details: (10.4 points, 5.0 required) pts rule name description -- -- 0.1 FORGED_RCVD_HELO Received: contains a forged HELO 0.0 DK_POLICY_SIGNSOME Domain Keys: policy says domain signs some mails 1.7 SARE_PROLOSTOCK_SYM3 BODY: Last week's hot stock scam 0.0 HTML_MESSAGE BODY: HTML included in message 3.5 BAYES_99 BODY: Bayesian spam probability is 99 to 100% [score: 1.] 0.5 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/) 1.5 RAZOR2_CF_RANGE_E4_51_100 Razor2 gives engine 4 confidence level above 50% [cf: 100] 0.5 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level above 50% [cf: 100] 1.0 RCVD_IN_JANET_DUL RBL: Relay in JANET MAPS RBL+ DUL [60.215.113.19 listed in rbl-plus.mail-abuse.ja.net] 1.6 RCVD_IN_BL_SPAMCOP_NET RBL: Received via a relay in bl.spamcop.net [Blocked - see http://www.spamcop.net/bl.shtml?60.215.113.19] How I have to do to get the score for the same message on my platform? Looking at this my Bayes scores it highly, but so does a rules from the SARE_STOCKS rule set. There are also a number of network tests which get this. And so? How do you justify this? What I miss? 1. You need to download and install the SARE_STOCKS ruleset from http://www.rulesemporium.com/ 2. You should enable network tests Can you so us which tests these emails hit on your system? -- Anthony Peacock CHIME, Royal Free University College Medical School WWW:http://www.chime.ucl.ac.uk/~rmhiajp/ If you have an apple and I have an apple and we exchange apples then you and I will still each have one apple. But if you have an idea and I have an idea and we exchange these ideas, then each of us will have two ideas. -- George Bernard Shaw
Re: ANTIDRUG rulesets
Rocco Scappatura wrote: Antidrug has been mereged into 20_drugs.cf from the standard ruleset. If you read through the file, you'll find the antidrug rules. It's about halfway down. OK. Now Its all clear!! I have an old 'antidrug.cf' file in SA config dir.. maybe this overcome 20_drugs.cf? I don't know.. but I have removed it as well and restarted Amavisd-new, as Docs state for SA3.0.1 (I have SA 3.1.7). But I note that some 'pharma message' still is not blocked.. Do I have to install some other ruleset? (If yes how I have to configure automatic update with rdj?) I've not maintained antidrug in a long time, and about 6 months ago one of the pharma spammers finally found a few good holes in the regexes and have been using them extensively. If I were to update antidrug, I'd do so by posting it to the SA official tree, so it would wind up shipping out over sa-update. That said, I don't have much free time these days. Now that I have a family, that's become a more important priority. As for other rulesets, I dono who else has been working on such things.
