Spamassassin Rule Scores
Can someone tell me how we can find out the Spamassassin rule scores? Also, in the mailllog, I would like to see spamd show not just the tests, but the scores of each. At present we get just: spamd[7403]: spamd: result: Y 4 - DKIM_SIGNED,HTML_MESSAGE,MIME_HTML_ONLY,T_DKIM_INVALID,T_REMOTE_IMAGE scantime=2.8,size=60575,user=qscand,uid=10002,required_score=4.0,rhost=local host,raddr=127.0.0.1,rport=43670,mid=0.0.15.ABC.1CF5F4E42D3972E.EDCA@vmta-d -12.lstrk.net,autolearn=disabled I understand that adding a line to local.conf: add_header all HP _TESTSSCORES(,)_ will add something to emails themselves but can anyone tell me where we can show the scores in the maillog. Many thanks, in advance for any help, Christoph
Re: Spamassassin Rule Scores
On Thursday 24 April 2014 at 11:12, emailitis.com wrote: Can someone tell me how we can find out the Spamassassin rule scores? https://spamassassin.apache.org/tests_3_3_x.html Antony -- I want to build a machine that will be proud of me. - Danny Hillis, creator of The Connection Machine Please reply to the list; please don't CC me.
false positives by FREEMAIL_FORGED_REPLYTO
Since Yahoo and AOL have moved to a DMARC policy of reject, mail senders are changing the way they are sending their emails. Instead of using the email address of an user in RFC5322.From they use their own address and put the address of the user in the Reply-To field. FREEMAIL_FORGED_REPLYTO fires on these emails and produce false positives. From examples taken from log lines of amavisd: From: GIVENNAME_SURNAME_via_LinkedIn_mem...@linkedin.com (dkim:AUTHOR) From: NAME_via_Dropbox_no-re...@dropbox.com (dkim:AUTHOR) Since more and more such emails will occur, for example all web forms will send their emails in this way, the rule does not make sense anymore. -- Michael
Re: false positives by FREEMAIL_FORGED_REPLYTO
On 04/24/2014 12:52 PM, Michael Storz wrote: Since Yahoo and AOL have moved to a DMARC policy of reject, mail senders are changing the way they are sending their emails. Instead of using the email address of an user in RFC5322.From they use their own address and put the address of the user in the Reply-To field. FREEMAIL_FORGED_REPLYTO fires on these emails and produce false positives. From examples taken from log lines of amavisd: From: GIVENNAME_SURNAME_via_LinkedIn_mem...@linkedin.com (dkim:AUTHOR) From: NAME_via_Dropbox_no-re...@dropbox.com (dkim:AUTHOR) Since more and more such emails will occur, for example all web forms will send their emails in this way, the rule does not make sense anymore. good thing you can lower the score if that rule can cause FPs on its own. The rule does what it was designed to.
Re: false positives by FREEMAIL_FORGED_REPLYTO
Am 2014-04-24 12:58, schrieb Axb: On 04/24/2014 12:52 PM, Michael Storz wrote: Since Yahoo and AOL have moved to a DMARC policy of reject, mail senders are changing the way they are sending their emails. Instead of using the email address of an user in RFC5322.From they use their own address and put the address of the user in the Reply-To field. FREEMAIL_FORGED_REPLYTO fires on these emails and produce false positives. From examples taken from log lines of amavisd: From: GIVENNAME_SURNAME_via_LinkedIn_mem...@linkedin.com (dkim:AUTHOR) From: NAME_via_Dropbox_no-re...@dropbox.com (dkim:AUTHOR) Since more and more such emails will occur, for example all web forms will send their emails in this way, the rule does not make sense anymore. good thing you can lower the score if that rule can cause FPs on its own. Sure, that's what I have done already. The rule does what it was designed to. Well, if we want to do hairsplitting, then the answer is no: it is not forged anymore, therefore the name is wrong ;-) -- Michael
Re: false positives by FREEMAIL_FORGED_REPLYTO
On 04/24/2014 01:22 PM, Michael Storz wrote: Am 2014-04-24 12:58, schrieb Axb: On 04/24/2014 12:52 PM, Michael Storz wrote: Since Yahoo and AOL have moved to a DMARC policy of reject, mail senders are changing the way they are sending their emails. Instead of using the email address of an user in RFC5322.From they use their own address and put the address of the user in the Reply-To field. FREEMAIL_FORGED_REPLYTO fires on these emails and produce false positives. From examples taken from log lines of amavisd: From: GIVENNAME_SURNAME_via_LinkedIn_mem...@linkedin.com (dkim:AUTHOR) From: NAME_via_Dropbox_no-re...@dropbox.com (dkim:AUTHOR) Since more and more such emails will occur, for example all web forms will send their emails in this way, the rule does not make sense anymore. good thing you can lower the score if that rule can cause FPs on its own. Sure, that's what I have done already. The rule does what it was designed to. Well, if we want to do hairsplitting, then the answer is no: it is not forged anymore, therefore the name is wrong ;-) pls pastebin a sample msg including full headers.
Re: false positives by FREEMAIL_FORGED_REPLYTO
Michael Storz skrev den 2014-04-24 12:52: From: GIVENNAME_SURNAME_via_LinkedIn_mem...@linkedin.com (dkim:AUTHOR) From: NAME_via_Dropbox_no-re...@dropbox.com (dkim:AUTHOR) Since more and more such emails will occur, for example all web forms will send their emails in this way, the rule does not make sense anymore. let it fire, opendkim still see the dkim key break on maillist that break it but since its dkim pass above, whats the problem in add linkedin to whitelist_from_dkim *@linkedin.com whitelist_from_dkim *@drobbox.com maybe just use def_whitelist_from_dkim if scores is ok with it ? blame yahoo and aol if thay dont send dkim passed emails, then its there fault, not users that try to hide there problem note here i do not use amavisd as a spam checker, but dkim can be policy banked on the above with diff reject score note my msg here is dkim pass and dmarc pass in you local dkim tester, then i am sure it will pass dmarc aswell if you test it end of life :=)
Re: false positives by FREEMAIL_FORGED_REPLYTO
Michael Storz skrev den 2014-04-24 13:22: Sure, that's what I have done already. shuting your own foots with it Well, if we want to do hairsplitting, then the answer is no: it is not forged anymore, therefore the name is wrong ;-) +1, if its not forged, compensate with the fact its not forged not changing score of the forged rule, spamassassin does count on scores not on single rules that fire rule name change suggest FREEMAIL_DIFF_REPLYTO make a bug if you like it
RE: Spamassassin Rule Scores
Thank you very much Antony, I had been looking at http://spamassassin.apache.org/tests.html before which is where the Tests menu goes to from the page you gave me of https://spamassassin.apache.org/tests_3_3_x.html. I'm not yet mad which is reassuring! Can anyone help with how to get scores showing in the maillog as well? Kind Regards, Christoph -Original Message- From: Antony Stone [mailto:antony.st...@spamassassin.open.source.it] Sent: 24 April 2014 11:27 To: users@spamassassin.apache.org Subject: Re: Spamassassin Rule Scores On Thursday 24 April 2014 at 11:12, emailitis.com wrote: Can someone tell me how we can find out the Spamassassin rule scores? https://spamassassin.apache.org/tests_3_3_x.html Antony -- I want to build a machine that will be proud of me. - Danny Hillis, creator of The Connection Machine Please reply to the list; please don't CC me.
Re: high cpu load
Finally i found message caused high load. It looks like one message sent all the time from ticket system. Message size is ~4M. We scan messages with this size in amavis. Content of message is complex and has multiple items Content-Type: image/gif Content-Transfer-Encoding: base64 Content-Type: application/pdf Results from debug, with % 1: dbg: rules: timing: Total time: 131.6748 s dbg: rules: [...] rulename ovl(s) max(s) #run %tot dbg: rules: [...] __FILL_THIS_FORM_LONG2 26.3811 26.3811 1 20.04% dbg: rules: [...] __FILL_THIS_FORM_SHORT2 26.3050 26.3050 1 19.98% dbg: rules: [...] __FILL_THIS_FORM_FRAUD_PHISH1 10.0878 10.0878 1 7.66% dbg: rules: [...] __FILL_THIS_FORM_LOAN1 7.2766 7.2766 1 5.53% dbg: rules: [...] __FILL_THIS_FORM_SHORT1 2.3360 2.3360 1 1.77% dbg: rules: [...] __FILL_THIS_FORM_LONG1 2.3051 2.3051 1 1.75% 1.8 FUZZY_XPILLBODY: Attempt to obfuscate words in spam 0.0 FUZZY_CPILLBODY: Attempt to obfuscate words in spam 0.5 FUZZY_VPILLBODY: Attempt to obfuscate words in spam 0.8 HTML_IMAGE_RATIO_02BODY: HTML has a low ratio of text to image area 0.0 HTML_MESSAGE BODY: HTML included in message 0.0 LOTS_OF_MONEY Huge... sums of money Thanks all for the help! 2014-04-24 1:39 GMT+03:00 John Hardin jhar...@impsec.org: On Wed, 23 Apr 2014, Nick I wrote: Another interesting thing. Today when daily cron executed at 5 am load calmed to ~0. As it was before. Sa-update executed at that time. Amavisd has been reloaded at 7 am and load raised back again. Also i see that some messages checked 150329 ms, 158742 ms. But most messages checked ~400ms. I used @debug_recipient_maps and sa_debug but did not see any userful info. Can anyone suggest how to look inside tests_pri_0 ? The first thing you need to do is capture one of the messages that took a very long time to scan, so that it can be tested in a controlled environment. There are tools that will allow you to capture timing data for every rule, and if the message is a spam you could provide it to us for analysis. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- Today: Max Planck's 156th birthday
RE: Spamassassin Rule Scores
emailitis.com skrev den 2014-04-24 14:00: Can anyone help with how to get scores showing in the maillog as well? this needs a patch to spamassassin to support syslog but spampd does what you want note not spamc/spamd
Re: Spamassassin Rule Scores
On 04/24/2014 02:00 PM, emailitis.com wrote: Thank you very much Antony, I had been looking at http://spamassassin.apache.org/tests.html before which is where the Tests menu goes to from the page you gave me of https://spamassassin.apache.org/tests_3_3_x.html. I'm not yet mad which is reassuring! Can anyone help with how to get scores showing in the maillog as well? according to spamd.raw my $yorn = $status-is_spam() ? 'Y' : '.'; my $score = $status-get_score(); my $tests = join(,, sort(grep(length,$status-get_names_of_tests_hit(; my $log = sprintf(spamd: result: %s %2d - %s %s, $yorn, $score, $tests, join(,, @extra)); info($log); It *seems* to me it's not possible without hacking spamd
Re: false positives by FREEMAIL_FORGED_REPLYTO
Am 2014-04-24 13:27, schrieb Axb: On 04/24/2014 01:22 PM, Michael Storz wrote: Am 2014-04-24 12:58, schrieb Axb: On 04/24/2014 12:52 PM, Michael Storz wrote: Since Yahoo and AOL have moved to a DMARC policy of reject, mail senders are changing the way they are sending their emails. Instead of using the email address of an user in RFC5322.From they use their own address and put the address of the user in the Reply-To field. FREEMAIL_FORGED_REPLYTO fires on these emails and produce false positives. From examples taken from log lines of amavisd: From: GIVENNAME_SURNAME_via_LinkedIn_mem...@linkedin.com (dkim:AUTHOR) From: NAME_via_Dropbox_no-re...@dropbox.com (dkim:AUTHOR) Since more and more such emails will occur, for example all web forms will send their emails in this way, the rule does not make sense anymore. good thing you can lower the score if that rule can cause FPs on its own. Sure, that's what I have done already. The rule does what it was designed to. Well, if we want to do hairsplitting, then the answer is no: it is not forged anymore, therefore the name is wrong ;-) pls pastebin a sample msg including full headers. http://pastebin.com/fSj4azex (will expire in one week) since I had to change personal information of my customer, evaluaton of DKIM will fail. But FREEMAIL_FORGED_REPLYTO will still fire. -- Michael
Re: false positives by FREEMAIL_FORGED_REPLYTO
On 04/24/2014 02:20 PM, Michael Storz wrote: Am 2014-04-24 13:27, schrieb Axb: On 04/24/2014 01:22 PM, Michael Storz wrote: Am 2014-04-24 12:58, schrieb Axb: On 04/24/2014 12:52 PM, Michael Storz wrote: Since Yahoo and AOL have moved to a DMARC policy of reject, mail senders are changing the way they are sending their emails. Instead of using the email address of an user in RFC5322.From they use their own address and put the address of the user in the Reply-To field. FREEMAIL_FORGED_REPLYTO fires on these emails and produce false positives. From examples taken from log lines of amavisd: From: GIVENNAME_SURNAME_via_LinkedIn_mem...@linkedin.com (dkim:AUTHOR) From: NAME_via_Dropbox_no-re...@dropbox.com (dkim:AUTHOR) Since more and more such emails will occur, for example all web forms will send their emails in this way, the rule does not make sense anymore. good thing you can lower the score if that rule can cause FPs on its own. Sure, that's what I have done already. The rule does what it was designed to. Well, if we want to do hairsplitting, then the answer is no: it is not forged anymore, therefore the name is wrong ;-) pls pastebin a sample msg including full headers. http://pastebin.com/fSj4azex (will expire in one week) since I had to change personal information of my customer, evaluaton of DKIM will fail. But FREEMAIL_FORGED_REPLYTO will still fire. the rule does the right thing.. # header FREEMAIL_FROM eval:check_freemail_from(['regex']) # #Checks all possible from headers to see if sender is freemail. #Uses SA all_from_addrs() function (includes 'Resent-From', 'From', #'EnvelopeFrom' etc). Linkedin have chosen to modify the From: ... let's avoid the DMARC /Y!/AOL discussion here - there's enough noise about it all over the places. for once I have to agree with Benny that some ppl may want to whitelist_from_dkim *@linkedin.com and maybe others. To lower the score or modify the rule would make it loose its teeth and it is very valuable outside the edge cases which tamper with the From:
Re: false positives by FREEMAIL_FORGED_REPLYTO
Am 2014-04-24 14:31, schrieb Axb: On 04/24/2014 02:20 PM, Michael Storz wrote: Am 2014-04-24 13:27, schrieb Axb: On 04/24/2014 01:22 PM, Michael Storz wrote: Am 2014-04-24 12:58, schrieb Axb: On 04/24/2014 12:52 PM, Michael Storz wrote: Since Yahoo and AOL have moved to a DMARC policy of reject, mail senders are changing the way they are sending their emails. Instead of using the email address of an user in RFC5322.From they use their own address and put the address of the user in the Reply-To field. FREEMAIL_FORGED_REPLYTO fires on these emails and produce false positives. From examples taken from log lines of amavisd: From: GIVENNAME_SURNAME_via_LinkedIn_mem...@linkedin.com (dkim:AUTHOR) From: NAME_via_Dropbox_no-re...@dropbox.com (dkim:AUTHOR) Since more and more such emails will occur, for example all web forms will send their emails in this way, the rule does not make sense anymore. good thing you can lower the score if that rule can cause FPs on its own. Sure, that's what I have done already. The rule does what it was designed to. Well, if we want to do hairsplitting, then the answer is no: it is not forged anymore, therefore the name is wrong ;-) pls pastebin a sample msg including full headers. http://pastebin.com/fSj4azex (will expire in one week) since I had to change personal information of my customer, evaluaton of DKIM will fail. But FREEMAIL_FORGED_REPLYTO will still fire. the rule does the right thing.. # header FREEMAIL_FROM eval:check_freemail_from(['regex']) # #Checks all possible from headers to see if sender is freemail. #Uses SA all_from_addrs() function (includes 'Resent-From', 'From', #'EnvelopeFrom' etc). Linkedin have chosen to modify the From: ... let's avoid the DMARC /Y!/AOL discussion here - there's enough noise about it all over the places. for once I have to agree with Benny that some ppl may want to whitelist_from_dkim *@linkedin.com and maybe others. I have answered that already, why this is not a good idea. To lower the score or modify the rule would make it loose its teeth and it is very valuable outside the edge cases which tamper with the From: It depends on how many false positives you are willing to accept, I am already seeing more false positives than spammails where the detection relies on this rule. And this will change in the near future to be even worse. BTW. in addition I found FPs today with regular emails from Badoo. Thanks for looking into this issue. -- Michael
Re: false positives by FREEMAIL_FORGED_REPLYTO
On 04/24/2014 03:22 PM, Michael Storz wrote: Am 2014-04-24 14:31, schrieb Axb: On 04/24/2014 02:20 PM, Michael Storz wrote: Am 2014-04-24 13:27, schrieb Axb: On 04/24/2014 01:22 PM, Michael Storz wrote: Am 2014-04-24 12:58, schrieb Axb: On 04/24/2014 12:52 PM, Michael Storz wrote: Since Yahoo and AOL have moved to a DMARC policy of reject, mail senders are changing the way they are sending their emails. Instead of using the email address of an user in RFC5322.From they use their own address and put the address of the user in the Reply-To field. FREEMAIL_FORGED_REPLYTO fires on these emails and produce false positives. From examples taken from log lines of amavisd: From: GIVENNAME_SURNAME_via_LinkedIn_mem...@linkedin.com (dkim:AUTHOR) From: NAME_via_Dropbox_no-re...@dropbox.com (dkim:AUTHOR) Since more and more such emails will occur, for example all web forms will send their emails in this way, the rule does not make sense anymore. good thing you can lower the score if that rule can cause FPs on its own. Sure, that's what I have done already. The rule does what it was designed to. Well, if we want to do hairsplitting, then the answer is no: it is not forged anymore, therefore the name is wrong ;-) pls pastebin a sample msg including full headers. http://pastebin.com/fSj4azex (will expire in one week) since I had to change personal information of my customer, evaluaton of DKIM will fail. But FREEMAIL_FORGED_REPLYTO will still fire. the rule does the right thing.. # header FREEMAIL_FROM eval:check_freemail_from(['regex']) # #Checks all possible from headers to see if sender is freemail. #Uses SA all_from_addrs() function (includes 'Resent-From', 'From', #'EnvelopeFrom' etc). Linkedin have chosen to modify the From: ... let's avoid the DMARC /Y!/AOL discussion here - there's enough noise about it all over the places. for once I have to agree with Benny that some ppl may want to whitelist_from_dkim *@linkedin.com and maybe others. I have answered that already, why this is not a good idea. To lower the score or modify the rule would make it loose its teeth and it is very valuable outside the edge cases which tamper with the From: It depends on how many false positives you are willing to accept, I am already seeing more false positives than spammails where the detection relies on this rule. And this will change in the near future to be even worse. BTW. in addition I found FPs today with regular emails from Badoo. Thanks for looking into this issue. feel free to re-open https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6744 and pls include a few samples where this issue may apply
Re: sa-learn from a cronjob?
On Wed, 23 Apr 2014 19:15:13 -0700 Ian Zimmerman wrote: On Sun, 20 Apr 2014 12:14:37 -0700 (PDT) Dan Mahoney, System Admin d...@prime.gushi.org wrote: Most of my users aren't command-line friendly. I'd like to basically have my IMAP server default to handing out two imap mailboxes that get auto-crontabbed to training bayes. Here is my cronjob for that purpose, in its entirety. I don't think it will work for the purpose mentioned, and if it's working properly for you, there's a lot you're not mentioning. It's only looking for mail in the immediate post-delivery state after it's been put into the mailbox by an MTA or MDA and before it's been detected as new mail by an MUA (directly or via IMAP). It wont learn mail put into the folders by an MUA or IMAP at all. You need to use separate destination mailboxes.
Re: false positives by FREEMAIL_FORGED_REPLYTO
Michael Storz skrev den 2014-04-24 15:22: I have answered that already, why this is not a good idea. so freemail_whitelist *@linkedin.com ? do linkedin break there own dkim ?
Re: high cpu load
On Thu, 24 Apr 2014, Nick I wrote: Finally i found message caused high load. It looks like one message sent all the time from ticket system. Message size is ~4M. We scan messages with this size in amavis. Content of message is complex and has multiple items Content-Type: image/gif Content-Transfer-Encoding: base64 Content-Type: application/pdf Results from debug, with % 1: dbg: rules: timing: Total time: 131.6748 s dbg: rules: [...] rulename ovl(s) max(s) #run %tot dbg: rules: [...] __FILL_THIS_FORM_LONG2 26.3811 26.3811 1 20.04% dbg: rules: [...] __FILL_THIS_FORM_SHORT2 26.3050 26.3050 1 19.98% dbg: rules: [...] __FILL_THIS_FORM_FRAUD_PHISH1 10.0878 10.0878 1 7.66% dbg: rules: [...] __FILL_THIS_FORM_LOAN1 7.2766 7.2766 1 5.53% dbg: rules: [...] __FILL_THIS_FORM_SHORT1 2.3360 2.3360 1 1.77% dbg: rules: [...] __FILL_THIS_FORM_LONG1 2.3051 2.3051 1 1.75% That's not too surprising if the content is 4MB. Would you be willing to share it with me so that I can try to find the problem with the FILLFORM rules? Alternatively, you might want to configure your system to not scan mails from the ticket system (which I assume is internal and trusted). -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- Yet another example of a Mexican doing a job Americans are unwilling to do. -- Reno Sepulveda, on UniVision reporters asking President Obama some pointed questions about the BATFE Fast and Furious scandal. --- 693 days since the first successful private support mission to ISS (SpaceX)
Re: high cpu load
On 04/24/2014 04:16 PM, John Hardin wrote: On Thu, 24 Apr 2014, Nick I wrote: Finally i found message caused high load. It looks like one message sent all the time from ticket system. Message size is ~4M. We scan messages with this size in amavis. Content of message is complex and has multiple items Content-Type: image/gif Content-Transfer-Encoding: base64 Content-Type: application/pdf Results from debug, with % 1: dbg: rules: timing: Total time: 131.6748 s dbg: rules: [...] rulename ovl(s) max(s) #run %tot dbg: rules: [...] __FILL_THIS_FORM_LONG2 26.3811 26.3811 1 20.04% dbg: rules: [...] __FILL_THIS_FORM_SHORT2 26.3050 26.3050 1 19.98% dbg: rules: [...] __FILL_THIS_FORM_FRAUD_PHISH1 10.0878 10.0878 1 7.66% dbg: rules: [...] __FILL_THIS_FORM_LOAN1 7.2766 7.2766 1 5.53% dbg: rules: [...] __FILL_THIS_FORM_SHORT1 2.3360 2.3360 1 1.77% dbg: rules: [...] __FILL_THIS_FORM_LONG1 2.3051 2.3051 1 1.75% That's not too surprising if the content is 4MB. Would you be willing to share it with me so that I can try to find the problem with the FILLFORM rules? Alternatively, you might want to configure your system to not scan mails from the ticket system (which I assume is internal and trusted). Why does this smell like replace_tag noise? (I hate that stuff :)
Re: false positives by FREEMAIL_FORGED_REPLYTO
On Thu, 24 Apr 2014, Axb wrote: On 04/24/2014 02:20 PM, Michael Storz wrote: Am 2014-04-24 13:27, schrieb Axb: On 04/24/2014 01:22 PM, Michael Storz wrote: Am 2014-04-24 12:58, schrieb Axb: On 04/24/2014 12:52 PM, Michael Storz wrote: Since Yahoo and AOL have moved to a DMARC policy of reject, mail senders are changing the way they are sending their emails. Instead of using the email address of an user in RFC5322.From they use their own address and put the address of the user in the Reply-To field. FREEMAIL_FORGED_REPLYTO fires on these emails and produce false positives. From examples taken from log lines of amavisd: From: GIVENNAME_SURNAME_via_LinkedIn_mem...@linkedin.com (dkim:AUTHOR) From: NAME_via_Dropbox_no-re...@dropbox.com (dkim:AUTHOR) Since more and more such emails will occur, for example all web forms will send their emails in this way, the rule does not make sense anymore. good thing you can lower the score if that rule can cause FPs on its own. Sure, that's what I have done already. The rule does what it was designed to. Well, if we want to do hairsplitting, then the answer is no: it is not forged anymore, therefore the name is wrong ;-) pls pastebin a sample msg including full headers. http://pastebin.com/fSj4azex (will expire in one week) since I had to change personal information of my customer, evaluaton of DKIM will fail. But FREEMAIL_FORGED_REPLYTO will still fire. the rule does the right thing.. # header FREEMAIL_FROM eval:check_freemail_from(['regex']) # # Checks all possible from headers to see if sender is freemail. # Uses SA all_from_addrs() function (includes 'Resent-From', 'From', # 'EnvelopeFrom' etc). Linkedin have chosen to modify the From: ... let's avoid the DMARC /Y!/AOL discussion here - there's enough noise about it all over the places. for once I have to agree with Benny that some ppl may want to whitelist_from_dkim *@linkedin.com and maybe others. To lower the score or modify the rule would make it loose its teeth and it is very valuable outside the edge cases which tamper with the From: add a meta with DKIM_VALID to subtract some points? -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- Yet another example of a Mexican doing a job Americans are unwilling to do. -- Reno Sepulveda, on UniVision reporters asking President Obama some pointed questions about the BATFE Fast and Furious scandal. --- 693 days since the first successful private support mission to ISS (SpaceX)
Re: high cpu load
On Thu, 24 Apr 2014, Axb wrote: On 04/24/2014 04:16 PM, John Hardin wrote: On Thu, 24 Apr 2014, Nick I wrote: Finally i found message caused high load. It looks like one message sent all the time from ticket system. Message size is ~4M. We scan messages with this size in amavis. Content of message is complex and has multiple items Content-Type: image/gif Content-Transfer-Encoding: base64 Content-Type: application/pdf Results from debug, with % 1: dbg: rules: timing: Total time: 131.6748 s dbg: rules: [...] rulename ovl(s) max(s) #run %tot dbg: rules: [...] __FILL_THIS_FORM_LONG2 26.3811 26.3811 1 20.04% dbg: rules: [...] __FILL_THIS_FORM_SHORT2 26.3050 26.3050 1 19.98% dbg: rules: [...] __FILL_THIS_FORM_FRAUD_PHISH1 10.0878 10.0878 1 7.66% dbg: rules: [...] __FILL_THIS_FORM_LOAN1 7.2766 7.2766 1 5.53% dbg: rules: [...] __FILL_THIS_FORM_SHORT1 2.3360 2.3360 1 1.77% dbg: rules: [...] __FILL_THIS_FORM_LONG1 2.3051 2.3051 1 1.75% That's not too surprising if the content is 4MB. Would you be willing to share it with me so that I can try to find the problem with the FILLFORM rules? Alternatively, you might want to configure your system to not scan mails from the ticket system (which I assume is internal and trusted). Why does this smell like replace_tag noise? (I hate that stuff :) The FILLFORM rules are complex and inherently repetitive - there's really no other way to detect a form. They've had some minor problems with boundedness in the past, but I thought I had fixed them. Apparently here's another bad case. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- Yet another example of a Mexican doing a job Americans are unwilling to do. -- Reno Sepulveda, on UniVision reporters asking President Obama some pointed questions about the BATFE Fast and Furious scandal. --- 693 days since the first successful private support mission to ISS (SpaceX)
Re: false positives by FREEMAIL_FORGED_REPLYTO
On 04/24/2014 04:23 PM, John Hardin wrote: On Thu, 24 Apr 2014, Axb wrote: On 04/24/2014 02:20 PM, Michael Storz wrote: Am 2014-04-24 13:27, schrieb Axb: On 04/24/2014 01:22 PM, Michael Storz wrote: Am 2014-04-24 12:58, schrieb Axb: On 04/24/2014 12:52 PM, Michael Storz wrote: Since Yahoo and AOL have moved to a DMARC policy of reject, mail senders are changing the way they are sending their emails. Instead of using the email address of an user in RFC5322.From they use their own address and put the address of the user in the Reply-To field. FREEMAIL_FORGED_REPLYTO fires on these emails and produce false positives. From examples taken from log lines of amavisd: From: GIVENNAME_SURNAME_via_LinkedIn_mem...@linkedin.com (dkim:AUTHOR) From: NAME_via_Dropbox_no-re...@dropbox.com (dkim:AUTHOR) Since more and more such emails will occur, for example all web forms will send their emails in this way, the rule does not make sense anymore. good thing you can lower the score if that rule can cause FPs on its own. Sure, that's what I have done already. The rule does what it was designed to. Well, if we want to do hairsplitting, then the answer is no: it is not forged anymore, therefore the name is wrong ;-) pls pastebin a sample msg including full headers. http://pastebin.com/fSj4azex (will expire in one week) since I had to change personal information of my customer, evaluaton of DKIM will fail. But FREEMAIL_FORGED_REPLYTO will still fire. the rule does the right thing.. # header FREEMAIL_FROM eval:check_freemail_from(['regex']) # # Checks all possible from headers to see if sender is freemail. # Uses SA all_from_addrs() function (includes 'Resent-From', 'From', # 'EnvelopeFrom' etc). Linkedin have chosen to modify the From: ... let's avoid the DMARC /Y!/AOL discussion here - there's enough noise about it all over the places. for once I have to agree with Benny that some ppl may want to whitelist_from_dkim *@linkedin.com and maybe others. To lower the score or modify the rule would make it loose its teeth and it is very valuable outside the edge cases which tamper with the From: add a meta with DKIM_VALID to subtract some points? possibly, though not something I'd want to impose on everybody, per default. eg: not everybody finds linkedin.com so cozy that they want to WL :)
Re: false positives by FREEMAIL_FORGED_REPLYTO
Am 2014-04-24 16:11, schrieb Benny Pedersen: Michael Storz skrev den 2014-04-24 15:22: I have answered that already, why this is not a good idea. so freemail_whitelist *@linkedin.com ? Does not work: rule: meta FREEMAIL_FORGED_REPLYTO __freemail_hdr_replyto !FREEMAIL_FROM !__freemail_safe example was: From: Givenname2 Surname2 via LinkedIn mem...@linkedin.com Reply-To: givenname2.surna...@gmx.de linkedin.com is not a freemail domain, gmx.de is. Therefore the rule fires. do linkedin break there own dkim ? No. -- Michael
Re: false positives by FREEMAIL_FORGED_REPLYTO
John Hardin skrev den 2014-04-24 16:23: add a meta with DKIM_VALID to subtract some points? or shortcicuit it based on just that ? shortcircuit DKIM_VALID spam no no use DKIM_VALID_AU if anything its just that this rule is not specific to linkedin :( end of life
Re: high cpu load
On 04/24/2014 04:28 PM, John Hardin wrote: On Thu, 24 Apr 2014, Axb wrote: On 04/24/2014 04:16 PM, John Hardin wrote: On Thu, 24 Apr 2014, Nick I wrote: Finally i found message caused high load. It looks like one message sent all the time from ticket system. Message size is ~4M. We scan messages with this size in amavis. Content of message is complex and has multiple items Content-Type: image/gif Content-Transfer-Encoding: base64 Content-Type: application/pdf Results from debug, with % 1: dbg: rules: timing: Total time: 131.6748 s dbg: rules: [...] rulename ovl(s) max(s) #run %tot dbg: rules: [...] __FILL_THIS_FORM_LONG2 26.3811 26.3811 1 20.04% dbg: rules: [...] __FILL_THIS_FORM_SHORT2 26.3050 26.3050 1 19.98% dbg: rules: [...] __FILL_THIS_FORM_FRAUD_PHISH1 10.0878 10.0878 1 7.66% dbg: rules: [...] __FILL_THIS_FORM_LOAN1 7.2766 7.2766 1 5.53% dbg: rules: [...] __FILL_THIS_FORM_SHORT1 2.3360 2.3360 1 1.77% dbg: rules: [...] __FILL_THIS_FORM_LONG1 2.3051 2.3051 1 1.75% That's not too surprising if the content is 4MB. Would you be willing to share it with me so that I can try to find the problem with the FILLFORM rules? Alternatively, you might want to configure your system to not scan mails from the ticket system (which I assume is internal and trusted). Why does this smell like replace_tag noise? (I hate that stuff :) The FILLFORM rules are complex and inherently repetitive - there's really no other way to detect a form. They've had some minor problems with boundedness in the past, but I thought I had fixed them. Apparently here's another bad case. seems like a LOT of work going on for the results :) score FILL_THIS_FORM1.456 0.001 1.456 0.001 I'd +1 to start from scratch with simpler conditions
Re: false positives by FREEMAIL_FORGED_REPLYTO
Michael Storz skrev den 2014-04-24 16:30: linkedin.com is not a freemail domain, gmx.de is. Therefore the rule fires. then add it, freemail_domain linkedin.com but only if you at sametime add freemail_whitelist untested do linkedin break there own dkim ? No. good, others do ?
Re: SA 3.4 'make test' fails in 't/sa_sompile.t' with Not found: able-to-use
Bizarre... The non-replicatable behavior is very confusing. And you were able to replicate this on a modern CentOS box? On 4/23/2014 6:54 PM, David Gibbs wrote: On 04/23/2014 04:42 PM, Kevin A. McGrail wrote: If you run make distclean and then perl Makefile.PL and then make tardist, does that work? I blew away the directory and untar'ed it again ... this time it didn't complain about Config.pm. /usr/local/home/david/work/Mail-SpamAssassin-3.4.0/t/log/d.sa_compile/inst.basic/foo/bin/spamassassin -p log/test_default.cf -D -Lt /usr/local/home/david/work/Mail-SpamAssassin-3.4.0/t/data/spam/001 ok 1 Checking FOO ok 2 /usr/local/home/david/work/Mail-SpamAssassin-3.4.0/t/log/d.sa_compile/inst.basic/foo/bin/sa-compile --keep-tmps Apr 23 17:49:33.185 [5749] info: generic: base extraction starting. this can take a while... Apr 23 17:49:33.185 [5749] info: generic: extracting from rules of type body_0 100% [===] 52.90 rules/sec 00m00s DONE 100% [===] 229.31 bases/sec 00m00s DONE Apr 23 17:49:33.280 [5749] info: body_0: 4 base strings extracted in 0 seconds cd /tmp/.spamassassin5749FmkqRetmp reading bases_body_0.in cd Mail-SpamAssassin-CompiledRegexps-body_0 re2c -i -b -o scanner1.c scanner1.re /usr/bin/perl Makefile.PL PREFIX=/tmp/.spamassassin5749FmkqRetmp/ignored INSTALLSITEARCH=/usr/local/home/david/work/Mail-SpamAssassin-3.4.0/t/log/d.sa_compile/inst.basic/foo/var/spamassassin/compiled/5.010/3.004000 Generating a Unix-style Makefile Writing Makefile for Mail::SpamAssassin::CompiledRegexps::body_0 Writing MYMETA.yml and MYMETA.json make cp body_0.pm blib/lib/Mail/SpamAssassin/CompiledRegexps/body_0.pm Running Mkbootstrap for Mail::SpamAssassin::CompiledRegexps::body_0 () chmod 644 body_0.bs /usr/bin/perl /usr/lib/perl5/5.10.0/ExtUtils/xsubpp -typemap /usr/lib/perl5/5.10.0/ExtUtils/typemap body_0.xs body_0.xsc mv body_0.xsc body_0.c gcc -c -D_REENTRANT -D_GNU_SOURCE -DPERL_USE_SAFE_PUTENV -DDEBUGGING -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m32 -march=i386 -mtune=generic -fasynchronous-unwind-tables -DVERSION=\1.0\ -DXS_VERSION=\1.0\ -fPIC -I/usr/lib/perl5/5.10.0/i386-linux-thread-multi/CORE body_0.c gcc -c -D_REENTRANT -D_GNU_SOURCE -DPERL_USE_SAFE_PUTENV -DDEBUGGING -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m32 -march=i386 -mtune=generic -fasynchronous-unwind-tables -DVERSION=\1.0\ -DXS_VERSION=\1.0\ -fPIC -I/usr/lib/perl5/5.10.0/i386-linux-thread-multi/CORE scanner1.c rm -f blib/arch/auto/Mail/SpamAssassin/CompiledRegexps/body_0/body_0.so gcc -shared -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m32 -march=i386 -mtune=generic -fasynchronous-unwind-tables -L/usr/local/lib body_0.o scanner1.o -o blib/arch/auto/Mail/SpamAssassin/CompiledRegexps/body_0/body_0.so \ \ chmod 755 blib/arch/auto/Mail/SpamAssassin/CompiledRegexps/body_0/body_0.so /usr/bin/perl -MExtUtils::Command::MM -e 'cp_nonempty' -- body_0.bs blib/arch/auto/Mail/SpamAssassin/CompiledRegexps/body_0/body_0.bs 644 Manifying blib/man3/Mail::SpamAssassin::CompiledRegexps::body_0.3pm make install Running Mkbootstrap for Mail::SpamAssassin::CompiledRegexps::body_0 () chmod 644 body_0.bs Files found in blib/arch: installing files in blib/lib into architecture dependent library tree Installing /usr/local/home/david/work/Mail-SpamAssassin-3.4.0/t/log/d.sa_compile/inst.basic/foo/var/spamassassin/compiled/5.010/3.004000/auto/Mail/SpamAssassin/CompiledRegexps/body_0/body_0.so Installing /usr/local/home/david/work/Mail-SpamAssassin-3.4.0/t/log/d.sa_compile/inst.basic/foo/var/spamassassin/compiled/5.010/3.004000/Mail/SpamAssassin/CompiledRegexps/body_0.pm Installing /tmp/.spamassassin5749FmkqRetmp/ignored/share/man/man3/Mail::SpamAssassin::CompiledRegexps::body_0.3pm Appending installation info to /tmp/.spamassassin5749FmkqRetmp/ignored/lib/perl5/5.10.0/i386-linux-thread-multi/perllocal.pod cp /tmp/.spamassassin5749FmkqRetmp/bases_body_0.pl /usr/local/home/david/work/Mail-SpamAssassin-3.4.0/t/log/d.sa_compile/inst.basic/foo/var/spamassassin/compiled/5.010/3.004000/bases_body_0.pl temporary dir left due to --keep-tmps: /tmp/.spamassassin5749FmkqRetmp /usr/local/home/david/work/Mail-SpamAssassin-3.4.0/t/log/d.sa_compile/inst.basic/foo/bin/spamassassin -p log/test_default.cf -D -Lt /usr/local/home/david/work/Mail-SpamAssassin-3.4.0/t/data/spam/001 ok 3 Checking able-to-use Not found: able-to-use = able to use 1/1 'body_0' compiled rules at t/sa_compile.t line 148. not ok 4 #
Re: Spamassassin Rule Scores
On 4/24/2014 6:12 AM, emailitis.com wrote: Can someone tell me how we can find out the Spamassassin rule scores? Best way is to view the applicable 50_scores.cf directly, i.e. /var/lib/spamassassin/3.004000/updates_spamassassin_org/50_scores.cf for 3.4.0 in the default location. Also, in the mailllog, I would like to see spamd show not just the tests, but the scores of each. At present we get just: spamd[7403]: spamd: result: Y 4 - DKIM_SIGNED,HTML_MESSAGE,MIME_HTML_ONLY,T_DKIM_INVALID,T_REMOTE_IMAGE scantime=2.8,size=60575,user=qscand,uid=10002,required_score=4.0,rhost=localhost,raddr=127.0.0.1,rport=43670,mid=0.0.15.abc.1cf5f4e42d3972e.e...@vmta-d-12.lstrk.net,autolearn=disabled I don't believe that functionality exists. Feel free to submit a patch to add the option, etc. though. Always love people submitting patches!
Re: Spamassassin Rule Scores
On Thu, 24 Apr 2014 10:50:25 -0400
Kevin A. McGrail kmcgr...@pccc.com wrote:
I don't believe that functionality exists. Feel free to submit a
patch to add the option, etc. though. Always love people submitting
patches!
We integrate with SpamAssassin at the Perl library level, and we reach
into the innards to get at the test scores. Here's our code:
my $conf = $sa_status-{conf} || {};
my $scores = $conf-{scores} || {};
my $testnames = join(',', (map { (exists($scores-{$_})
defined($scores-{$_})) ? $_ . ':' . $scores-{$_} : $_:? } (split(/\s*,\s*/,
$sa_status-get_names_of_tests_hit();
We end up getting output like this in our logs:
HTML_IMAGE_ONLY_20:0.7;HTML_MESSAGE:0.001;MIME_HTML_ONLY:1.105;RDNS_NONE:1.274;TO_EQ_FM_DOM_HTML_ONLY:0.489;TO_EQ_FM_HTML_ONLY:0.036;TO_NO_BRKTS_NORDNS:0.001;T_REMOTE_IMAGE:0.01
Could we make an official API call something like this?
#==
package Mail::SpamAssassin::PerMsgStatus;
sub get_hits_with_scores
{
my ($self) = @_;
my $conf = $self-{conf} || {};
my $scores = $conf-{scores} || {};
my $ans = {};
foreach my $hit ($self-get_names_of_tests_hit()) {
# Not really sure if defaulting to 0 is correct below...
$ans-{$hit} = (exists($scores-{$hit})
defined($scores-{$hit})) ? $scores-{$hit} : 0;
}
return $ans
}
#==
which returns a hashref of testname = score
Regards,
David.
Re: Spamassassin Rule Scores
Kevin A. McGrail skrev den 2014-04-24 16:50: Always love people submitting patches! or opensource with think like: make syslog plugin, make it basicly same options as in add_header just for sysloging then each admin can make there own login format based on same tags as in add_header i would love to see this happend aswell, i just cant make the perl module :(
Re: Spamassassin Rule Scores
On 4/24/2014 11:06 AM, Benny Pedersen wrote: Kevin A. McGrail skrev den 2014-04-24 16:50: Always love people submitting patches! or opensource with think like: make syslog plugin, make it basicly same options as in add_header just for sysloging then each admin can make there own login format based on same tags as in add_header i would love to see this happend aswell, i just cant make the perl module :( Sounds like overkill without enough demand, personally...
Re: Spamassassin Rule Scores
Kevin A. McGrail skrev den 2014-04-24 17:12: Sounds like overkill without enough demand, personally... only thing i see here is that you writed to me personally
Re: Spamassassin Rule Scores
On 4/24/2014 11:28 AM, Benny Pedersen wrote: Kevin A. McGrail skrev den 2014-04-24 17:12: Sounds like overkill without enough demand, personally... only thing i see here is that you writed to me personally Ahhh... Well not sure it warrants going on the list. I like the idea but need someone to step forward and implement it and I don't think enough will care.
Feature question
I'm hope I'm asking this in the right place. I'm wanting to replace a useless Sonicwall ES300 device so I'd like to know if SpamAssassin has the ability to store spam and send a daily email style report to users. That's the one major feature my people would miss. Thanks for any help.
Re: Spamassassin Rule Scores
Kevin A. McGrail skrev den 2014-04-24 17:30: Ahhh... Well not sure it warrants going on the list. I like the idea but need someone to step forward and implement it and I don't think enough will care. replyed private lots of idears, no coders :(
Re: Spamassassin Rule Scores
On 4/24/2014 11:37 AM, Benny Pedersen wrote: Kevin A. McGrail skrev den 2014-04-24 17:30: Ahhh... Well not sure it warrants going on the list. I like the idea but need someone to step forward and implement it and I don't think enough will care. replyed private lots of idears, no coders :( Exactly. -- *Kevin A. McGrail* President Peregrine Computer Consultants Corporation 3927 Old Lee Highway, Suite 102-C Fairfax, VA 22030-2422 http://www.pccc.com/ 703-359-9700 x50 / 800-823-8402 (Toll-Free) 703-359-8451 (fax) kmcgr...@pccc.com mailto:kmcgr...@pccc.com
Re: Feature question
So are you only looking for AntiSpam appliance or a UTM kindaa stuff? On Thu, Apr 24, 2014 at 9:11 PM, Greg Ledford gledf...@phhwtechnology.comwrote: I'm hope I'm asking this in the right place. I'm wanting to replace a useless Sonicwall ES300 device so I'd like to know if SpamAssassin has the ability to store spam and send a daily email style report to users. That's the one major feature my people would miss. Thanks for any help.
Re: Feature question
Greg Ledford skrev den 2014-04-24 17:41: I'm hope I'm asking this in the right place. I'm wanting to replace a useless Sonicwall ES300 device so I'd like to know if SpamAssassin has the ability to store spam and send a daily email style report to users. That's the one major feature my people would miss. Thanks for any help. sure, google amavisd-new mailzu, just the loging part, not sql quarantine, then google quartine report, with a link to mailzu http://www.maiamailguard.com/maia/wiki/process-quarantine.pl
RE: Feature question
A UTM type setup would be ideal but mostly anti-spam. I use it as a front end for Exchange. Greg Ledford PHHW Technology Services LLC 1000 Corporate Centre Dr, Ste 200 Franklin, TN 37067 Office (615) 778-1777 Fax (615) 771-0081 gledf...@phhwtechnology.com -Original Message- From: Blason R [blaso...@gmail.com] Received: Thursday, 24 Apr 2014, 10:52am To: Greg Ledford [gledf...@phhwtechnology.com] CC: users@spamassassin.apache.org [users@spamassassin.apache.org] Subject: Re: Feature question So are you only looking for AntiSpam appliance or a UTM kindaa stuff? On Thu, Apr 24, 2014 at 9:11 PM, Greg Ledford gledf...@phhwtechnology.commailto:gledf...@phhwtechnology.com wrote: I'm hope I'm asking this in the right place. I'm wanting to replace a useless Sonicwall ES300 device so I'd like to know if SpamAssassin has the ability to store spam and send a daily email style report to users. That's the one major feature my people would miss. Thanks for any help.
RE: Feature question
I'll check this out. I really appreciate it. I'm so tired of spending a fortune on something that isn't functional or configurable. Greg Ledford PHHW Technology Services LLC 1000 Corporate Centre Dr, Ste 200 Franklin, TN 37067 Office (615) 778-1777 Fax (615) 771-0081 gledf...@phhwtechnology.com -Original Message- From: Benny Pedersen [m...@junc.eu] Received: Thursday, 24 Apr 2014, 10:54am To: users@spamassassin.apache.org [users@spamassassin.apache.org] Subject: Re: Feature question Greg Ledford skrev den 2014-04-24 17:41: I'm hope I'm asking this in the right place. I'm wanting to replace a useless Sonicwall ES300 device so I'd like to know if SpamAssassin has the ability to store spam and send a daily email style report to users. That's the one major feature my people would miss. Thanks for any help. sure, google amavisd-new mailzu, just the loging part, not sql quarantine, then google quartine report, with a link to mailzu http://www.maiamailguard.com/maia/wiki/process-quarantine.pl
Re: Feature question
Try out mailcleaner!!! Its nice and provides all the feature On Thu, Apr 24, 2014 at 9:25 PM, Greg Ledford gledf...@phhwtechnology.comwrote: A UTM type setup would be ideal but mostly anti-spam. I use it as a front end for Exchange. Greg Ledford PHHW Technology Services LLC 1000 Corporate Centre Dr, Ste 200 Franklin, TN 37067 Office (615) 778-1777 Fax (615) 771-0081 gledf...@phhwtechnology.com -Original Message- *From:* Blason R [blaso...@gmail.com] *Received:* Thursday, 24 Apr 2014, 10:52am *To:* Greg Ledford [gledf...@phhwtechnology.com] *CC:* users@spamassassin.apache.org [users@spamassassin.apache.org] *Subject:* Re: Feature question So are you only looking for AntiSpam appliance or a UTM kindaa stuff? On Thu, Apr 24, 2014 at 9:11 PM, Greg Ledford gledf...@phhwtechnology.com wrote: I'm hope I'm asking this in the right place. I'm wanting to replace a useless Sonicwall ES300 device so I'd like to know if SpamAssassin has the ability to store spam and send a daily email style report to users. That's the one major feature my people would miss. Thanks for any help.
Re: confirm unsubscribe from users@spamassassin.apache.org
On Thu, 2014-04-24 at 15:50 +, users-h...@spamassassin.apache.org wrote: Hi! This is the ezmlm program. I'm managing the users@spamassassin.apache.org mailing list. To confirm that you would like skenn...@office.vcn.com removed from the users mailing list, please send a short reply to this address: users-uc.1398354644.pjnddmmkfjchkimbeenb-skennedy=office.vcn@spamassassin.apache.org Usually, this happens when you just hit the reply button. If this does not work, simply copy the address and paste it into the To: field of a new message. I haven't checked whether your address is currently on the mailing list. To see what address you used to subscribe, look at the messages you are receiving from the mailing list. Each message has your address hidden inside its return path; for example, m...@xdd.ff.com receives messages with return path: users-return-number-mary=xdd.ff@spamassassin.apache.org. Some mail programs are broken and cannot handle long addresses. If you cannot reply to this request, instead send a message to users-requ...@spamassassin.apache.org and put the entire address listed above into the Subject: line. --- Administrative commands for the users list --- I can handle administrative requests automatically. Please do not send them to the list address! Instead, send your message to the correct command address: To subscribe to the list, send a message to: users-subscr...@spamassassin.apache.org To remove your address from the list, send a message to: users-unsubscr...@spamassassin.apache.org Send mail to the following for info and FAQ for this list: users-i...@spamassassin.apache.org users-...@spamassassin.apache.org Similar addresses exist for the digest list: users-digest-subscr...@spamassassin.apache.org users-digest-unsubscr...@spamassassin.apache.org To get messages 123 through 145 (a maximum of 100 per request), mail: users-get.123_...@spamassassin.apache.org To get an index with subject and author for messages 123-456 , mail: users-index.123_...@spamassassin.apache.org They are always returned as sets of 100, max 2000 per request, so you'll actually get 100-499. To receive all messages with the same subject as message 12345, send a short message to: users-thread.12...@spamassassin.apache.org The messages should contain one line or word of text to avoid being treated as sp@m, but I will ignore their content. Only the ADDRESS you send to is important. You can start a subscription for an alternate address, for example john@host.domain, just add a hyphen and your address (with '=' instead of '@') after the command word: users-subscribe-john=host.dom...@spamassassin.apache.org To stop subscription for this address, mail: users-unsubscribe-john=host.dom...@spamassassin.apache.org In both cases, I'll send a confirmation message to that address. When you receive it, simply reply to it to complete your subscription. If despite following these instructions, you do not get the desired results, please contact my owner at users-ow...@spamassassin.apache.org. Please be patient, my owner is a lot slower than I am ;-) --- Enclosed is a copy of the request I received. Return-Path: skenn...@office.vcn.com Received: (qmail 30929 invoked by uid 99); 24 Apr 2014 15:50:44 - Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 24 Apr 2014 15:50:44 + X-ASF-Spam-Status: No, hits=-14.7 required=10.0 tests=ASF_EMPTY_LIST_OPS,ASF_LIST_OPS,ASF_LIST_UNSUB_A,EMPTY_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of skenn...@office.vcn.com designates 209.193.90.171 as permitted sender) Received: from [209.193.90.171] (HELO thor.geekdom.vcn.com) (209.193.90.171) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 24 Apr 2014 15:50:39 + Received: from [192.168.1.250] (hannah.geekdom.vcn.com [192.168.1.250]) by thor.geekdom.vcn.com (Postfix) with ESMTP id 9DD371ACB01A1 for users-unsubscr...@spamassassin.apache.org; Thu, 24 Apr 2014 09:50:16 -0600 (MDT) Message-ID: 1398354618.18885.19.camel@hannah Subject: From: Sean Kennedy skenn...@office.vcn.com To: users-unsubscr...@spamassassin.apache.org Date: Thu, 24 Apr 2014 09:50:18 -0600 Content-Type: text/plain X-Mailer: Evolution 3.10.4-0ubuntu1 Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org
Re: Feature question
On 04/24/2014 05:41 PM, Greg Ledford wrote: I'm hope I'm asking this in the right place. I'm wanting to replace a useless Sonicwall ES300 device so I'd like to know if SpamAssassin has the ability to store spam and send a daily email style report to users. That's the one major feature my people would miss. Thanks for any help. If you want well supported and efficient packages which use SA I can recommend Can-IT by http://www.roaringpenguin.com/ Can-IT BarricadeMX by http://fsl.com/
Re: Feature question
I'm hope I'm asking this in the right place. I'm wanting to replace a useless Sonicwall ES300 device so I'd like to know if SpamAssassin has the ability to store spam and send a daily email style report to users. That's the one major feature my people would miss. Thanks for any help. SpamAssassin doesn't, it merely classifies messages. But you can do what you want with various different glue packages. I use MailScanner with MailWatch to send a daily spam quarantine report to users, so they can see a list of all the messages (in increasing spam score) that have been quarantined at the server. Anthony -- www.fonant.com - Quality web sites Tel. 01903 867 810 Fonant Ltd is registered in England and Wales, company No. 7006596 Registered office: Amelia House, Crescent Road, Worthing, West Sussex, BN11 1QR
Re: SA 3.4 'make test' fails in 't/sa_sompile.t' with Not found: able-to-use
On 4/24/2014 9:47 AM, Kevin A. McGrail wrote: Bizarre... The non-replicatable behavior is very confusing. And you were able to replicate this on a modern CentOS box? Yep. Even tried it on another system that's running Centos 5. OK, I just tried it on yet another box ... this one is more 'virgin', as it's ONLY used as a XEN virtual host. This time all the tests passed. So it's got to be something in the other systems configuration or environment ... but the question is what. It's clearly focused on the able-to-use test. Are there any external dependencies related to that test? Maybe some of my modules are out of date need to be updated? I did have a bunch of modules that wanted upgrades, so I upgraded a handful of them but no difference. david -- IBM i on Power Systems: For when you can't afford to be out of business! I'm riding a metric century (100 km / 62 miles) in the 2014 Chicagoland Tour de Cure to raise money for diabetes research, education, and advocacy. Sponsor me by visiting http://email.diabetessucks.net. Any amount is appreciated. See where I get my donations from ... visit http://email.diabetessucks.net/mapdonations.php for an interactive map (it's a geeky thing).
Re: SA 3.4 'make test' fails in 't/sa_sompile.t' with Not found: able-to-use
On 4/24/2014 12:12 PM, Gibbs, David wrote: On 4/24/2014 9:47 AM, Kevin A. McGrail wrote: Bizarre... The non-replicatable behavior is very confusing. And you were able to replicate this on a modern CentOS box? Yep. Even tried it on another system that's running Centos 5. OK, I just tried it on yet another box ... this one is more 'virgin', as it's ONLY used as a XEN virtual host. This time all the tests passed. So it's got to be something in the other systems configuration or environment ... but the question is what. It's clearly focused on the able-to-use test. Are there any external dependencies related to that test? Maybe some of my modules are out of date need to be updated? I did have a bunch of modules that wanted upgrades, so I upgraded a handful of them but no difference. That's a test that basically tests if the compilation worked. On the sa_compile.t, there is a line that says: system_or_die $instdir/foo/$temp_binpath/sa-compile --keep-tmps; # --debug Can you turn on --debug and run prove -v t/sa_compile again. Then hopefully sa-compile throws a hint...
Re: sa-learn from a cronjob?
On Thu, 24 Apr 2014 15:07:32 +0100 RW rwmailli...@googlemail.com wrote: RW I don't think it will work for the purpose mentioned, and if it's RW working properly for you, there's a lot you're not mentioning. RW It's only looking for mail in the immediate post-delivery state RW after it's been put into the mailbox by an MTA or MDA and before RW it's been detected as new mail by an MUA (directly or via IMAP). It RW wont learn mail put into the folders by an MUA or IMAP at all. RW You need to use separate destination mailboxes. These are _not_ general purpose Maildirs. The normal mail processing pipe (MTA - LDA - IMAP - MUA) knows nothing about them. To mark something as spam/ham, a user (me) executes a custom macro in the MUA which pipes the message through the safecat command to deliver it explicitly to one of these directories. Basically, Maildir is just a convenient container format here. It could be a database or whatever. Does that answer your objections? -- Please *no* private copies of mailing list or newsgroup messages. gpg public key: 2048R/984A8AE4 fingerprint: 7953 ADA1 0E8E AB57 FB79 FFD2 360A 88B2 984A 8AE4 Funny pic: http://bit.ly/ZNE2MX
Re: false positives by FREEMAIL_FORGED_REPLYTO
Interesting, thanks for pointing it out This syntax has been used in a while by some other software, like JIRA, RT, … so not something new. In general, I would say spamassassin needs a few extra rules to now handle domain reputation/blocking (as it seems this is where we are going), I even found some rules are not IPv6 aware. I’m looking for some free time to write such rules and provide them to the community, I think the focus was helping mailman first. ;) On Apr 24, 2014, at 3:52 AM, Michael Storz michael.st...@lrz.de wrote: Since Yahoo and AOL have moved to a DMARC policy of reject, mail senders are changing the way they are sending their emails. Instead of using the email address of an user in RFC5322.From they use their own address and put the address of the user in the Reply-To field. FREEMAIL_FORGED_REPLYTO fires on these emails and produce false positives. From examples taken from log lines of amavisd: From: GIVENNAME_SURNAME_via_LinkedIn_mem...@linkedin.com (dkim:AUTHOR) From: NAME_via_Dropbox_no-re...@dropbox.com (dkim:AUTHOR) Since more and more such emails will occur, for example all web forms will send their emails in this way, the rule does not make sense anymore. -- Michael signature.asc Description: Message signed with OpenPGP using GPGMail
Re: SA 3.4 'make test' fails in 't/sa_sompile.t' with Not found: able-to-use
On 4/24/2014 11:25 AM, Kevin A. McGrail wrote: On the sa_compile.t, there is a line that says: system_or_die $instdir/foo/$temp_binpath/sa-compile --keep-tmps; # --debug Can you turn on --debug and run prove -v t/sa_compile again. Then hopefully sa-compile throws a hint... Nothing useful that I can see :( https://qtemp.net/sa-compile-test-fail-log-1.txt david -- IBM i on Power Systems: For when you can't afford to be out of business! I'm riding a metric century (100 km / 62 miles) in the 2014 Chicagoland Tour de Cure to raise money for diabetes research, education, and advocacy. Sponsor me by visiting http://email.diabetessucks.net. Any amount is appreciated. See where I get my donations from ... visit http://email.diabetessucks.net/mapdonations.php for an interactive map (it's a geeky thing).
Re: SA 3.4 'make test' fails in 't/sa_sompile.t' with Not found: able-to-use
On 4/24/2014 12:40 PM, Gibbs, David wrote: On 4/24/2014 11:25 AM, Kevin A. McGrail wrote: On the sa_compile.t, there is a line that says: system_or_die $instdir/foo/$temp_binpath/sa-compile --keep-tmps; # --debug Can you turn on --debug and run prove -v t/sa_compile again. Then hopefully sa-compile throws a hint... Nothing useful that I can see :( https://qtemp.net/sa-compile-test-fail-log-1.txt But now you appear to be failing Checking FOO not the able-to-use Can you try this as root?
Re: SA 3.4 'make test' fails in 't/sa_sompile.t' with Not found: able-to-use
On 4/24/2014 11:59 AM, Kevin A. McGrail wrote: https://qtemp.net/sa-compile-test-fail-log-1.txt But now you appear to be failing Checking FOO not the able-to-use That's what has been failing from the beginning, as far as I can tell. Can you try this as root? Yes, and it worked. Permissions problem then? david -- IBM i on Power Systems: For when you can't afford to be out of business! I'm riding a metric century (100 km / 62 miles) in the 2014 Chicagoland Tour de Cure to raise money for diabetes research, education, and advocacy. Sponsor me by visiting http://email.diabetessucks.net. Any amount is appreciated. See where I get my donations from ... visit http://email.diabetessucks.net/mapdonations.php for an interactive map (it's a geeky thing).
Re: Spamassassin Rule Scores
On 4/24/2014 11:04 AM, David F. Skoll wrote:
We integrate with SpamAssassin at the Perl library level, and we reach
into the innards to get at the test scores. Here's our code:
my $conf = $sa_status-{conf} || {};
my $scores = $conf-{scores} || {};
my $testnames = join(',', (map { (exists($scores-{$_}) defined($scores-{$_})) ? $_
. ':' . $scores-{$_} : $_:? } (split(/\s*,\s*/, $sa_status-get_names_of_tests_hit();
We end up getting output like this in our logs:
HTML_IMAGE_ONLY_20:0.7;HTML_MESSAGE:0.001;MIME_HTML_ONLY:1.105;RDNS_NONE:1.274;TO_EQ_FM_DOM_HTML_ONLY:0.489;TO_EQ_FM_HTML_ONLY:0.036;TO_NO_BRKTS_NORDNS:0.001;T_REMOTE_IMAGE:0.01
Could we make an official API call something like this?
Keeping more in the like and kind with the existing code in PMS,
wouldn't this be closer to what you need? Completely untested but
passed tests.
I debated making the routine get_names_of_tests_hit_with_scores call
get_names_of_tests_hit_with_scores to unify things a small bit but then
I would need to convert to an array and sort. Alternately, perhaps I
could use wantarray and combine the two functions.
Once this is done, it lays the groundwork to put
get_names_of_tests_hit_with_scores into spamd.
Index: lib/Mail/SpamAssassin/PerMsgStatus.pm
===
--- lib/Mail/SpamAssassin/PerMsgStatus.pm (revision 1587672)
+++ lib/Mail/SpamAssassin/PerMsgStatus.pm (working copy)
@@ -727,6 +727,55 @@
return join(',', sort(@{$self-{test_names_hit}}));
}
+=item $list = $status-get_names_of_tests_hit_with_scores_hash ()
+
+After a mail message has been checked, this method can be called. It will
+return a pointer to a hash for rule score pairs for all the symbolic
+test names and individual scores of the tests which were trigged by the
mail.
+
+=cut
+sub get_names_of_tests_hit_with_scores_hash {
+ my ($self) = @_;
+
+ my ($line, %testsscores);
+
+ #BASED ON CODE FOR TESTSSCORES TAG - KAM 2014-04-24
+ foreach my $test (@{$self-{test_names_hit}}) {
+my $score = $self-{conf}-{scores}-{$test};
+$score = '0' if !defined $score;
+
+$testsscores{$test} = $score;
+ }
+
+ return \%testsscores;
+}
+
+=item $list = $status-get_names_of_tests_hit_with_scores ()
+
+After a mail message has been checked, this method can be called. It will
+return a comma-separated string of rule=score pairs for all the symbolic
+test names and individual scores of the tests which were trigged by the
mail.
+
+=cut
+sub get_names_of_tests_hit_with_scores {
+ my ($self) = @_;
+
+ my ($line, %testsscores);
+
+ #BASED ON CODE FOR TESTSSCORES TAG - KAM 2014-04-24
+ foreach my $test (sort @{$self-{test_names_hit}}) {
+my $score = $self-{conf}-{scores}-{$test};
+$score = '0' if !defined $score;
+$line .= ',' if $line ne '';
+$line .= $test . '=' . $score;
+ }
+
+ $line ||= 'none';
+
+ return $line;
+}
+
+
###
Regards,
KAM
Re: Spamassassin Rule Scores
On Thu, 24 Apr 2014 13:28:56 -0400 Kevin A. McGrail kmcgr...@pccc.com wrote: Keeping more in the like and kind with the existing code in PMS, wouldn't this be closer to what you need? Completely untested but passed tests. Sure... I'm not picky about how it's implemented. ;) Just so long as there's an official API to get the test names and scores, I'm happy. Regards, David.
Re: SA 3.4 'make test' fails in 't/sa_sompile.t' with Not found: able-to-use
On 4/24/2014 1:08 PM, Gibbs, David wrote: On 4/24/2014 11:59 AM, Kevin A. McGrail wrote: https://qtemp.net/sa-compile-test-fail-log-1.txt But now you appear to be failing Checking FOO not the able-to-use That's what has been failing from the beginning, as far as I can tell. The subject of your email chain led me otherwise ;-) Can you try this as root? Yes, and it worked. Permissions problem then? Not sure. The test does not appear to run well unless root for me either. Perhaps we need to move this to root_sa_compile.t and move it to the run_root_tests? Or add a check for root? Overall, it should let you know you can proceed.
Re: Spamassassin Rule Scores
On 4/24/2014 1:34 PM, David F. Skoll wrote: On Thu, 24 Apr 2014 13:28:56 -0400 Kevin A. McGrail kmcgr...@pccc.com wrote: Keeping more in the like and kind with the existing code in PMS, wouldn't this be closer to what you need? Completely untested but passed tests. Sure... I'm not picky about how it's implemented. ;) Just so long as there's an official API to get the test names and scores, I'm happy. It's in trunk if you want to test it and give feedback, that would be helpful. From a quick basic print of output, looks sane. svn commit -m 'added get_names_of_tests_hit_with_scores_hash, get_names_of_tests_hit_with_scores functions to PMS along with trivial fixing of triggered being misspelled.' Sendinglib/Mail/SpamAssassin/PerMsgStatus.pm Transmitting file data . Committed revision 1589804. regards, KAM
Re: Spamassassin Rule Scores
On 04/24/2014 07:44 PM, Kevin A. McGrail wrote: On 4/24/2014 1:34 PM, David F. Skoll wrote: On Thu, 24 Apr 2014 13:28:56 -0400 Kevin A. McGrail kmcgr...@pccc.com wrote: Keeping more in the like and kind with the existing code in PMS, wouldn't this be closer to what you need? Completely untested but passed tests. Sure... I'm not picky about how it's implemented. ;) Just so long as there's an official API to get the test names and scores, I'm happy. It's in trunk if you want to test it and give feedback, that would be helpful. From a quick basic print of output, looks sane. svn commit -m 'added get_names_of_tests_hit_with_scores_hash, get_names_of_tests_hit_with_scores functions to PMS along with trivial fixing of triggered being misspelled.' Sendinglib/Mail/SpamAssassin/PerMsgStatus.pm Transmitting file data . Committed revision 1589804. regards, KAM IMO this should be configurable as it could break stats/loggers/etc
Re: sa-learn from a cronjob?
On Thu, 24 Apr 2014 09:29:21 -0700 Ian Zimmerman wrote: On Thu, 24 Apr 2014 15:07:32 +0100 RW rwmailli...@googlemail.com wrote: RW I don't think it will work for the purpose mentioned, and if it's RW working properly for you, there's a lot you're not mentioning. RW It's only looking for mail in the immediate post-delivery state RW after it's been put into the mailbox by an MTA or MDA and before RW it's been detected as new mail by an MUA (directly or via IMAP). RW It wont learn mail put into the folders by an MUA or IMAP at all. RW You need to use separate destination mailboxes. These are _not_ general purpose Maildirs. The normal mail processing pipe (MTA - LDA - IMAP - MUA) knows nothing about them. To mark something as spam/ham, a user (me) executes a custom macro in the MUA which pipes the message through the safecat command to deliver it explicitly to one of these directories. You might have mentioned that because it means it's not the solution you implied when you wrote Here is my cronjob for that purpose. It's certainly not appropriate to users that don't like the command line. Basically, Maildir is just a convenient container format here. It could be a database or whatever. Does that answer your objections? A Maildir isn't any more convenient than two simple directories. It doesn't really matter if you are the only user, but in general putting a Maildir that mustn't be opened in home directories wouldn't be a very good idea.
Re: Spamassassin Rule Scores
On 4/24/2014 2:14 PM, Axb wrote: IMO this should be configurable as it could break stats/loggers/etc The change right now just adds additional API functions. Nothing uses them. spamd could be configured to use them and should be a configuration option, I agree.
Re: SA 3.4 'make test' fails in 't/sa_sompile.t' with Not found: able-to-use
On 4/24/2014 12:36 PM, Kevin A. McGrail wrote: Overall, it should let you know you can proceed. Kevin: Thanks for your help. Got the update installed running fine now. david -- IBM i on Power Systems: For when you can't afford to be out of business! I'm riding a metric century (100 km / 62 miles) in the 2014 Chicagoland Tour de Cure to raise money for diabetes research, education, and advocacy. Sponsor me by visiting http://email.diabetessucks.net. Any amount is appreciated. See where I get my donations from ... visit http://email.diabetessucks.net/mapdonations.php for an interactive map (it's a geeky thing).
Re: sa-learn from a cronjob?
RW wrote:
Ian Zimmerman wrote:
RW wrote:
RW I don't think it will work for the purpose mentioned, and if it's
RW working properly for you, there's a lot you're not mentioning.
I looked at the script and it looks like an example that would work
for Ian fine. There are some points of shell programming style that I
would like to avoid seeing propagated in an example though. :-) But I
think that it is great that Ian shared his script just the same. This
is one of those things where if ten of us showed all of our working
examples that we would have 12 different scripts.
The biggest thing that hurts Ian's script as a general example is that
it is using ssh to connect to the server running spamassassin. Most
developers use ssh every day and so that is very normal. But most of
the masses of email users will not be in a position to use ssh
effectively. A mail adminstrator would be able to see the example for
what it is and then write that part differently though.
RW It's only looking for mail in the immediate post-delivery state
RW after it's been put into the mailbox by an MTA or MDA and before
RW it's been detected as new mail by an MUA (directly or via IMAP).
RW It wont learn mail put into the folders by an MUA or IMAP at all.
No. That isn't what the script is doing.
The script is looping through mail files in a maildir and processing
them remotely on the server through sa-learn. After processing the
messages it is moving the messages to mark them as having been read.
The script is obviously meant to be run periodically by cron. At that
time it will walk through every message that has been stored into the
ham and spam mailboxes. A user would only need to store the message
into the appropriate mailbox. A spam message into the spam mailbox
and then later in the background the cron task will send the spam
message through sa-learn --spam for learning. Same for --ham. The
script is fairly obvious, straight forward, and brute force.
RW You need to use separate destination mailboxes.
These are _not_ general purpose Maildirs. The normal mail processing
pipe (MTA - LDA - IMAP - MUA) knows nothing about them. To mark
something as spam/ham, a user (me) executes a custom macro in the MUA
which pipes the message through the safecat command to deliver it
explicitly to one of these directories.
You might have mentioned that because it means it's not the solution you
implied when you wrote Here is my cronjob for that purpose. It's
certainly not appropriate to users that don't like the command line.
Sorry but you are incorrect. Users of Ian's system need not use the
command line. His solution directly answered the Dan's question.
Dan Mahoney wrote:
I'd like to basically have my IMAP server default to handing out two
imap mailboxes that get auto-crontabbed to training bayes.
Ian Zimmerman wrote:
Here is my cronjob for that purpose, in its entirety. Note that
each of ~/spam-corpora{ham,spam} is a Maildir. There is a small
race condition between the sa-learn run and the move to cur, which
wasn't worth fixing in my case; if you use this and fix it let me
know :)
Which is exactly what his script does. (I don't like the
implementation as written because the shell scripting has some rough
spots. But...)
Basically, Maildir is just a convenient container format here. It
could be a database or whatever.
Does that answer your objections?
A Maildir isn't any more convenient than two simple directories. It
doesn't really matter if you are the only user, but in general putting
a Maildir that mustn't be opened in home directories wouldn't be a
very good idea.
I am having a hard time understanding what you are objecting to here.
Dan was the one with the question. Ian shared something that would do
the task. It looks like you are having a hard time understanding how
this worked. If so then please ask questions so as to understand it.
It doesn't make sense to gripe about it without reason. Sharing and
commenting and peer review and iterating a solution and improving it
is how community efforts work and succeed and grow.
Your comment that a maildir isn't better than two simple directories
implies that you are not familiar with the maildir mailbox format.
Maildir is an ad-hoc standard mailbox format used by most imap
servers. Using maildir mailboxes would definitely be better than
using two simple directories. Standard is better than better!
There isn't any reason that it mustn't be opened. In fact the
opposite. The user must be able to open the mailbox and must be able
to save misclassified messages there for learning. If they do that by
mistake then they can pull the message back out before the crontask
runs. (That timing is one of my issues with the script that I would
want to see improved.)
Using a maildir for these two purposes makes a lot of sense. The user
reading email using any of the popular ways to read email these days
then can
Re: sa-learn from a cronjob?
Ian Zimmerman wrote:
Here is my cronjob for that purpose, in its entirety. Note that each of
~/spam-corpora{ham,spam} is a Maildir. There is a small race condition
between the sa-learn run and the move to cur, which wasn't worth fixing
in my case; if you use this and fix it let me know :)
I looked over your script. I think the use of the ssh for remote
processing will probably make it less available to most people. You
might consider setting up spamd and spamc for this purpose instead.
Also, to give people a known time to react to mistakes it is nice to
not process email immediately but to specify some time such as five
minutes after saving it or some such. I use find with a ! -newerct 5
minutes ago to process messages older than five minutes. That way if
I save something by mistake I have a few minutes to react and remove
the message from the learning.
Instead of mv I have used safecat for moving messages around. And
generally I avoid worrying about whitespace in filenames for this
since I am guaranteed the file names are well formed without any
whitespace.
Instead of:
for m in `ls ~/spam-corpora/${food}/new` ; do
cat ~/spam-corpora/${food}/new/${m} | formail
done | ssh $server sa-learn --${food} --mbox -
I would suggest something more along the lines of this different and
not not equivalent but similar script.
cd $MAILBOXDIR || exit 1
for f in $(find spam-new/new spam-new/cur -ignore_readdir_race -type f !
-newerct 6 minutes ago -print); do
spamc -x -d $server --learntype=spam $f
rc=$?
if [ $rc -eq 0 ] || [ $rc -eq 98 ]; then
# rc=98: This appears to be the return (undocumented) when spamc
# can't learn the message because it is already learned. The
# docs say that EX_TOOBIG 98 is not otherwise used.
if safecat spam/tmp spam/cur $f /dev/null; then
rm -f $f
fi
else
echo sa-learn failed $rc on $f
fi
done
Perhaps the comments about spamc return code 98 would cause someone
here to look at that part of the code. It has been years since I put
in that comment. Perhaps it is even different now. Don't know.
I have thought about refactoring this into two scripts so that the
find could -exec the second. That would eliminate the for f in
arguments syntax which would save memory. But the memory use is small
for my case, I do not need to worry about filenames with whitespace,
and I like having one script instead of two so that I can see everything.
Something to think about. The above is not in its entirety because I
cut it down from a larger case that is doing other things. It would
need a little work. But it might give some ideas.
Bob
Re: confirm unsubscribe from users@spamassassin.apache.org
I do not want to unsubscribe. Quoting Sean Kennedy skenn...@office.vcn.com: On Thu, 2014-04-24 at 15:50 +, users-h...@spamassassin.apache.org wrote: Hi! This is the ezmlm program. I'm managing the users@spamassassin.apache.org mailing list. To confirm that you would like skenn...@office.vcn.com removed from the users mailing list, please send a short reply to this address: users-uc.1398354644.pjnddmmkfjchkimbeenb-skennedy=office.vcn@spamassassin.apache.org Usually, this happens when you just hit the reply button. If this does not work, simply copy the address and paste it into the To: field of a new message. I haven't checked whether your address is currently on the mailing list. To see what address you used to subscribe, look at the messages you are receiving from the mailing list. Each message has your address hidden inside its return path; for example, m...@xdd.ff.com receives messages with return path: users-return-number-mary=xdd.ff@spamassassin.apache.org. Some mail programs are broken and cannot handle long addresses. If you cannot reply to this request, instead send a message to users-requ...@spamassassin.apache.org and put the entire address listed above into the Subject: line. --- Administrative commands for the users list --- I can handle administrative requests automatically. Please do not send them to the list address! Instead, send your message to the correct command address: To subscribe to the list, send a message to: users-subscr...@spamassassin.apache.org To remove your address from the list, send a message to: users-unsubscr...@spamassassin.apache.org Send mail to the following for info and FAQ for this list: users-i...@spamassassin.apache.org users-...@spamassassin.apache.org Similar addresses exist for the digest list: users-digest-subscr...@spamassassin.apache.org users-digest-unsubscr...@spamassassin.apache.org To get messages 123 through 145 (a maximum of 100 per request), mail: users-get.123_...@spamassassin.apache.org To get an index with subject and author for messages 123-456 , mail: users-index.123_...@spamassassin.apache.org They are always returned as sets of 100, max 2000 per request, so you'll actually get 100-499. To receive all messages with the same subject as message 12345, send a short message to: users-thread.12...@spamassassin.apache.org The messages should contain one line or word of text to avoid being treated as sp@m, but I will ignore their content. Only the ADDRESS you send to is important. You can start a subscription for an alternate address, for example john@host.domain, just add a hyphen and your address (with '=' instead of '@') after the command word: users-subscribe-john=host.dom...@spamassassin.apache.org To stop subscription for this address, mail: users-unsubscribe-john=host.dom...@spamassassin.apache.org In both cases, I'll send a confirmation message to that address. When you receive it, simply reply to it to complete your subscription. If despite following these instructions, you do not get the desired results, please contact my owner at users-ow...@spamassassin.apache.org. Please be patient, my owner is a lot slower than I am ;-) --- Enclosed is a copy of the request I received. Return-Path: skenn...@office.vcn.com Received: (qmail 30929 invoked by uid 99); 24 Apr 2014 15:50:44 - Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 24 Apr 2014 15:50:44 + X-ASF-Spam-Status: No, hits=-14.7 required=10.0 tests=ASF_EMPTY_LIST_OPS,ASF_LIST_OPS,ASF_LIST_UNSUB_A,EMPTY_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of skenn...@office.vcn.com designates 209.193.90.171 as permitted sender) Received: from [209.193.90.171] (HELO thor.geekdom.vcn.com) (209.193.90.171) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 24 Apr 2014 15:50:39 + Received: from [192.168.1.250] (hannah.geekdom.vcn.com [192.168.1.250]) by thor.geekdom.vcn.com (Postfix) with ESMTP id 9DD371ACB01A1 for users-unsubscr...@spamassassin.apache.org; Thu, 24 Apr 2014 09:50:16 -0600 (MDT) Message-ID: 1398354618.18885.19.camel@hannah Subject: From: Sean Kennedy skenn...@office.vcn.com To: users-unsubscr...@spamassassin.apache.org Date: Thu, 24 Apr 2014 09:50:18 -0600 Content-Type: text/plain X-Mailer: Evolution 3.10.4-0ubuntu1 Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org -- This email was Virus checked by UTM 9. http://www.astaro.com
