Re: Program failure (69) of "spamc"
2014-09-10 23:25 Geoff Soper wrote:
Hi,
I'm calling spamc (3.3.2) from procmail as suggested at
https://wiki.apache.org/spamassassin/UsedViaProcmail
Having made a change to not use the lockfile (as suggsted in another
thread) I'm now getting the following error in my procmail log:
procmail: Program failure (69) of "spamc"
procmail: Rescue of unfiltered data succeeded
From Wed Sep 10 22:10:47 2014
Subject:
Folder: /var/qmail/mailnames/*
This persists even after SA has been restarted and the procmailrc file
restored to its original state.
Can anyone suggest what the issue may be?
According to spamc/libspamc.c the EX_UNAVAILABLE (code 69)
can be a result of any of the following conditions:
/*
* translate_connect_errno()
*
* Given a UNIX error number obtained (probably) from "connect(2)",
* translate this to a failure code. This module is shared by both
* transport modules - UNIX and TCP.
*
* This should ONLY be called when there is an error.
*/
static int _translate_connect_errno(int err)
{
switch (err) {
[...]
case ECONNREFUSED:
case ETIMEDOUT:
case ENETUNREACH:
return EX_UNAVAILABLE;
Mark
Program failure (69) of "spamc"
Hi, I'm calling spamc (3.3.2) from procmail as suggested at https://wiki.apache.org/spamassassin/UsedViaProcmail Having made a change to not use the lockfile (as suggsted in another thread) I'm now getting the following error in my procmail log: procmail: Program failure (69) of "spamc" procmail: Rescue of unfiltered data succeeded From Wed Sep 10 22:10:47 2014 Subject: Folder: /var/qmail/mailnames/* This persists even after SA has been restarted and the procmailrc file restored to its original state. Can anyone suggest what the issue may be? Thanks, Geoff
Re: bayes_auto_learn_threshold_nonspam
On Wed, 10 Sep 2014 20:57:35 +0200 Axb wrote: > > In practice this means that, without custom rules, ham can only be > > autolearned if it hits a DNS whitelist rule or RP_MATCHES_RCVD. > > > > from what I'm seeing is that it takes lower scored ham to autolearn > ham. I don't use DNS whitelists and RP_MATCHES_RCVD is disabled To reach -1.0 it has to hit some negative scoring rules. If you look at all the rules with negative scores and eliminate rules marked learn, noautolearn or userconf you have the rules listed below. score DCC_REPUT_13_19 0 -0.1 0 -0.1 score DKIMDOMAIN_IN_DWL 0 -3.5 0 -3.5 score DKIMDOMAIN_IN_DWL_UNKNOWN 0 -0.01 0 -0.01 score DKIM_VALID -0.1 score DKIM_VALID_AU -0.1 score RCVD_IN_DNSWL_HI 0 -5 0 -5 score RCVD_IN_DNSWL_LOW 0 -0.7 0 -0.7 score RCVD_IN_DNSWL_MED 0 -2.3 0 -2.3 score RCVD_IN_DNSWL_NONE 0 -0.0001 0 -0.0001 score RCVD_IN_IADB_DK 0 -0.223 0 -0.095 # n=0 n=1 n=2 score RCVD_IN_IADB_DOPTIN 0 -4 0 -4 score RCVD_IN_IADB_DOPTIN_LT50 0 -0.001 0 -0.001 # n=0 n=1 n=2 score RCVD_IN_IADB_LISTED 0 -0.380 0 -0.001 # n=0 n=2 score RCVD_IN_IADB_MI_CPR_MAT 0 -0.332 0 -0.000 # n=0 n=1 n=2 score RCVD_IN_IADB_ML_DOPTIN 0 -6 0 -6 score RCVD_IN_IADB_OPTIN 0 -2.057 0 -1.470 # n=0 n=1 n=2 score RCVD_IN_IADB_OPTIN_GT50 0 -1.208 0 -0.007 # n=0 n=2 score RCVD_IN_IADB_RDNS 0 -0.167 0 -0.235 # n=0 n=1 n=2 score RCVD_IN_IADB_SENDERID 0 -0.001 0 -0.001 # n=0 n=2 score RCVD_IN_IADB_SPF 0 -0.001 0 -0.059 # n=0 n=2 score RCVD_IN_IADB_UT_CPR_MAT 0 -0.095 0 -0.001 # n=0 n=1 n=2 score RCVD_IN_IADB_VOUCHED 0 -2.2 0 -2.2 score RCVD_IN_MSPIKE_H3 -0.01 score RCVD_IN_MSPIKE_H4 -0.01 score RCVD_IN_MSPIKE_H5 -1.0 score RCVD_IN_MSPIKE_WL -0.01 score RCVD_IN_MSPIKE_H2 0.001 -1.772 0.001 -1.772 score RCVD_IN_MSPIKE_H3 0.001 -0.010 0.001 -0.010 score RCVD_IN_MSPIKE_H4 0.001 -0.010 0.001 -0.010 score RCVD_IN_MSPIKE_H5 0.001 -1.000 0.001 -1.000 score RCVD_IN_MSPIKE_WL 0.001 -0.010 0.001 -0.010 score RCVD_IN_RP_CERTIFIED 0.0 -3.0 0.0 -3.0 score RCVD_IN_RP_SAFE 0.0 -2.0 0.0 -2.0 score RP_MATCHES_RCVD -1.302 -2.499 -1.302 -2.499
Re: MSPIKE in older SA ?
On 10. sep. 2014 19.58.18 "Kevin A. McGrail" wrote: I will say, I don't know how long sa-update will work for 3.3.2. Eventually, we have to move on and not support old releases though right now the status quo of trying our best is ok. 3.3.2 is still latest stable in gentoo, i have unofficial 3.4.1 in my overlay 3.4.0 is in my overlay, but still miss fixing dependics for redis
Re: bayes_auto_learn_threshold_nonspam
On 09/10/2014 08:23 PM, RW wrote: On Wed, 10 Sep 2014 15:47:48 +0200 Axb wrote: for quite a while I've been playing with autolearn settings SA's default is: bayes_auto_learn_threshold_nonspam0.1 this *can* cause low scored spam to be learnt as ham. For several months I've been using bayes_auto_learn_threshold_nonspam -1.0 and so far no more false negatives have been learnt as ham which is was hoping for. If you're using autolearn, you may want to play with that threshold.. In practice this means that, without custom rules, ham can only be autolearned if it hits a DNS whitelist rule or RP_MATCHES_RCVD. from what I'm seeing is that it takes lower scored ham to autolearn ham. I don't use DNS whitelists and RP_MATCHES_RCVD is disabled
Where spam comes from
http://qz.com/263013/for-390-you-can-buy-a-harvard-email-account-on-chinas-biggest-online-marketplace/ Most of the article is off topic, but I liked the mention of being able to buy *.edu email addresses. We see them from time to time, especially Harvard, and it always makes me wonder how much the universities know about this.
Re: bayes_auto_learn_threshold_nonspam
Hi, SA's default is: bayes_auto_learn_threshold_nonspam0.1 this *can* cause low scored spam to be learnt as ham. For several months I've been using bayes_auto_learn_threshold_nonspam -1.0 and so far no more false negatives have been learnt as ham which is was hoping for. If you're using autolearn, you may want to play with that threshold.. In practice this means that, without custom rules, ham can only be autolearned if it hits a DNS whitelist rule or RP_MATCHES_RCVD. I thought that might be the case, although wasn't exactly sure under what conditions that would happen. I've also set RP_MATCHES_RCVD to near zero these days because it was affecting too much spam. My concern was always with learning too much spam as ham, and nearly all of my ham is bayes00 already anyway... It's my spam that frequently has bayes50 or so, so it's that which I hope to improve... Thanks, Alex
Re: Possible pattern here?
John Hardin wrote: > Bob Proulx wrote: > >Is there a way to use this to create a SpamAssassin rule to try to > >catch this type of spam? > > Grab the RAND_HEADER rules (there are several related, get them all) from my > sandbox and score as you see fit. Ah... Already discussed earlier. Sorry for not having found that before. Thanks for the work on it. I will give those a try and see how they perform on this spam. It looks to me like they are hitting nicely. Thanks! Bob
Re: bayes_auto_learn_threshold_nonspam
Hi, For several months I've been using bayes_auto_learn_threshold_nonspam -1.0 Any reason you chose -1.0 rather than something a bit closer to 0, like -0.5 or -0.2? Most of my low-scoring spam is pretty close to 0, so I'm just wondering. I know I made the decision years ago to lower it to -1.0 just to be safe. My ham and spam bayes ratios remain pretty equal. Really, though, I had no real way to calculate what exactly the right value it should be other than seeing a sufficient number of nonspam that were lower than the default. I've currently got a dozen or so "help all my money's been stolen" fraud spam that's scoring at 0.6 :-( Thanks, Alex
Re: bayes_auto_learn_threshold_nonspam
On Wed, 10 Sep 2014 15:47:48 +0200 Axb wrote: > for quite a while I've been playing with autolearn settings > > SA's default is: > bayes_auto_learn_threshold_nonspam0.1 > > this *can* cause low scored spam to be learnt as ham. > > For several months I've been using > bayes_auto_learn_threshold_nonspam -1.0 > > and so far no more false negatives have been learnt as ham which is > was hoping for. > If you're using autolearn, you may want to play with that threshold.. In practice this means that, without custom rules, ham can only be autolearned if it hits a DNS whitelist rule or RP_MATCHES_RCVD.
Re: MSPIKE in older SA ?
On 9/10/2014 1:36 PM, Jesse Norell wrote: Would you consider changing the version check in official 20_mailspike.cf to allow 3.3.2 to use those by default? Jesse, For me, I am neutral on the matter as my energies are focused on 3.4.1 to release on 9/30. But this will need 3 explicit +1's from committers/PMC members because there is clear debate from prior that would have to be overridden. I will say, I don't know how long sa-update will work for 3.3.2. Eventually, we have to move on and not support old releases though right now the status quo of trying our best is ok. Regards, KAM
Re: MSPIKE in older SA ?
On Wed, 2014-09-10 at 13:10 -0400, Kevin A. McGrail wrote: > On 9/10/2014 12:59 PM, Jesse Norell wrote: > > Is there any reason you should not use MSPIKE in versions older than > > 3.4.0? > > > > Eg. on debian box with 3.3.2, I have 20_mailspike.cf; I commented the > > version check out, tested that spamassassin --lint was happy, recompiled > > rules and I now have MSPIKE rules hitting. Am I missing something? Or > > is that just there to try to persuade people towards upgrading versions? > > When Mailspike was evaluated, the project did not like the concept of > adding RBLs except during larger releases. > > However, I am not aware of a technical reason it will not work well with > 3.3.2. Would you consider changing the version check in official 20_mailspike.cf to allow 3.3.2 to use those by default? > There was no concern about pushing people to new versions though the > project did clarify both our release goals and how long we will support > an older release at http://wiki.apache.org/spamassassin/ReleaseGoals > > This means as of August 11th, 2014 3.3.2 is effectively unsupported. Pretty, please? :) Thanks, Jesse > Regards, > KAM -- Jesse Norell Kentec Communications, Inc. 970-522-8107 - www.kci.net
Re: Rule priority
On Wed, 10 Sep 2014, Philip Prindeville wrote: I ask because I’m trying to address this comment: https://issues.apache.org/SpamAssassin/show_bug.cgi?id=7060#c10 This might be better on the dev list. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- Tomorrow: the 13th anniversary of 9/11
Re: bayes_auto_learn_threshold_nonspam
On Sep 10, 2014, at 7:47 AM, Axb wrote: > For several months I've been using > bayes_auto_learn_threshold_nonspam -1.0 Any reason you chose -1.0 rather than something a bit closer to 0, like -0.5 or -0.2? Most of my low-scoring spam is pretty close to 0, so I'm just wondering. Thanks. --- Amir
Re: MSPIKE in older SA ?
On 9/10/2014 12:59 PM, Jesse Norell wrote: Is there any reason you should not use MSPIKE in versions older than 3.4.0? Eg. on debian box with 3.3.2, I have 20_mailspike.cf; I commented the version check out, tested that spamassassin --lint was happy, recompiled rules and I now have MSPIKE rules hitting. Am I missing something? Or is that just there to try to persuade people towards upgrading versions? When Mailspike was evaluated, the project did not like the concept of adding RBLs except during larger releases. However, I am not aware of a technical reason it will not work well with 3.3.2. There was no concern about pushing people to new versions though the project did clarify both our release goals and how long we will support an older release at http://wiki.apache.org/spamassassin/ReleaseGoals This means as of August 11th, 2014 3.3.2 is effectively unsupported. Regards, KAM
Rule priority
Is there a good discussion on how rule priority works, and short-circuited evaluation, etc? I must be looking in the wrong places because I can’t find much. I found register_method_priority() in ::Plugin but I wasn’t sure if that’s all there is… It only seems to be called in Plugin::Reuse::new() (well, you’d expect it in the constructor). Looking in the rules themselves, also, there aren’t that many rules which have an explicitly configured priority. I ask because I’m trying to address this comment: https://issues.apache.org/SpamAssassin/show_bug.cgi?id=7060#c10 but the source base doesn’t really contain a lot of useful examples. Thanks, -Philip
MSPIKE in older SA ?
Is there any reason you should not use MSPIKE in versions older than 3.4.0? Eg. on debian box with 3.3.2, I have 20_mailspike.cf; I commented the version check out, tested that spamassassin --lint was happy, recompiled rules and I now have MSPIKE rules hitting. Am I missing something? Or is that just there to try to persuade people towards upgrading versions? Thanks, -- Jesse Norell Kentec Communications, Inc. 970-522-8107 - www.kci.net
Re: RP_MATCHES_RCVD
On 9/5/2014 2:37 AM, Reindl Harald wrote: > Hi > > i got recently a clear spam message which would have > a score of 6.9 but RP_MATCHES_RCVD removed 1.7 points > > is that not a little too much? > This has been a problem for about 6 months now. I complained about it back in April 2014, and there was a much larger discussion back in Aug 2013. After the Aug 2013 discussion it was fixed, but then something broke it in Mar/Apr 2014.
Re: RCVD_IN_MSPIKE_* strange scoring
On Wed, 10 Sep 2014, Reindl Harald wrote: Am 10.09.2014 um 16:50 schrieb Jose Borges Ferreira: On Wed, Sep 10, 2014 at 12:22 PM, Reindl Harald wrote: something is here terrible wrong why does "average" is preferred over "excellent" why do H3 and H4 get a very less WL score? recently a clear spam message slipped by the -1.7 through describe RCVD_IN_MSPIKE_H2 Average reputation (+2) describe RCVD_IN_MSPIKE_H3 Good reputation (+3) describe RCVD_IN_MSPIKE_H4 Very Good reputation (+4) describe RCVD_IN_MSPIKE_H5 Excellent reputation (+5) score RCVD_IN_MSPIKE_H2 0.001 -1.772 0.001 -1.772 score RCVD_IN_MSPIKE_H3 0.001 -0.010 0.001 -0.010 score RCVD_IN_MSPIKE_H4 0.001 -0.010 0.001 -0.010 score RCVD_IN_MSPIKE_H5 0.001 -1.000 0.001 -1.000 __ Thats probably the QA system that scores that based on the available corpus data .. so received nightly with sa-update? Yep. It's possible that these scores should be static. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- How can you reason with someone who thinks we're on a glidepath to a police state and yet their solution is to grant the government a monopoly on force? They are insane. --- Tomorrow: the 13rd anniversary of 9/11
Re: RCVD_IN_MSPIKE_* strange scoring
Am 10.09.2014 um 16:50 schrieb Jose Borges Ferreira: > On Wed, Sep 10, 2014 at 12:22 PM, Reindl Harald > wrote: >> something is here terrible wrong >> >> why does "average" is preferred over "excellent" >> why do H3 and H4 get a very less WL score? >> recently a clear spam message slipped by the -1.7 through >> >> describe RCVD_IN_MSPIKE_H2 Average reputation (+2) >> describe RCVD_IN_MSPIKE_H3 Good reputation (+3) >> describe RCVD_IN_MSPIKE_H4 Very Good reputation (+4) >> describe RCVD_IN_MSPIKE_H5 Excellent reputation (+5) >> >> score RCVD_IN_MSPIKE_H2 0.001 -1.772 0.001 -1.772 >> score RCVD_IN_MSPIKE_H3 0.001 -0.010 0.001 -0.010 >> score RCVD_IN_MSPIKE_H4 0.001 -0.010 0.001 -0.010 >> score RCVD_IN_MSPIKE_H5 0.001 -1.000 0.001 -1.000 >> __ > > Thats probably the QA system that scores that based on the available > corpus data .. so received nightly with sa-update? >> i changed that in "local.cf" to the following >> >> score RCVD_IN_MSPIKE_H2 0.001 -0.5 0.001 -0.5 >> score RCVD_IN_MSPIKE_H3 0.001 -0.8 0.001 -0.8 >> score RCVD_IN_MSPIKE_H4 0.001 -1.1 0.001 -1.1 >> score RCVD_IN_MSPIKE_H5 0.001 -1.5 0.001 -1.5 > > That's the expected scoring distribution. We have in our system a more > generous scoring ranging from -0.5 to -3.5 -3.5 is very much - i saw many crap from even H5 listed servers they may lose that reputation as follow up but too late signature.asc Description: OpenPGP digital signature
Re: bayes_auto_learn_threshold_nonspam
On 09/10/2014 04:29 PM, Alex Regan wrote: Hi, For several months I've been using bayes_auto_learn_threshold_nonspam -1.0 and so far no more false negatives have been learnt as ham which is was hoping for. If you're using autolearn, you may want to play with that threshold.. Based on your expertise with Bayes, should we change the default for 3.4.1? I'm +1 for it. I'd really like some more ppl to test that and hear some feedback before we change any defaults. I't's nothing we can test via masschecks My nonspam threshold has been -1.0 for many years - since the first time I also saw low-scoring spam hit this value. I also have quite a few messages at -100.0 from whitelisting, but I somehow figured out long ago that they are exempt from being added to bayes, correct? correct! tflags USER_IN_WHITELISTuserconf nice noautolearn
Re: RCVD_IN_MSPIKE_* strange scoring
On Wed, Sep 10, 2014 at 12:22 PM, Reindl Harald wrote: > something is here terrible wrong > > why does "average" is preferred over "excellent" > why do H3 and H4 get a very less WL score? > recently a clear spam message slipped by the -1.7 through > > describe RCVD_IN_MSPIKE_H2 Average reputation (+2) > describe RCVD_IN_MSPIKE_H3 Good reputation (+3) > describe RCVD_IN_MSPIKE_H4 Very Good reputation (+4) > describe RCVD_IN_MSPIKE_H5 Excellent reputation (+5) > > score RCVD_IN_MSPIKE_H2 0.001 -1.772 0.001 -1.772 > score RCVD_IN_MSPIKE_H3 0.001 -0.010 0.001 -0.010 > score RCVD_IN_MSPIKE_H4 0.001 -0.010 0.001 -0.010 > score RCVD_IN_MSPIKE_H5 0.001 -1.000 0.001 -1.000 > __ Thats probably the QA system that scores that based on the available corpus data .. > i changed that in "local.cf" to the following > > score RCVD_IN_MSPIKE_H2 0.001 -0.5 0.001 -0.5 > score RCVD_IN_MSPIKE_H3 0.001 -0.8 0.001 -0.8 > score RCVD_IN_MSPIKE_H4 0.001 -1.1 0.001 -1.1 > score RCVD_IN_MSPIKE_H5 0.001 -1.5 0.001 -1.5 > That's the expected scoring distribution. We have in our system a more generous scoring ranging from -0.5 to -3.5
Re: bayes_auto_learn_threshold_nonspam
Hi, For several months I've been using bayes_auto_learn_threshold_nonspam -1.0 and so far no more false negatives have been learnt as ham which is was hoping for. If you're using autolearn, you may want to play with that threshold.. Based on your expertise with Bayes, should we change the default for 3.4.1? I'm +1 for it. I'd really like some more ppl to test that and hear some feedback before we change any defaults. I't's nothing we can test via masschecks My nonspam threshold has been -1.0 for many years - since the first time I also saw low-scoring spam hit this value. I also have quite a few messages at -100.0 from whitelisting, but I somehow figured out long ago that they are exempt from being added to bayes, correct? Thanks, Alex
Re: bayes_auto_learn_threshold_nonspam
On 09/10/2014 04:05 PM, Kevin A. McGrail wrote: On 9/10/2014 9:47 AM, Axb wrote: for quite a while I've been playing with autolearn settings SA's default is: bayes_auto_learn_threshold_nonspam0.1 this *can* cause low scored spam to be learnt as ham. For several months I've been using bayes_auto_learn_threshold_nonspam -1.0 and so far no more false negatives have been learnt as ham which is was hoping for. If you're using autolearn, you may want to play with that threshold.. Based on your expertise with Bayes, should we change the default for 3.4.1? I'm +1 for it. I'd really like some more ppl to test that and hear some feedback before we change any defaults. I't's nothing we can test via masschecks I will be rolling a release candidate soon so we can hopefully release on 9/30 per our schedule. Do you think we can include the TLD .cf stuff in that release? As from thsi week I've set calendar reminders to update/deply RegistrarBoundaries.pm every Sunday - boring to do and the rest of the world isn't getting them
Re: Whitelist one mail with multiple destinations
On Wed, 10 Sep 2014 10:59:16 -0300 "M. Rodrigo Monteiro" wrote: > > Option 2 is to accept the message unfiltered, split it into > > multiple copies, and remail each copy so it can be scanned > > per-recipient. > How can I do it? It depends on the MTA you're using. If you use one that supports milter, you can use MIMEDefang to do it. If you are processing the mail with procmail or some non-milter-supporting MTA, then I have no idea... you probably will have to write something custom to do it. Regards, David.
Re: bayes_auto_learn_threshold_nonspam
On 9/10/2014 9:47 AM, Axb wrote: for quite a while I've been playing with autolearn settings SA's default is: bayes_auto_learn_threshold_nonspam0.1 this *can* cause low scored spam to be learnt as ham. For several months I've been using bayes_auto_learn_threshold_nonspam -1.0 and so far no more false negatives have been learnt as ham which is was hoping for. If you're using autolearn, you may want to play with that threshold.. Based on your expertise with Bayes, should we change the default for 3.4.1? I'm +1 for it. I will be rolling a release candidate soon so we can hopefully release on 9/30 per our schedule.
Re: Whitelist one mail with multiple destinations
2014-09-10 10:17 GMT-03:00 Antony Stone : > On Wednesday 10 September 2014 at 14:56:06 (EU time), M. Rodrigo Monteiro > wrote: > >> Hi. Here is my scenario: >> >> Internet -> MX (Postfix) -> Relay (Postfix + Amavis with SpamAssassin) -> >> Zimbra > >> My problem is that when an e-mail comes to multiple destinations and >> one of them is whitelisted, all these destinations becomes whitelisted >> too. > > Looks like you want to set smtp_destination_recipient_limit = 1 in your front > end (MX) postfix setup: > > http://postfix.1071664.n5.nabble.com/Split-multiple-recipient-mail-td48458.html That not worked. I tested both on MX and Relay. Still the same problem. # postconf smtp_destination_recipient_limit smtp_destination_recipient_limit = $default_destination_recipient_limit # postconf default_destination_recipient_limit default_destination_recipient_limit = 1
Re: Whitelist one mail with multiple destinations
2014-09-10 10:23 GMT-03:00 David F. Skoll : > Option 2 is to accept the message unfiltered, split it into multiple copies, > and remail each copy so it can be scanned per-recipient. This avoids > the delay, but it also means you cannot reject spam with a 5xx SMTP failure > code or you'll be blacklisted for backscatter. How can I do it? All my Spams passes, none are blocked. It's no problem not reject them. > > Here at Roaring Penguin, we picked Option 2 as the lesser of the two > evils. > > Regards, > > David. Thanks, Rodrigo.
Re: RCVD_IN_MSPIKE_* strange scoring
On Wed, 10 Sep 2014, Matus UHLAR - fantomas wrote: On 10.09.14 13:22, Reindl Harald wrote: something is here terrible wrong why does "average" is preferred over "excellent" why do H3 and H4 get a very less WL score? I'd say, it's because of number of spams/hams received from hosts there. seems like only mail from hosts with average reputation appears on the net widely... s/on the net widely/in the masscheck corpora/ -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- USMC Rules of Gunfighting #9: Accuracy is relative: most combat shooting standards will be more dependent on "pucker factor" than the inherent accuracy of the gun. --- Tomorrow: the 13rd anniversary of 9/11
Re: Possible pattern here?
On Tue, 9 Sep 2014, Bob Proulx wrote: I am helping a friend who is getting hit with a lot of spam. He is running SpamAssassin. While looking at the spam that he is receiving I am seeing a pattern in the headers. Along with the normal headers the messages also contain a random set of "random" headers. Here are just the pattern headers from the message. Spam 1: Martian-Scurf: d4b0a3f064bc16518af081b52350787f9442861 Gonad-Marfa: 9442861.d4b0a3f064bc16518af081b52350787f.9442861 Diamant-Hop: d4b0a3f064bc16518af081b52350787f22464616.9442861d4b0a3f064bc16518af Mutiny-Tardo: 22464616-22464616 Odinist-Gawsy: d4b0a3f064bc16518af081b52350787f-22464616 Pennant-Agape: 9442861-22464616 That sort of random garbage was reported last week and there's a rule in the sandbox for it, but there's almost none in the masscheck corpus so it won't be scored or released. http://ruleqa.spamassassin.org/?daterev=20140909-r1623698-n&rule=%2FRAND_HEADER If it starts hitting the corpora it might get scored and released... Is there a way to use this to create a SpamAssassin rule to try to catch this type of spam? Grab the RAND_HEADER rules (there are several related, get them all) from my sandbox and score as you see fit. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- USMC Rules of Gunfighting #9: Accuracy is relative: most combat shooting standards will be more dependent on "pucker factor" than the inherent accuracy of the gun. --- Tomorrow: the 13rd anniversary of 9/11
bayes_auto_learn_threshold_nonspam
for quite a while I've been playing with autolearn settings SA's default is: bayes_auto_learn_threshold_nonspam0.1 this *can* cause low scored spam to be learnt as ham. For several months I've been using bayes_auto_learn_threshold_nonspam -1.0 and so far no more false negatives have been learnt as ham which is was hoping for. If you're using autolearn, you may want to play with that threshold.. Axb
Re: Whitelist one mail with multiple destinations
On Wed, 10 Sep 2014 09:56:06 -0300 "M. Rodrigo Monteiro" wrote: > My problem is that when an e-mail comes to multiple destinations and > one of them is whitelisted, all these destinations becomes whitelisted > too. There are really only two ways to get around this, and neither one is particularly pleasant. Option 1 is to tempfail all RCPT: commands after the first successful one. This lets you process per-user rules, but has the very bad side-effect of significantly delaying messages to a large number of recipients. Depending on the other end, the sender may get a delivery-delayed warning or the message might not even reach all recipients. Also, some marginal SMTP implementations are not tested very well and do not react correctly if some RCPT commans succeed and others are tempfailed. Option 2 is to accept the message unfiltered, split it into multiple copies, and remail each copy so it can be scanned per-recipient. This avoids the delay, but it also means you cannot reject spam with a 5xx SMTP failure code or you'll be blacklisted for backscatter. Here at Roaring Penguin, we picked Option 2 as the lesser of the two evils. Regards, David.
Re: Whitelist one mail with multiple destinations
On Wednesday 10 September 2014 at 15:17:29 (EU time), Kevin A. McGrail wrote: > On 9/10/2014 8:56 AM, M. Rodrigo Monteiro wrote: > > Hi. Here is my scenario: > > > > Internet -> MX (Postfix) -> Relay (Postfix + Amavis with SpamAssassin) -> > > Zimbra > > > > In SpamAssassin, I have a whitelist/blacklist. All the e-mail passes > > through, but Spams are taged (header and subject). > > > > My problem is that when an e-mail comes to multiple destinations and > > one of them is whitelisted, all these destinations becomes whitelisted > > too. > > > > In the real example below, the e-mail cs...@mydomain.com is > > whitelisted (-200 score). An unique e-mail (spam) comes to 20, 30 > > destinations and one of them is cs...@mydomain.com. All the > > destinations were whitelisted (-200 score). > > > > Here is the header of one e-mail and the log of Postfix. > > This behavior is SpamAssassin or Amavisd-new? > > The behavior is Amavis. You need to look at settings (if Amavis can do > it) or a glue like MIMEDefang that can do stream by domain or stream by > recipient type solutions to separate the one email into multiple emails > for individualized test and scoring. > > My understanding is that this will negate your ability to decline spam > during the SMTP connection, though. Surely that's been negated already, because the MX isn't running SA, therefore by the time SA sees the mail and can decide spam/ham, it's already been accepted? Antony. -- "It wouldn't be a good idea to talk about him behind his back in front of him." - murble Please reply to the list; please *don't* CC me.
Re: Whitelist one mail with multiple destinations
On Wednesday 10 September 2014 at 14:56:06 (EU time), M. Rodrigo Monteiro wrote: > Hi. Here is my scenario: > > Internet -> MX (Postfix) -> Relay (Postfix + Amavis with SpamAssassin) -> > Zimbra > My problem is that when an e-mail comes to multiple destinations and > one of them is whitelisted, all these destinations becomes whitelisted > too. Looks like you want to set smtp_destination_recipient_limit = 1 in your front end (MX) postfix setup: http://postfix.1071664.n5.nabble.com/Split-multiple-recipient-mail-td48458.html Antony. -- APL [is a language], in which you can write a program to simulate shuffling a deck of cards and then dealing them out to several players, in four characters, none of which appear on a standard keyboard. - David Given Please reply to the list; please *don't* CC me.
Re: Whitelist one mail with multiple destinations
On 9/10/2014 8:56 AM, M. Rodrigo Monteiro wrote: Hi. Here is my scenario: Internet -> MX (Postfix) -> Relay (Postfix + Amavis with SpamAssassin) -> Zimbra In SpamAssassin, I have a whitelist/blacklist. All the e-mail passes through, but Spams are taged (header and subject). My problem is that when an e-mail comes to multiple destinations and one of them is whitelisted, all these destinations becomes whitelisted too. In the real example below, the e-mail cs...@mydomain.com is whitelisted (-200 score). An unique e-mail (spam) comes to 20, 30 destinations and one of them is cs...@mydomain.com. All the destinations were whitelisted (-200 score). Here is the header of one e-mail and the log of Postfix. This behavior is SpamAssassin or Amavisd-new? The behavior is Amavis. You need to look at settings (if Amavis can do it) or a glue like MIMEDefang that can do stream by domain or stream by recipient type solutions to separate the one email into multiple emails for individualized test and scoring. My understanding is that this will negate your ability to decline spam during the SMTP connection, though. Regards, KAM
Whitelist one mail with multiple destinations
Hi. Here is my scenario:
Internet -> MX (Postfix) -> Relay (Postfix + Amavis with SpamAssassin) -> Zimbra
In SpamAssassin, I have a whitelist/blacklist. All the e-mail passes
through, but Spams are taged (header and subject).
My problem is that when an e-mail comes to multiple destinations and
one of them is whitelisted, all these destinations becomes whitelisted
too.
In the real example below, the e-mail cs...@mydomain.com is
whitelisted (-200 score). An unique e-mail (spam) comes to 20, 30
destinations and one of them is cs...@mydomain.com. All the
destinations were whitelisted (-200 score).
Here is the header of one e-mail and the log of Postfix.
This behavior is SpamAssassin or Amavisd-new?
Return-Path: laura...@semarh.goias.gov.br
Received: from eticesrv007.mydomain.com (LHLO
eticesrv007.mydomain.com) (172.26.70.7) by eticesrv007.mydomain.com
with LMTP; Tue, 9 Sep 2014 23:31:39 -0300 (BRT)
Received: from filtrodeconteudo1.mydomain.com (unknown [172.26.2.44])
by eticesrv007.mydomain.com (Postfix) with ESMTPS id 8F987884A55;
Tue, 9 Sep 2014 23:31:39 -0300 (BRT)
Received: from localhost (localhost [127.0.0.1])
by filtrodeconteudo1.mydomain.com (Postfix) with ESMTP id B3DEB2A016F;
Tue, 9 Sep 2014 23:31:39 -0300 (BRT)
X-Virus-Scanned: amavisd-new at mydomain.com
X-Spam-Flag: NO
X-Spam-Score: -200.771
X-Spam-Level:
X-Spam-Status: No, score=-200.771 required=5 tests=[AWL=-5.000, BAYES_00=-4,
DCC_CHECK=10, RCVD_IN_MSPIKE_H2=-1.77, SPF_PASS=-0.001,
USER_IN_WHITELIST_TO=-200] autolearn=no autolearn_force=no
Received: from filtrodeconteudo1.mydomain.com ([127.0.0.1])
by localhost (intsrv044.mydomain.com [127.0.0.1]) (amavisd-new, port 10024)
with LMTP id lTZPuM5PkD9Y; Tue, 9 Sep 2014 23:31:37 -0300 (BRT)
Received: from mx1.mydomain.com (mx1.mydomain.com [MX_IP])
by filtrodeconteudo1.mydomain.com (Postfix) with ESMTPS id A55772A016D;
Tue, 9 Sep 2014 23:31:37 -0300 (BRT)
X-Greylist: delayed 636 seconds by postgrey-1.35 at
intsrv036.mydomain.com; Tue, 09 Sep 2014 23:31:24 BRT
DKIM-Filter: OpenDKIM Filter v2.9.2 mx1.mydomain.com DEEE41A0057
DMARC-Filter: OpenDMARC Filter v1.2.0 mx1.mydomain.com DEEE41A0057
Authentication-Results: intsrv036.mydomain.com; dmarc=none
header.from=semarh.goias.gov.br
Received-SPF: pass (semarh.goias.gov.br: 189.2.188.131 is authorized
to use 'laura...@semarh.goias.gov.br' in 'mfrom' identity (mechanism
'mx' matched)) receiver=intsrv036; identity=mailfrom;
envelope-from="laura...@semarh.goias.gov.br";
helo=as.segplan.go.gov.br; client-ip=189.2.188.131
Received: from as.segplan.go.gov.br (as.segplan.go.gov.br [189.2.188.131])
by mx1.mydomain.com (Postfix) with SMTP id DEEE41A0057;
Tue, 9 Sep 2014 23:31:24 -0300 (BRT)
Received: from artemis.ecomunic.goias.gov.br (unknown [10.6.1.16])
by as.segplan.go.gov.br (Postfix) with SMTP id B2D617B902;
Tue, 9 Sep 2014 23:20:34 -0300 (BRT)
X-Virus-Scanned: amavisd-new at artemis.ecomunic.goias.gov.br
Date: Tue, 9 Sep 2014 23:20:31 -0300 (BRT)
From: Web Admin
Message-ID: <97597813.546385.1410315631612.javamail.r...@semarh.goias.gov.br>
Subject: att
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
X-Originating-IP: [10.6.128.44]
X-Mailer: Zimbra 7.2.7_GA_2942 (zclient/7.2.7_GA_2942)
To: undisclosed-recipients:;
Sep 9 23:31:39 intsrv044 postfix/smtpd[22327]: B3DEB2A016F:
client=localhost[127.0.0.1]
Sep 9 23:31:39 intsrv044 postfix/cleanup[22033]: B3DEB2A016F:
message-id=<97597813.546385.1410315631612.javamail.r...@semarh.goias.gov.br>
Sep 9 23:31:39 intsrv044 postfix/qmgr[11246]: B3DEB2A016F:
from=, size=2665, nrcpt=20 (queue
active)
Sep 9 23:31:39 intsrv044 amavis[18826]: (18826-11) Passed CLEAN
{RelayedInbound}, [IP_MX1]:35863 [189.2.188.131]
->
Queue-ID: A55772A016D, Message-ID:
<97597813.546385.1410315631612.javamail.r...@semarh.goias.gov.br>,
mail_id: lTZPuM5PkD9Y, Hits: -200.771, size: 1984, queued_as:
B3DEB2A016F, 2073 ms
Sep 9 23:31:39 intsrv044 postfix/lmtp[20175]: A55772A016D:
to=, relay=127.0.0.1[127.0.0.1]:10024,
delay=2.1, delays=0.04/0/0/2.1, dsn=2.0.0, status=sent (250 2.0.0 from
MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as B3DEB2A016F)
Sep 9 23:31:39 intsrv044 postfix/lmtp[20175]: A55772A016D:
to=, relay=127.0.0.1[127.0.0.1]:10024, delay=2.1,
delays=0.04/0/0/2.1, dsn=2.0.0, status=sent (250 2.0.0 from
MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as B3DEB2A016F)
Sep 9 23:31:39 intsrv044 postfix/lmtp[20175]: A55772A016D:
to=, relay=127.0.0.1[127.0.0.1]:10024, delay=2.1,
delays=0.04/0/0/2.1, dsn=2.0.0, status=sent (250 2.0.0 from
MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as B3DEB2A016F)
Sep 9 23:31:39 intsrv044 postfix/lmtp[20175]: A55772A016D:
to=, relay=127.0.0.1[127.0.0.1]:10024,
delay=2.1, delays=0.04/0/0/2.1, dsn=2.0.0, status=sent (250 2.0.0 from
MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as B3DEB2A016F)
Sep 9 23:31:39 intsrv044 postfix/lmtp[20175]: A55772A016D:
to=, relay=127.0.0.1[127.0.0.1]:10024,
de
Re: RCVD_IN_MSPIKE_* strange scoring
Am 10.09.2014 um 13:33 schrieb Matus UHLAR - fantomas: > On 10.09.14 13:22, Reindl Harald wrote: >> something is here terrible wrong >> >> why does "average" is preferred over "excellent" >> why do H3 and H4 get a very less WL score? > > I'd say, it's because of number of spams/hams received from hosts there. > seems like only mail from hosts with average reputation appears on the net > widely... not really [root@localhost:~]$ cat maillog | grep RCVD_IN_MSPIKE_H2 | wc -l 2996 [root@localhost:~]$ cat maillog | grep RCVD_IN_MSPIKE_H3 | wc -l 7494 [root@localhost:~]$ cat maillog | grep RCVD_IN_MSPIKE_H4 | wc -l 2255 [root@localhost:~]$ cat maillog | grep RCVD_IN_MSPIKE_H5 | wc -l 190 >> describe RCVD_IN_MSPIKE_H2 Average reputation (+2) >> describe RCVD_IN_MSPIKE_H3 Good reputation (+3) >> describe RCVD_IN_MSPIKE_H4 Very Good reputation (+4) >> describe RCVD_IN_MSPIKE_H5 Excellent reputation (+5) >> >> score RCVD_IN_MSPIKE_H2 0.001 -1.772 0.001 -1.772 >> score RCVD_IN_MSPIKE_H3 0.001 -0.010 0.001 -0.010 >> score RCVD_IN_MSPIKE_H4 0.001 -0.010 0.001 -0.010 >> score RCVD_IN_MSPIKE_H5 0.001 -1.000 0.001 -1.000 signature.asc Description: OpenPGP digital signature
Re: RCVD_IN_MSPIKE_* strange scoring
On 10.09.14 13:22, Reindl Harald wrote: something is here terrible wrong why does "average" is preferred over "excellent" why do H3 and H4 get a very less WL score? I'd say, it's because of number of spams/hams received from hosts there. seems like only mail from hosts with average reputation appears on the net widely... describe RCVD_IN_MSPIKE_H2 Average reputation (+2) describe RCVD_IN_MSPIKE_H3 Good reputation (+3) describe RCVD_IN_MSPIKE_H4 Very Good reputation (+4) describe RCVD_IN_MSPIKE_H5 Excellent reputation (+5) score RCVD_IN_MSPIKE_H2 0.001 -1.772 0.001 -1.772 score RCVD_IN_MSPIKE_H3 0.001 -0.010 0.001 -0.010 score RCVD_IN_MSPIKE_H4 0.001 -0.010 0.001 -0.010 score RCVD_IN_MSPIKE_H5 0.001 -1.000 0.001 -1.000 -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Support bacteria - they're the only culture some people have.
RCVD_IN_MSPIKE_* strange scoring
something is here terrible wrong why does "average" is preferred over "excellent" why do H3 and H4 get a very less WL score? recently a clear spam message slipped by the -1.7 through describe RCVD_IN_MSPIKE_H2 Average reputation (+2) describe RCVD_IN_MSPIKE_H3 Good reputation (+3) describe RCVD_IN_MSPIKE_H4 Very Good reputation (+4) describe RCVD_IN_MSPIKE_H5 Excellent reputation (+5) score RCVD_IN_MSPIKE_H2 0.001 -1.772 0.001 -1.772 score RCVD_IN_MSPIKE_H3 0.001 -0.010 0.001 -0.010 score RCVD_IN_MSPIKE_H4 0.001 -0.010 0.001 -0.010 score RCVD_IN_MSPIKE_H5 0.001 -1.000 0.001 -1.000 __ i changed that in "local.cf" to the following score RCVD_IN_MSPIKE_H2 0.001 -0.5 0.001 -0.5 score RCVD_IN_MSPIKE_H3 0.001 -0.8 0.001 -0.8 score RCVD_IN_MSPIKE_H4 0.001 -1.1 0.001 -1.1 score RCVD_IN_MSPIKE_H5 0.001 -1.5 0.001 -1.5 signature.asc Description: OpenPGP digital signature
Re: Possible pattern here?
On 09/10/2014 08:48 AM, Joolee wrote: Sounds like a case of http://www.gossamer-threads.com/lists/spamassassin/users/187586 You might be able to find the rule mentioned here: http://svn.apache.org/viewvc/spamassassin/trunk/rulesrc/sandbox/jhardin/ RAND_HEADER_MANY On 10 September 2014 07:38, Bob Proulx wrote: I am helping a friend who is getting hit with a lot of spam. He is running SpamAssassin. While looking at the spam that he is receiving I am seeing a pattern in the headers. Along with the normal headers the messages also contain a random set of "random" headers. Here are just the pattern headers from the message. Spam 1: Martian-Scurf: d4b0a3f064bc16518af081b52350787f9442861 Gonad-Marfa: 9442861.d4b0a3f064bc16518af081b52350787f.9442861 Diamant-Hop: d4b0a3f064bc16518af081b52350787f22464616.9442861d4b0a3f064bc16518af Mutiny-Tardo: 22464616-22464616 Odinist-Gawsy: d4b0a3f064bc16518af081b52350787f-22464616 Pennant-Agape: 9442861-22464616 Spam 2: Mispage-Slav: 16035617 Irra-Etna: 9493147 Brigand-Parry: 1603561716035617 Peatier-Fthm: d4b0a3f064bc16518af081b52350787f Spam 3: Penang-Titan: d4b0a3f064bc16518af081b52350787f12517557 Imbrue-Gaol: 12517557.12517557 Tousle-Zany: d4b0a3f064bc16518af081b52350787f Callie-Scale: 19474509.19474509 Spam 4: Felda-Elayl: 1-15546426 Bluma-Spoom: 15546426-14093545455-9801 Prs-Cathy: 14093545-ag84js-dk3k32 Quest-Argue: 0.a4-052.15546426 You get the idea. I have 187 spams from a recent burst like this. Here is a more complete header example. I am not showing my buddy's address intentionally so redacted the To: line but all of the other headers are there. http://pastebin.com/0jmiDBt1 And here is a full sample. Notice how the header data is repeated in the message body. http://pastebin.com/0Ga7g0UX Looking at the headers by eye and flipping from message to message it is pretty easy to visually see the pattern that is created. Is there a way to use this to create a SpamAssassin rule to try to catch this type of spam? Thanks, Bob P.S. Note that if I run these through my Bayes my database almost always scores them quite high. But on his, not so much. Improving his Bayes training will help. But the pattern seems ripe too.
