Re: spam score question
On Fri, 24 Apr 2015 02:50:11 +0200 Mark Martinec mark.martinec...@ijs.si wrote: On April 22, 2015 8:44:59 PM EDT, Thom Miller t...@cagroups.com wrote: On Sat, 18 Apr 2015 08:16:40 -0700 Michael Williamson michael.h.william...@gmail.com wrote: It appears to me that spamassassin can produce different spam scores for the same email. In particular, I have noticed that points are omitted for RCVD_IN_SBL_CSS (Spamhaus blacklist) sometimes. Why? In the past I noticed that network tests were sometimes completely omitted. I believe sa checks for network connectivity before perfoming these tests, and incorrectly determines that there is no network available. In my case, adding: dns_available yes to my local.cf solved this issue. 2015-04-24 01:38, Thom Miller wrote: Kevin A. McGrail kmcgr...@pccc.com wrote: On 4/22/2015 11:19 PM, Thom Miller wrote: According to https://spamassassin.apache.org/full/3.2.x/doc/Mail_SpamAssassin_Conf.html : By default, SpamAssassin will query some default hosts on the internet to attempt to check if DNS is working or not. The problem is that it can introduce some delay if your network connection is down, and in some cases it can wrongly guess that DNS is unavailable because the test connections failed. I decided that since the network should always be available, there's no reason for spamassassin to test it. Interesting. What version of SA are you using? I'm running 3.4.0 now, but I made this addition to local.cf when I was running 3.2. I don't know if the change is still necessary in my case, but I haven't bothered to remove it. -Thom The 'dns_available yes' is a default since 3.4.0. 3.4.0 release notes: * A default setting for option 'dns_available' was changed from 'test' to 'yes' (bug 6770, bug 6769), so SpamAssassin now assumes by default that it is running on a host with an internet connection and a working DNS resolver. If this is not the case, please configure this option explicitly. The change avoids surprises on an otherwise well connected host which may experience a temporary DNS unavailability at the system startup time or a temporary network outage when spamd was starting, and the initial failed test would disable DNS queries permanently. The option is documented in the Mail::SpamAssassin::Conf POD or man page. Mark Thank you for the info. I'll go ahead and comment it out in local.cf. -Thom
Re: spam score question
On 4/22/2015 11:19 PM, Thom Miller wrote: On Wed, 22 Apr 2015 21:23:22 -0400 Kevin A. McGrail kmcgr...@pccc.com wrote: Are you starting spamd before your networking and local dns are started? Regards, KAM No. spamd is started after the network is up and running. According to https://spamassassin.apache.org/full/3.2.x/doc/Mail_SpamAssassin_Conf.html : By default, SpamAssassin will query some default hosts on the internet to attempt to check if DNS is working or not. The problem is that it can introduce some delay if your network connection is down, and in some cases it can wrongly guess that DNS is unavailable because the test connections failed. I decided that since the network should always be available, there's no reason for spamassassin to test it. Interesting. What version of SA are you using? -Thom On April 22, 2015 8:44:59 PM EDT, Thom Miller t...@cagroups.com wrote: On Sat, 18 Apr 2015 08:16:40 -0700 Michael Williamson michael.h.william...@gmail.com wrote: Hi, I have another question. It appears to me that spamassassin can produce different spam scores for the same email. In particular, I have noticed that points are omitted for RCVD_IN_SBL_CSS (Spamhaus blacklist) sometimes. Why? In the past I noticed that network tests were sometimes completely omitted. I believe sa checks for network connectivity before perfoming these tests, and incorrectly determines that there is no network available. In my case, adding: dns_available yes to my local.cf solved this issue. -Thom -- *Kevin A. McGrail* President Peregrine Computer Consultants Corporation 3927 Old Lee Highway, Suite 102-C Fairfax, VA 22030-2422 http://www.pccc.com/ 703-359-9700 x50 / 800-823-8402 (Toll-Free) 703-798-0171 (wireless) kmcgr...@pccc.com mailto:kmcgr...@pccc.com
Re: spam score question
On Thu, 23 Apr 2015 11:17:12 -0400 Kevin A. McGrail kmcgr...@pccc.com wrote: On 4/22/2015 11:19 PM, Thom Miller wrote: On Wed, 22 Apr 2015 21:23:22 -0400 Kevin A. McGrail kmcgr...@pccc.com wrote: Are you starting spamd before your networking and local dns are started? Regards, KAM No. spamd is started after the network is up and running. According to https://spamassassin.apache.org/full/3.2.x/doc/Mail_SpamAssassin_Conf.html : By default, SpamAssassin will query some default hosts on the internet to attempt to check if DNS is working or not. The problem is that it can introduce some delay if your network connection is down, and in some cases it can wrongly guess that DNS is unavailable because the test connections failed. I decided that since the network should always be available, there's no reason for spamassassin to test it. Interesting. What version of SA are you using? I'm running 3.4.0 now, but I made this addition to local.cf when I was running 3.2. I don't know if the change is still necessary in my case, but I haven't bothered to remove it. -Thom -Thom On April 22, 2015 8:44:59 PM EDT, Thom Miller t...@cagroups.com wrote: On Sat, 18 Apr 2015 08:16:40 -0700 Michael Williamson michael.h.william...@gmail.com wrote: Hi, I have another question. It appears to me that spamassassin can produce different spam scores for the same email. In particular, I have noticed that points are omitted for RCVD_IN_SBL_CSS (Spamhaus blacklist) sometimes. Why? In the past I noticed that network tests were sometimes completely omitted. I believe sa checks for network connectivity before perfoming these tests, and incorrectly determines that there is no network available. In my case, adding: dns_available yes to my local.cf solved this issue. -Thom
Re: spam score question
On April 22, 2015 8:44:59 PM EDT, Thom Miller t...@cagroups.com wrote: On Sat, 18 Apr 2015 08:16:40 -0700 Michael Williamson michael.h.william...@gmail.com wrote: It appears to me that spamassassin can produce different spam scores for the same email. In particular, I have noticed that points are omitted for RCVD_IN_SBL_CSS (Spamhaus blacklist) sometimes. Why? In the past I noticed that network tests were sometimes completely omitted. I believe sa checks for network connectivity before perfoming these tests, and incorrectly determines that there is no network available. In my case, adding: dns_available yes to my local.cf solved this issue. 2015-04-24 01:38, Thom Miller wrote: Kevin A. McGrail kmcgr...@pccc.com wrote: On 4/22/2015 11:19 PM, Thom Miller wrote: According to https://spamassassin.apache.org/full/3.2.x/doc/Mail_SpamAssassin_Conf.html : By default, SpamAssassin will query some default hosts on the internet to attempt to check if DNS is working or not. The problem is that it can introduce some delay if your network connection is down, and in some cases it can wrongly guess that DNS is unavailable because the test connections failed. I decided that since the network should always be available, there's no reason for spamassassin to test it. Interesting. What version of SA are you using? I'm running 3.4.0 now, but I made this addition to local.cf when I was running 3.2. I don't know if the change is still necessary in my case, but I haven't bothered to remove it. -Thom The 'dns_available yes' is a default since 3.4.0. 3.4.0 release notes: * A default setting for option 'dns_available' was changed from 'test' to 'yes' (bug 6770, bug 6769), so SpamAssassin now assumes by default that it is running on a host with an internet connection and a working DNS resolver. If this is not the case, please configure this option explicitly. The change avoids surprises on an otherwise well connected host which may experience a temporary DNS unavailability at the system startup time or a temporary network outage when spamd was starting, and the initial failed test would disable DNS queries permanently. The option is documented in the Mail::SpamAssassin::Conf POD or man page. Mark
Re: spam score question
Are you starting spamd before your networking and local dns are started? Regards, KAM On April 22, 2015 8:44:59 PM EDT, Thom Miller t...@cagroups.com wrote: On Sat, 18 Apr 2015 08:16:40 -0700 Michael Williamson michael.h.william...@gmail.com wrote: Hi, I have another question. It appears to me that spamassassin can produce different spam scores for the same email. In particular, I have noticed that points are omitted for RCVD_IN_SBL_CSS (Spamhaus blacklist) sometimes. Why? In the past I noticed that network tests were sometimes completely omitted. I believe sa checks for network connectivity before perfoming these tests, and incorrectly determines that there is no network available. In my case, adding: dns_available yes to my local.cf solved this issue. -Thom
Re: spam score question
On Sat, 18 Apr 2015 08:16:40 -0700 Michael Williamson michael.h.william...@gmail.com wrote: Hi, I have another question. It appears to me that spamassassin can produce different spam scores for the same email. In particular, I have noticed that points are omitted for RCVD_IN_SBL_CSS (Spamhaus blacklist) sometimes. Why? In the past I noticed that network tests were sometimes completely omitted. I believe sa checks for network connectivity before perfoming these tests, and incorrectly determines that there is no network available. In my case, adding: dns_available yes to my local.cf solved this issue. -Thom
Re: spam score question
On Wed, 22 Apr 2015 21:23:22 -0400 Kevin A. McGrail kmcgr...@pccc.com wrote: Are you starting spamd before your networking and local dns are started? Regards, KAM No. spamd is started after the network is up and running. According to https://spamassassin.apache.org/full/3.2.x/doc/Mail_SpamAssassin_Conf.html : By default, SpamAssassin will query some default hosts on the internet to attempt to check if DNS is working or not. The problem is that it can introduce some delay if your network connection is down, and in some cases it can wrongly guess that DNS is unavailable because the test connections failed. I decided that since the network should always be available, there's no reason for spamassassin to test it. -Thom On April 22, 2015 8:44:59 PM EDT, Thom Miller t...@cagroups.com wrote: On Sat, 18 Apr 2015 08:16:40 -0700 Michael Williamson michael.h.william...@gmail.com wrote: Hi, I have another question. It appears to me that spamassassin can produce different spam scores for the same email. In particular, I have noticed that points are omitted for RCVD_IN_SBL_CSS (Spamhaus blacklist) sometimes. Why? In the past I noticed that network tests were sometimes completely omitted. I believe sa checks for network connectivity before perfoming these tests, and incorrectly determines that there is no network available. In my case, adding: dns_available yes to my local.cf solved this issue. -Thom
spam score question
Hi, I have another question. It appears to me that spamassassin can produce different spam scores for the same email. In particular, I have noticed that points are omitted for RCVD_IN_SBL_CSS (Spamhaus blacklist) sometimes. Why? Is the difference due to a difference in how spamassassin is invoked? (for example, due an environment variable). One way that I invoke spamassassin to get spam scores is from a program that is started as a cronjob for a user. This way sometimes omits the points for the test mentioned above. Then, when I invoke spamassassin from the command line as the same user, for the same email, I get a higher score because it includes points for RCVD_IN_SBL_CSS. I am using a fairly old version, SpamAssassin version 3.3.1 running on Perl version 5.10.1. The OS is CentOS 6.0. Thanks, -Mike
Re: spam score question
On Saturday 18 April 2015 at 17:16:40 (EU time), Michael Williamson wrote: Hi, I have another question. It appears to me that spamassassin can produce different spam scores for the same email. Do you mean *exactly* the same email - totally identical headers and body, with no changes between the two invocations? In particular, I have noticed that points are omitted for RCVD_IN_SBL_CSS (Spamhaus blacklist) sometimes. Why? Well, there's a chance that the machine you received the email from wasn't in the Spamhaus blacklist on one occasion, and was on the other... Is the difference due to a difference in how spamassassin is invoked? (for example, due an environment variable). One way that I invoke spamassassin to get spam scores is from a program that is started as a cronjob for a user. Does that job run as the user, or as another ID on the system? What exactly are you passing to SpamAssassin from the cron job (where are you getting the email from in the standard delivery path), and how else do you pass emails to SpamAssassin in the normal course of email delivery (you don't mention what your MTA is, or how SpamAssassin is plugged in to it)? This way sometimes omits the points for the test mentioned above. Then, when I invoke spamassassin from the command line as the same user, for the same email, I get a higher score because it includes points for RCVD_IN_SBL_CSS. Since you say you are running both checks as the same user, and also you're focusing on the score for one specific test, I'll omit any possibility that you've got different Bayes databases on the machine, each being used by the different ways you're passing the email to SpamAssassin. When you've plucked an email out of the delivery path and sent it (via the cron job) to SpamAssassin, do you then re-insert it back into the same place in the delivery path, and is that place immediately before it would get passed to SpamAssassin by some milter or similar feature? If not, please describe your email delivery path, paying particular attention to where you're taking the emails out (for cron job processing), where you're reinserting them, and where SpamAssassin otherwise gets invoked. I am using a fairly old version, SpamAssassin version 3.3.1 running on Perl version 5.10.1. The OS is CentOS 6.0. Out of interest, why are you passing emails to SpamAssassin from a cron job, and then apparently later getting them scored in the normal course of email delivery? What's the purpose of the cron job? Antony. -- Atheism is a non-prophet-making organisation. Please reply to the list; please *don't* CC me.
Re: spam score question
On 4/18/15, Antony Stone antony.st...@spamassassin.open.source.it wrote: On Saturday 18 April 2015 at 17:16:40 (EU time), Michael Williamson wrote: Hi, I have another question. It appears to me that spamassassin can produce different spam scores for the same email. Do you mean *exactly* the same email - totally identical headers and body, with no changes between the two invocations? Yes, I believe so, exactly identical. In particular, I have noticed that points are omitted for RCVD_IN_SBL_CSS (Spamhaus blacklist) sometimes. Why? Well, there's a chance that the machine you received the email from wasn't in the Spamhaus blacklist on one occasion, and was on the other... Something like this is possible, although I think it would be more likely that the failure is due to a timeout or communication problem with the Spamhaus server. Is the difference due to a difference in how spamassassin is invoked? (for example, due an environment variable). One way that I invoke spamassassin to get spam scores is from a program that is started as a cronjob for a user. Does that job run as the user, or as another ID on the system? It is run from the users cron table. For score comparison, from the command line, I do # su username # spamassassin -t email_filename I know that there might actually be some different environment variables doing it this way (like PATH). What exactly are you passing to SpamAssassin from the cron job (where are you getting the email from in the standard delivery path), and how else do you pass emails to SpamAssassin in the normal course of email delivery (you don't mention what your MTA is, or how SpamAssassin is plugged in to it)? The email server runs postfix, amavis, and dovecot (and roundcube). I elaborate on this below. This way sometimes omits the points for the test mentioned above. Then, when I invoke spamassassin from the command line as the same user, for the same email, I get a higher score because it includes points for RCVD_IN_SBL_CSS. Since you say you are running both checks as the same user, and also you're focusing on the score for one specific test, I'll omit any possibility that you've got different Bayes databases on the machine, each being used by the different ways you're passing the email to SpamAssassin. When you've plucked an email out of the delivery path and sent it (via the cron job) to SpamAssassin, do you then re-insert it back into the same place in the delivery path, and is that place immediately before it would get passed to SpamAssassin by some milter or similar feature? If not, please describe your email delivery path, paying particular attention to where you're taking the emails out (for cron job processing), where you're reinserting them, and where SpamAssassin otherwise gets invoked. I am using a fairly old version, SpamAssassin version 3.3.1 running on Perl version 5.10.1. The OS is CentOS 6.0. Out of interest, why are you passing emails to SpamAssassin from a cron job, and then apparently later getting them scored in the normal course of email delivery? What's the purpose of the cron job? The reason that I am using a cronjob for users, is that I could never get the dovecot 'sieve' plugin to work. So instead, I wrote a program using inotify to watch for new email files to appear in the directory 'Maildir/new/', and move them immediately to 'Maildir/tmp/' before dovecot gets them. Then the program either moves the file back to 'Maildir/new/' or into a spam folder. In order to decide where to move it, the program runs spamassassin again (since the mail has already been scored at this point) using fork/exec. The reason that spamassassin is run again, is that some users use the spamassassin bayes database training program sa-learn for their individual accounts, but that bayes database(s) is not used, as far as I can tell, when amavis first invokes spamassassin, before mail is put into 'Maildir/new', so the scores are too low. When I re-run spamassassin (both of the two different ways mentioned), it is using the -t option and the email content is piped in from the standard input. This does not modify the original email content including the original inserted spam scores, but it does generate a new score, using the user database. This method has been working pretty well for a about a week, until this Spamhaus issue. An alternative that I have considered is to simply set up a new email server, but without amavis. Thanks, -Mike
Re: spam score question
From: Dhaval Patel [EMAIL PROTECTED]
Daryl C. W. O'Shea [EMAIL PROTECTED] said:
Dhaval Patel wrote:
I hope that I am asking for this kind of help in the right place.
I always look into why any spam got into my Inbox and found the reasons for this
message troubling. I use spamc in the maildrop rule but I put that message
through spamassassin -t -D and get the same score so I am assuming that it is
running the exact same way. I have pasted the output below.
I checked the sender IP and it seems that they are blacklisted on sbl-xbl as well as
a few other lists. But spamassassin did not pick this up. Can anybody give any
insight into this? I do see other spams being caught because of RBLs.
What is -3.3 ALL_TRUSTEDDid not pass through any untrusted hosts How
is it determined that the host (71.214.161.98) was trusted.
You need to configure your trusted_networks. See this wiki article:
http://wiki.apache.org/spamassassin/TrustPath
OK, adding trusted_networks myIP to local.cf works. Now I get
3.5 HELO_DYNAMIC_IPADDR2 Relay HELO'd using suspicious hostname (IP addr 2)
which makes the spam score 8.9 which is considered as spam.
I have read that article before but assumed that it was only if your mail
server was
behind a NAT, which mine is not. Do other people on this list find that they
must set
this option in order for SA to avoid problems like I experienced?
If email comes to your machine via an MX record then you probably need
to trust your machine by its address on the Internet not (just) by
127.0.0.1.
{^_^}
Re: spam score question
From: Jim Knuth [EMAIL PROTECTED]
Heute (17.09.2006/06:46 Uhr) schrieb Daryl C. W. O'Shea,
Jim Knuth wrote:
How can I find out, which is my trusted network?
By reading the documentation and comparing it with your network config? :)
Is this my Server IP address or 127.0.0.0 or what? ;)
Any IP that appears in your received headers from your MX all the way to
the machine running SA (which may be the same machine as your MX).
Including 127.0.0.1 is a good idea.
Daryl
thanx. I only wonna hear this. ;) It`s only 127.0.0.1. Mail is
going through Postfix and Amavis. SA is in this case secondarily.
If your machine's Internet address appears in the headers you must
trust that address, too.
{^_^}
Re: spam score question
Dhaval Patel wrote: I hope that I am asking for this kind of help in the right place. I always look into why any spam got into my Inbox and found the reasons for this message troubling. I use spamc in the maildrop rule but I put that message through spamassassin -t -D and get the same score so I am assuming that it is running the exact same way. I have pasted the output below. I checked the sender IP and it seems that they are blacklisted on sbl-xbl as well as a few other lists. But spamassassin did not pick this up. Can anybody give any insight into this? I do see other spams being caught because of RBLs. What is -3.3 ALL_TRUSTEDDid not pass through any untrusted hosts How is it determined that the host (71.214.161.98) was trusted. You need to configure your trusted_networks. See this wiki article: http://wiki.apache.org/spamassassin/TrustPath Daryl
Re: spam score question
Heute (17.09.2006/06:20 Uhr) schrieb Daryl C. W. O'Shea, Dhaval Patel wrote: I hope that I am asking for this kind of help in the right place. I always look into why any spam got into my Inbox and found the reasons for this message troubling. I use spamc in the maildrop rule but I put that message through spamassassin -t -D and get the same score so I am assuming that it is running the exact same way. I have pasted the output below. I checked the sender IP and it seems that they are blacklisted on sbl-xbl as well as a few other lists. But spamassassin did not pick this up. Can anybody give any insight into this? I do see other spams being caught because of RBLs. What is -3.3 ALL_TRUSTEDDid not pass through any untrusted hosts How is it determined that the host (71.214.161.98) was trusted. You need to configure your trusted_networks. See this wiki article: http://wiki.apache.org/spamassassin/TrustPath How can I find out, which is my trusted network? Is this my Server IP address or 127.0.0.0 or what? ;) Daryl -- Viele Gruesse, Kind regards, Jim Knuth [EMAIL PROTECTED] ICQ #277289867 -- Zufalls-Zitat -- Die Phönizier haben das Geld erfunden, aber warum so wenig? (J. Nestroy, öster. Volksschauspieler, 1801-1862) -- Der Text hat nichts mit dem Empfaenger der Mail zu tun -- Virus free. Checked by NOD32 Version 1.1759 Build 8029 16.09.2006
Re: spam score question
Jim Knuth wrote: How can I find out, which is my trusted network? By reading the documentation and comparing it with your network config? :) Is this my Server IP address or 127.0.0.0 or what? ;) Any IP that appears in your received headers from your MX all the way to the machine running SA (which may be the same machine as your MX). Including 127.0.0.1 is a good idea. Daryl
Re: spam score question
Daryl C. W. O'Shea [EMAIL PROTECTED] said: Dhaval Patel wrote: I hope that I am asking for this kind of help in the right place. I always look into why any spam got into my Inbox and found the reasons for this message troubling. I use spamc in the maildrop rule but I put that message through spamassassin -t -D and get the same score so I am assuming that it is running the exact same way. I have pasted the output below. I checked the sender IP and it seems that they are blacklisted on sbl-xbl as well as a few other lists. But spamassassin did not pick this up. Can anybody give any insight into this? I do see other spams being caught because of RBLs. What is -3.3 ALL_TRUSTEDDid not pass through any untrusted hosts How is it determined that the host (71.214.161.98) was trusted. You need to configure your trusted_networks. See this wiki article: http://wiki.apache.org/spamassassin/TrustPath OK, adding trusted_networks myIP to local.cf works. Now I get 3.5 HELO_DYNAMIC_IPADDR2 Relay HELO'd using suspicious hostname (IP addr 2) which makes the spam score 8.9 which is considered as spam. I have read that article before but assumed that it was only if your mail server was behind a NAT, which mine is not. Do other people on this list find that they must set this option in order for SA to avoid problems like I experienced? Thanks, Dhaval
Re: spam score question
Heute (17.09.2006/06:46 Uhr) schrieb Daryl C. W. O'Shea, Jim Knuth wrote: How can I find out, which is my trusted network? By reading the documentation and comparing it with your network config? :) Is this my Server IP address or 127.0.0.0 or what? ;) Any IP that appears in your received headers from your MX all the way to the machine running SA (which may be the same machine as your MX). Including 127.0.0.1 is a good idea. Daryl thanx. I only wonna hear this. ;) It`s only 127.0.0.1. Mail is going through Postfix and Amavis. SA is in this case secondarily. -- Viele Gruesse, Kind regards, Jim Knuth [EMAIL PROTECTED] ICQ #277289867 -- Zufalls-Zitat -- Wer nicht gerne denkt, sollte wenigstens von Zeit zu Zeit seine Vorurteile neu gruppieren. (Luther Burbank,Biologe,1849-1926) -- Der Text hat nichts mit dem Empfaenger der Mail zu tun -- Virus free. Checked by NOD32 Version 1.1759 Build 8029 16.09.2006
