Re: SPF and DKIM tests by default?
On 2/16/2012 6:18 PM, email builder wrote: Letting trusted_networks empty is not generally a good idea. In particular, if your SA server is using a private IP, it will default to trusting too much. Specify your local networks in trusted_networks and see if that helps your problem. Leaving trusted_networks empty does not mean trust nothing; it means let SA figure out what to trust. Makes sense, especially if my hunch about the relayed through one or more trusted relays, cannot use header-based Envelope-From, skipping part of the debug output I just sent to this list is on track. Is there a way to set trusted_networks on the command line of the spamassassin command just for testing? This didn't work: spamassassin -D --cf='trusted_networks 127.0.0.1' -t example_email_no_spf 21 | grep -i SPF All my local handoffs are to localhost [127.0.0.1] so I wouldn't know what else to use (it's an all-in-one single server simple system) I'm not sure if that format will work or not. If your normal process uses Amavisd_new or spamd, you can just edit the config files for your tests. Changes to the config files will not affect the daemons until they are restarted. At some point, there has to be an external IP to accept mail from the Internet. That is what you need to add to trusted_networks. 127.0.0.1 should be automatically trusted. The simplest solution (unless you don't trust your network for some reason) is to add all of the IP blocks that you control to trusted_networks. -- Bowie
Re: SPF and DKIM tests by default?
Q: Will some rules not fire if some condition exists based on other rules? A: Correct. There are plenty of rules that build on other rules. We call these meta rules. On 15.02.12 16:08, email builder wrote: Q: Are there any default rules as supplied by sa-update that would prevent SPF rules from firing? you can disable SPF or clear all scores Q: Any other ideas on how to learn what rules are actually being used? huh? Q: Any suggestions as to why SPF rules would not fire on a Gmail message where Gmail uses SPF, my SPF plugin and rule initiation seem to be in place, and a Return-Path header with the envelope from address exists? (please see my previous messages on this thread) I haven't found the headers in apache archive, maybe I didn't search carefully enough, but it's misconfigured trusted_networks and internal_networks what causes SPF to misfire... -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Windows found: (R)emove, (E)rase, (D)elete
Re: SPF and DKIM tests by default?
On 2/15/2012 7:08 PM, email builder wrote: OK, but: Q: Are there any default rules as supplied by sa-update that would prevent SPF rules from firing? Not that I can think of. Q: Any other ideas on how to learn what rules are actually being used? What I would likely do is save the gmail message to an mbox format file. Then I would run spamassassin -D -t /tmp/mboxfile 21 | grep -i SPF and see what I find. Regards, KAM
Re: SPF and DKIM tests by default?
Q: Will some rules not fire if some condition exists based on other rules? A: Correct. There are plenty of rules that build on other rules. We call these meta rules. Q: Are there any default rules as supplied by sa-update that would prevent SPF rules from firing? you can disable SPF or clear all scores The question was *as supplied by sa-update* Q: Any other ideas on how to learn what rules are actually being used? huh? Please read the rest of this thread. Q: Any suggestions as to why SPF rules would not fire on a Gmail message where Gmail uses SPF, my SPF plugin and rule initiation seem to be in place, and a Return-Path header with the envelope from address exists? (please see my previous messages on this thread) I haven't found the headers in apache archive, maybe I didn't search carefully enough, I recommend gmane.org but it's misconfigured trusted_networks and internal_networks what causes SPF to misfire... Thank you sincerely for your help. I can only imagine that SPF wouldn't fire if I accidentally specified Google in one of those settings or had an error in one of them. In this case, those are at their defaults of empty, so I'm hoping there are other suggestions. Thanks again..
Re: SPF and DKIM tests by default?
On 2/16/2012 4:54 PM, email builder wrote: but it's misconfigured trusted_networks and internal_networks what causes SPF to misfire... Thank you sincerely for your help. I can only imagine that SPF wouldn't fire if I accidentally specified Google in one of those settings or had an error in one of them. In this case, those are at their defaults of empty, so I'm hoping there are other suggestions. Thanks again.. Letting trusted_networks empty is not generally a good idea. In particular, if your SA server is using a private IP, it will default to trusting too much. Specify your local networks in trusted_networks and see if that helps your problem. Leaving trusted_networks empty does not mean trust nothing; it means let SA figure out what to trust. -- Bowie
Re: SPF and DKIM tests by default?
On 2/15/2012 7:08 PM, email builder wrote: OK, but: Q: Are there any default rules as supplied by sa-update that would prevent SPF rules from firing? Not that I can think of. Q: Any other ideas on how to learn what rules are actually being used? What I would likely do is save the gmail message to an mbox format file. Then I would run spamassassin -D -t /tmp/mboxfile 21 | grep -i SPF and see what I find. Well, that was actually the other more general question that you kindly already offered your help for - how to determine all rules currently in use at execution time. Short of other opinions, we'll wait to see how the bugzilla item I created progresses. But your advice here is in fact quite useful and may do a fine job at pointing to the issue. Keep in mind, all rules are as given by sa-update. I copied in all the output below but here are what I see as key points by line number: Line 8: Someone earlier pointed out that SA uses this Received-SPF header, but then I think it was you that pointed out that this shouldn't be necessary, and I added that it would seem odd to me if SA didn't also look for the quasi-standard Return-Path header which for some mailers such as Postfix will include the envelope from address. The lack of this header doesn't seem to stop SPF execution though. When I copy my Return-Path header into a Received-SPF header, line 8 becomes two lines: Feb 16 14:19:59.263 [12846] dbg: spf: found a Received-SPF header added by an internal host: Received-SPF: emailbuilde...@gmail.com Feb 16 14:19:59.263 [12846] dbg: spf: could not parse result from existing Received-SPF header I tried this with a couple different formats of email and/or domain name. Not sure what's going on here. The rest of the lines are the same in both cases. Line 12: Any comments why the SPF lookup returns nothing? How can I do this same lookup by hand? Could this be a DNS problem? Line 13: Weren't the last few lines DNS checks already? Line 14: I don't know why this happens. It is true that Postfix relays mail to amavis, then it goes back to Postfix then it is handed off to maildrop for delivery - SA is called from maildrop. So there is some local relaying here, but why does this stop SA from checking the hop from the outside to my Postfix? Is this where having a non-default trusted_networks setting would help? Thanks again for the great help and patience. 1) Feb 16 14:13:17.361 [12806] dbg: plugin: loading Mail::SpamAssassin::Plugin::SPF from @INC 2) Feb 16 14:13:17.774 [12806] dbg: config: fixed relative path: /var/lib/spamassassin/3.003001/updates_spamassassin_org/25_spf.cf 3) Feb 16 14:13:17.774 [12806] dbg: config: using /var/lib/spamassassin/3.003001/updates_spamassassin_org/25_spf.cf for included file 4) Feb 16 14:13:17.774 [12806] dbg: config: read file /var/lib/spamassassin/3.003001/updates_spamassassin_org/25_spf.cf 5) Feb 16 14:13:17.894 [12806] dbg: config: fixed relative path: /var/lib/spamassassin/3.003001/updates_spamassassin_org/60_whitelist_spf.cf 6) Feb 16 14:13:17.895 [12806] dbg: config: using /var/lib/spamassassin/3.003001/updates_spamassassin_org/60_whitelist_spf.cf for included file 7) Feb 16 14:13:17.895 [12806] dbg: config: read file /var/lib/spamassassin/3.003001/updates_spamassassin_org/60_whitelist_spf.cf 8) Feb 16 14:13:19.595 [12806] dbg: spf: checking to see if the message has a Received-SPF header that we can use 9) Feb 16 14:13:19.646 [12806] dbg: spf: using Mail::SPF for SPF checks 10) Feb 16 14:13:19.646 [12806] dbg: spf: checking HELO (helo=mail-iy0-f181.google.com, ip=209.85.210.181) 11) Feb 16 14:13:19.648 [12806] dbg: dns: providing a callback for id: 13553/mail-iy0-f181.google.com/SPF/IN 12) Feb 16 14:13:19.984 [12806] dbg: spf: query for /209.85.210.181/mail-iy0-f181.google.com: result: none, comment: , text: No applicable sender policy available 13) Feb 16 14:13:19.988 [12806] dbg: spf: already checked for Received-SPF headers, proceeding with DNS based checks 14) Feb 16 14:13:19.988 [12806] dbg: spf: relayed through one or more trusted relays, cannot use header-based Envelope-From, skipping 15) Feb 16 14:13:19.995 [12806] dbg: spf: def_spf_whitelist_from: already checked spf and didn't get pass, skipping whitelist check 16) Feb 16 14:13:19.997 [12806] dbg: spf: whitelist_from_spf: already checked spf and didn't get pass, skipping whitelist check 17) Feb 16 14:13:25.566 [12806] dbg: timing: total 8235 ms - init: 1912 (23.2%), parse: 1.82 (0.0%), extract_message_metadata: 74 (0.9%), poll_dns_idle: 381 (4.6%), get_uri_detail_list: 1.24 (0.0%), tests_pri_-1000: 27 (0.3%), compile_gen: 171 (2.1%), compile_eval: 39 (0.5%), tests_pri_-950: 9 (0.1%), tests_pri_-900: 9 (0.1%), tests_pri_-400: 8 (0.1%), tests_pri_0: 5996 (72.8%), dkim_load_modules: 33 (0.4%), check_dkim_signature: 11 (0.1%), check_spf: 389 (4.7%), check_dcc: 190 (2.3%), check_razor2: 5003 (60.8%), check_pyzor: 0.54 (0.0%), tests_pri_500: 100 (1.2%), tests_pri_1000: 15
Re: SPF and DKIM tests by default?
On 2/16/2012 4:54 PM, email builder wrote: but it's misconfigured trusted_networks and internal_networks what causes SPF to misfire... Thank you sincerely for your help. I can only imagine that SPF wouldn't fire if I accidentally specified Google in one of those settings or had an error in one of them. In this case, those are at their defaults of empty, so I'm hoping there are other suggestions. Thanks again.. Letting trusted_networks empty is not generally a good idea. In particular, if your SA server is using a private IP, it will default to trusting too much. Specify your local networks in trusted_networks and see if that helps your problem. Leaving trusted_networks empty does not mean trust nothing; it means let SA figure out what to trust. Makes sense, especially if my hunch about the relayed through one or more trusted relays, cannot use header-based Envelope-From, skipping part of the debug output I just sent to this list is on track. Is there a way to set trusted_networks on the command line of the spamassassin command just for testing?
Re: SPF and DKIM tests by default?
On 2/16/2012 4:54 PM, email builder wrote: but it's misconfigured trusted_networks and internal_networks what causes SPF to misfire... Thank you sincerely for your help. I can only imagine that SPF wouldn't fire if I accidentally specified Google in one of those settings or had an error in one of them. In this case, those are at their defaults of empty, so I'm hoping there are other suggestions. Thanks again.. Letting trusted_networks empty is not generally a good idea. In particular, if your SA server is using a private IP, it will default to trusting too much. Specify your local networks in trusted_networks and see if that helps your problem. Leaving trusted_networks empty does not mean trust nothing; it means let SA figure out what to trust. Makes sense, especially if my hunch about the relayed through one or more trusted relays, cannot use header-based Envelope-From, skipping part of the debug output I just sent to this list is on track. Is there a way to set trusted_networks on the command line of the spamassassin command just for testing? This didn't work: spamassassin -D --cf='trusted_networks 127.0.0.1' -t example_email_no_spf 21 | grep -i SPF All my local handoffs are to localhost [127.0.0.1] so I wouldn't know what else to use (it's an all-in-one single server simple system)
Re: SPF and DKIM tests by default?
Q: Will some rules not fire if some condition exists based on other rules? A: Correct. There are plenty of rules that build on other rules. We call these meta rules. OK, but: Q: Are there any default rules as supplied by sa-update that would prevent SPF rules from firing? Q: Any other ideas on how to learn what rules are actually being used? Q: Any suggestions as to why SPF rules would not fire on a Gmail message where Gmail uses SPF, my SPF plugin and rule initiation seem to be in place, and a Return-Path header with the envelope from address exists? (please see my previous messages on this thread)
Re: SPF and DKIM tests by default?
Q: Will some rules not fire if some condition exists based on other rules? A: Correct. There are plenty of rules that build on other rules. We call these meta rules. Regards, KAM
Re: SPF and DKIM tests by default?
On 2/10/2012 9:35 PM, email builder wrote:
Hi Kevin, thank you for your reply! But I think you should send it to the list
:)
Hi Thanks for bringing it back to the list. Sometimes I'm just trying
to bang out answers too quickly!
You should look in /var/lib/spamassassin. Because rules are no longer
paired to releases but released nearly continuously, there is no wiki
list of all the rules.
Gotcha - but is it certain that all rules in
/var/lib/spamassassin/3.003001/updates_spamassassin_org are being used?
No. There are many plugins, configuration options dependencies that
could affect what rules are used.
Oh, sorry for the noob question, but how do I know if I have
Mail::DKIM installed?
For example:
perl -e 'if (require Mail::DKIM) { print Mail::DKIM Version is:
$Mail::DKIM::VERSION\n; exit 0;} else {exit 1;}' || echo 'Mail::DKIM
Not Present!'
Mail::DKIM Version is: 0.37
Regards,
KAM
Re: SPF and DKIM tests by default?
On 2/10/2012 9:20 PM, email builder wrote:
Wonder if I can delete the older one
Sure. Worst case just run sa-update again if you delete the wrong one.
Hm, well is there a file or somewhere to look and see what rules are
active?
Do you mean something like: With my configuration, what rules might
possibly be triggered?
That's an interesting question. Perhaps we could use a spamassassin
parameter to run, parse config and dump all possible rules that would
run (with scores) based on all plugins, etc. that are believed to be
configured. If that is what you want, please open a bug at
https://issues.apache.org/SpamAssassin/ assuming no one knows a way this
can occur now.
I believe for SPF you *should* be doing the detecting at your MTA
(mail server software) and inserting a header for spamassassin to use:
Received-SPF. (Because SPF is supposed to use the envelope from,
which is not necessarily included in a header.)
I see. That makes sense. Is there a wiki page suggesting solutions for this?
Anyone know of tips for doing this in postfix? Or during amavis processing?
Interesting thought though while the envelope sender is not in a header
per se, it is in the From line for mbox format email, I believe. If you
are using procmail for delivery, for example, there shouldn't be an issue.
Me too. I sent emails to myself from Yahoo and Gmail and got these in my
X-Spam-Status:
Gmail: DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU
Yahoo: DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,T_DKIM_INVALID
(that last one is interesting - not sure how the message gets altered to break
the signature, especially if Gmail works fine (running SA from Maildrop))
Chasing down DKIM errors can be interesting to say the least. I found a
bug in Sendmail, for example, where it canonicalized the email address
in the To: Header which was case-sensitive on the signing so DKIM
validation failed.
Have you looked at the received headers to confirm it is in fact a valid
Yahoo! email?
I believe SPF tests are also enabled by default, but won't do quite the
right thing unless you're inserting the Received-SPF header at your MTA.
Well I guess so because I see no SPF hits and I think at least Gmail uses SPF.
I'd appreciate any tips on getting those headers inserted.
Gmail does publish SPF.
Check your *.pre files and see if you have loadplugin
Mail::SpamAssassin::Plugin::SPF
Also make sure you have Mail::SPF. This command can help determine that:
perl -e 'if (require Mail::SPF) { print Mail::SPF Version is:
$Mail::SPF::VERSION\n; exit 0;} else {exit 1;}' || echo 'Mail::SPF Not
Present!'
Mail::SPF Version is: v2.005
Regards,
KAM
Re: SPF and DKIM tests by default?
On 02/10, email builder wrote: I believe for SPF you *should* be doing the detecting at your MTA (mail server software) and inserting a header for spamassassin to use: Received-SPF. (Because SPF is supposed to use the envelope from, which is not necessarily included in a header.) I see. That makes sense. Is there a wiki page suggesting solutions for this? Anyone know of tips for doing this in postfix? Or during amavis processing? I use postfix-policyd-spf-perl. Which appears to currently be officially hosted at: https://launchpad.net/postfix-policyd-spf-perl/ -- For gasoline vapor, the explosive range is from 1.3 to 6.0% vapor to air...useful against soft targets such as...armored vehicles...and bunkers. - http://www.fas.org/man/dod-101/sys/dumb/fae.htm http://www.ChaosReigns.com
Re: SPF and DKIM tests by default?
On 2/10/2012 9:35 PM, email builder wrote:
Hi Kevin, thank you for your reply! But I think you should send it to the
list :)
Hi Thanks for bringing it back to the list. Sometimes I'm just trying
to bang out answers too quickly!
You should look in /var/lib/spamassassin. Because rules are no longer
paired to releases but released nearly continuously, there is no wiki
list of all the rules.
Gotcha - but is it certain that all rules in
/var/lib/spamassassin/3.003001/updates_spamassassin_org are being used?
No. There are many plugins, configuration options dependencies that could
affect what rules are used.
Oh OK, that's a little surprising, but I understand it can get complex, so
that's fine.
Oh, sorry for the noob question, but how do I know if I have Mail::DKIM
installed?
For example:
perl -e 'if (require Mail::DKIM) { print Mail::DKIM Version is:
$Mail::DKIM::VERSION\n; exit 0;} else {exit 1;}' || echo
'Mail::DKIM Not Present!'
Mail::DKIM Version is: 0.37
Great! Thank you
Re: SPF and DKIM tests by default?
On 2/10/2012 9:20 PM, email builder wrote:
Wonder if I can delete the older one
Sure. Worst case just run sa-update again if you delete the wrong one.
OK, thank you. I'll report back if it causes any problems but I can't imagine
it would.
Hm, well is there a file or somewhere to look and see what rules are
active?
Do you mean something like: With my configuration, what rules might possibly
be
triggered?
yes
That's an interesting question. Perhaps we could use a spamassassin
parameter to run, parse config and dump all possible rules that would run
(with
scores) based on all plugins, etc. that are believed to be configured. If
that
is what you want, please open a bug at
https://issues.apache.org/SpamAssassin/
assuming no one knows a way this can occur now.
OK it's a feature request then huh? I added it:
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6757
I believe for SPF you *should* be doing the detecting at your MTA
(mail server software) and inserting a header for spamassassin to use:
Received-SPF. (Because SPF is supposed to use the envelope
from,
which is not necessarily included in a header.)
I see. That makes sense. Is there a wiki page suggesting solutions for
this? Anyone know of tips for doing this in postfix? Or during amavis
processing?
Interesting thought though while the envelope sender is not in a header per
se,
it is in the From line for mbox format email, I believe. If you are using
procmail for delivery, for example, there shouldn't be an issue.
Actually, you're right - it seems as long as the envelope info is available you
would not need to add a new header, no? That depends if the SA SPF rules know
how to check the envelope or if they only look for a Received-SPF header.
Anyone know the details in that regard?
I use maildrop for delivery out of postfix (and SA runs from maildrop). Postfix
passes what I think is the envelope sender to maildrop by -f ${sender} (I'll
double check but I think that's accurate).
I'm uncertain if the envelope info gets to SA, though, as my maildrop call to
SA is: xfilter /usr/bin/spamc -u $LOGNAME
I'd rather not add a header if not necessary. Second choice is to do it using
amavis, as adding a policy server just for this to be pretty extreme.
Me too. I sent emails to myself from Yahoo and Gmail and got these in my
X-Spam-Status:
Gmail: DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU
Yahoo: DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,T_DKIM_INVALID
(that last one is interesting - not sure how the message gets altered to
break the signature, especially if Gmail works fine (running SA from
Maildrop))
Chasing down DKIM errors can be interesting to say the least. I found a bug
in
Sendmail, for example, where it canonicalized the email address in the To:
Header which was case-sensitive on the signing so DKIM validation failed.
Have you looked at the received headers to confirm it is in fact a valid
Yahoo!
email?
I believe SPF tests are also enabled by default, but won't do quite
the
right thing unless you're inserting the Received-SPF header at your
MTA.
Well I guess so because I see no SPF hits and I think at least Gmail uses
SPF. I'd appreciate any tips on getting those headers inserted.
Gmail does publish SPF.
Check your *.pre files and see if you have loadplugin
Mail::SpamAssassin::Plugin::SPF
Also make sure you have Mail::SPF. This command can help determine that:
perl -e 'if (require Mail::SPF) { print Mail::SPF Version is:
$Mail::SPF::VERSION\n; exit 0;} else {exit 1;}' || echo
'Mail::SPF Not Present!'
Mail::SPF Version is: v2.005
Regards,
KAM
Re: SPF and DKIM tests by default?
On 02/10, email builder wrote: I believe for SPF you *should* be doing the detecting at your MTA (mail server software) and inserting a header for spamassassin to use: Received-SPF. (Because SPF is supposed to use the envelope from, which is not necessarily included in a header.) I see. That makes sense. Is there a wiki page suggesting solutions for this? Anyone know of tips for doing this in postfix? Or during amavis processing? I use postfix-policyd-spf-perl. Which appears to currently be officially hosted at: https://launchpad.net/postfix-policyd-spf-perl/ Thanks for that, although see my last post - do you know if the SPF tests only know how to look for that Received-SPF header or can use the envelope sender if it's present?
Re: SPF and DKIM tests by default?
On Sun, 12 Feb 2012, email builder wrote:
On 02/10, email builder wrote:
I believe for SPF you *should* be doing the detecting at your MTA
(mail server software) and inserting a header for spamassassin to use:
Received-SPF. (Because SPF is supposed to use the envelope
from,
which is not necessarily included in a header.)
I see. That makes sense. Is there a wiki page suggesting solutions for
this? Anyone know of tips for doing this in postfix? Or during amavis
processing?
I use postfix-policyd-spf-perl.
Which appears to currently be officially hosted at:
https://launchpad.net/postfix-policyd-spf-perl/
Thanks for that, although see my last post - do you know if the SPF tests only
know how to look for that Received-SPF header or can use the envelope sender if
it's present?
If your MTA provides sufficient info for SA to determine the envelope
sender that is enough. I've been using sendmail+milter+sa for years
with SPF DKIM rules and never had any kind of special MTA added
'Received-SPF' header.
One thing that -is- a factor; sa depends upon specific perl modules
for that functionality; DNS, SPF, DKIM modules (EG Net::DNS, Mail::DKIM,
Mail::SPF ), and 'loadplugin' statements in the correct .pre files.
Occasionally issues arise with problematic versions of those modules.
For example, search this list archive for disussions about problems caused
by buggy versions of the DNS module.
If you execute the test:
% spamassassin --lint -D 21 | grep -i -E 'spf|dkim|dns'
You should see output that looks something like:
Feb 12 20:08:05.530 [13365] dbg: dns: is Net::DNS::Resolver available? yes
Feb 12 20:08:05.530 [13365] dbg: dns: Net::DNS version: 0.63
Feb 12 20:08:06.394 [13365] dbg: diag: [...] module installed: Net::DNS,
version 0.63
Feb 12 20:08:06.394 [13365] dbg: diag: [...] module installed: Mail::SPF,
version v2.007
Feb 12 20:08:06.395 [13365] dbg: diag: [...] module installed: Mail::DKIM,
version 0.39
Feb 12 20:08:06.409 [13365] dbg: plugin: loading
Mail::SpamAssassin::Plugin::URIDNSBL from @INC
Feb 12 20:08:06.414 [13365] dbg: plugin: loading
Mail::SpamAssassin::Plugin::SPF from @INC
Feb 12 20:08:06.429 [13365] dbg: plugin: loading
Mail::SpamAssassin::Plugin::DKIM from @INC
Feb 12 20:08:06.446 [13365] dbg: plugin: loading
Mail::SpamAssassin::Plugin::DNSEval from @INC
[snip ...]
If you don't see those 'plugin: loading' lines for SPF DKIM, then
there's your problem. Either they're not installed on your system in a
way that SA can find them, wrong verions, or not invoked by 'loadplugin'
statements.
--
Dave Funk University of Iowa
dbfunk (at) engineering.uiowa.eduCollege of Engineering
319/335-5751 FAX: 319/384-0549 1256 Seamans Center
Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527
#include std_disclaimer.h
Better is not better, 'standard' is better. B{
Re: SPF and DKIM tests by default?
On 02/10, email builder wrote: I believe for SPF you *should* be doing the detecting at your MTA (mail server software) and inserting a header for spamassassin to use: Received-SPF. (Because SPF is supposed to use the envelope from, which is not necessarily included in a header.) I see. That makes sense. Is there a wiki page suggesting solutions for this? Anyone know of tips for doing this in postfix? Or during amavis processing? I use postfix-policyd-spf-perl. Which appears to currently be officially hosted at: https://launchpad.net/postfix-policyd-spf-perl/ Thanks for that, although see my last post - do you know if the SPF tests only know how to look for that Received-SPF header or can use the envelope sender if it's present? If your MTA provides sufficient info for SA to determine the envelope sender that is enough. I agree and I've done some more research and found that Postfix adds the envelope sender as a Return-Path header (its pipe and virtual delivery agent at least do this). So I *do* have a header in my messages with the envelope sender. Either the SPF rules don't know how to look for Return-Path (which would surprise me given that it is quasi-standard and highly used) or I have some other problem. Will some rules not fire if some condition exists based on other rules? I've been using sendmail+milter+sa for years with SPF DKIM rules and never had any kind of special MTA added 'Received-SPF' header. OK, good. One thing that -is- a factor; sa depends upon specific perl modules for that functionality; DNS, SPF, DKIM modules (EG Net::DNS, Mail::DKIM, Mail::SPF ), and 'loadplugin' statements in the correct .pre files. I think I forgot to reply to the hints on checking for the SPF module earlier in this thread, but I do have it installed. And the SPF plugin is loaded from init.pre (is that OK?). Occasionally issues arise with problematic versions of those modules. For example, search this list archive for disussions about problems caused by buggy versions of the DNS module. If you execute the test: % spamassassin --lint -D 21 | grep -i -E 'spf|dkim|dns' [snip ...] If you don't see those 'plugin: loading' lines for SPF DKIM, then there's your problem. Either they're not installed on your system in a way that SA can find them, wrong verions, or not invoked by 'loadplugin' statements. Thanks that was helpful, and I did in fact find the plugin loading for the SPF plugin, so it's there, but I'm not getting hits on messages from Gmail which does use SPF. Hmmm any other suggestions anyone? Thanks for the excellent help so far!
Re: SPF and DKIM tests by default?
Thanks a lot for your reply Run: sa-update -D 21| grep DIR That will output something like: Feb 9 12:08:49.609 [20855] dbg: generic: Perl 5.010001, PREFIX=/usr, DEF_RULES_DIR=/usr/share/spamassassin, LOCAL_RULES_DIR=/etc/spamassassin, LOCAL_STATE_DIR=/var/lib/spamassassin On this system, sa-update downloads rules to /var/lib/spamassassin, so I guess you're looking for the LOCAL_STATE_DIR. OK, makes sense. Mine is the same as yours. That directory will contain a directory related to your SA version, something like 3.003001, which will contain updates_spamassassin_org, which will contain the files defining all the rules. Hmm, in there I find TWO directories: 3.002005 3.003001 Strangely, both have dates of today, but the *contents* of 3.002005 are from Apr 3 2011. So I guess my system uses 3.003001 since it's files are dated currently Wonder if I can delete the older one Although that doesn't necessarily tell you which are enabled by default. Some require configuration changes. Hm, well is there a file or somewhere to look and see what rules are active? I believe for SPF you *should* be doing the detecting at your MTA (mail server software) and inserting a header for spamassassin to use: Received-SPF. (Because SPF is supposed to use the envelope from, which is not necessarily included in a header.) I see. That makes sense. Is there a wiki page suggesting solutions for this? Anyone know of tips for doing this in postfix? Or during amavis processing? From that page, it seems that SPF checks are normal but DKIM is not. Is this right? Contrary to that, this page suggests that DKIM test are enabled by default in version 3.3: https://wiki.apache.org/spamassassin/Plugin/DKIM I don't have anything in my /etc/spamassassin/local.cf related to DKIM, and I'm getting DKIM rule hits, so I agree that DKIM is enabled by default (although I'm running trunk / v3.4.0 which is unreleased). Me too. I sent emails to myself from Yahoo and Gmail and got these in my X-Spam-Status: Gmail: DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU Yahoo: DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,T_DKIM_INVALID (that last one is interesting - not sure how the message gets altered to break the signature, especially if Gmail works fine (running SA from Maildrop)) I believe SPF tests are also enabled by default, but won't do quite the right thing unless you're inserting the Received-SPF header at your MTA. Well I guess so because I see no SPF hits and I think at least Gmail uses SPF. I'd appreciate any tips on getting those headers inserted. None of the SPF or DKIM rules are particularly highly ranked in spamassassin rule QA, so I wouldn't actually expect significant improvements in accuracy from it: http://ruleqa.spamassassin.org/?daterev=20120204 They both have some substantial flaws. I'm OK with that (have been weary about their limitations and not always 100% sure about using either of them on my domains), and it's actually the reason I'm asking about SA support for them because I would never want to use either of them to outright block mail.Just some influence on SA scoring is good.
Re: SPF and DKIM tests by default?
From: Kevin A. McGrail kmcgr...@pccc.com Hi Kevin, thank you for your reply! But I think you should send it to the list :) Is this the right place to look to know what tests the server should be running? https://spamassassin.apache.org/tests_3_0_x.html You should look in /var/lib/spamassassin. Because rules are no longer paired to releases but released nearly continuously, there is no wiki list of all the rules. Gotcha - but is it certain that all rules in /var/lib/spamassassin/3.003001/updates_spamassassin_org are being used? From that page, it seems that SPF checks are normal but DKIM is not. Is this right? Contrary to that, this page suggests that DKIM test are enabled by default in version 3.3: https://wiki.apache.org/spamassassin/Plugin/DKIM Yes, 3.1.2 enabled DKIM by default if you have Mail::DKIM installed, I believe. Oh, sorry for the noob question, but how do I know if I have Mail::DKIM installed? Also, where can I look to verify the tests/rules currently in place on the server? (per-user rules are not implemented) In the version dir under /var/lib/spamassassin. I looked in /usr/share/spamassassin and there are a few files with spf and dkim in their names. Does that mean those tests are active? ls *spf* -rw-r--r-- 1 root root 3100 Mar 15 2010 25_spf.cf -rw-r--r-- 1 root root 3584 Mar 15 2010 60_whitelist_spf.cf ls *dkim* -rw-r--r-- 1 root root 4407 Mar 15 2010 25_dkim.cf -rw-r--r-- 1 root root 9288 Mar 15 2010 60_adsp_override_dkim.cf -rw-r--r-- 1 root root 6455 Mar 15 2010 60_whitelist_dkim.cf I believe SPF and DKIM are enabled by default but that doesn't mean you have all the supporting modules installed. Did you configure the installation yourself or did you use a package? I used yum to install a package on centOS
Re: SPF and DKIM tests by default?
On 02/08, email builder wrote: Hello, I have a server where I never customized any of the SA rules/tests (SA v.3.3.1). The server does run sa-update every day. Is this the right place to look to know what tests the server should be running? https://spamassassin.apache.org/tests_3_0_x.html At the top of that page, it says Tests Performed: v3.0.x which is not the version you are running. https://spamassassin.apache.org/tests_3_3_x.html contains tests for 3.3. I don't know when they get updated, maybe only when 3.3.0 was released. I wouldn't trust it much. Run: sa-update -D 21| grep DIR That will output something like: Feb 9 12:08:49.609 [20855] dbg: generic: Perl 5.010001, PREFIX=/usr, DEF_RULES_DIR=/usr/share/spamassassin, LOCAL_RULES_DIR=/etc/spamassassin, LOCAL_STATE_DIR=/var/lib/spamassassin On this system, sa-update downloads rules to /var/lib/spamassassin, so I guess you're looking for the LOCAL_STATE_DIR. That directory will contain a directory related to your SA version, something like 3.003001, which will contain updates_spamassassin_org, which will contain the files defining all the rules. Although that doesn't necessarily tell you which are enabled by default. Some require configuration changes. I believe for SPF you *should* be doing the detecting at your MTA (mail server software) and inserting a header for spamassassin to use: Received-SPF. (Because SPF is supposed to use the envelope from, which is not necessarily included in a header.) From that page, it seems that SPF checks are normal but DKIM is not. Is this right? Contrary to that, this page suggests that DKIM test are enabled by default in version 3.3: https://wiki.apache.org/spamassassin/Plugin/DKIM I don't have anything in my /etc/spamassassin/local.cf related to DKIM, and I'm getting DKIM rule hits, so I agree that DKIM is enabled by default (although I'm running trunk / v3.4.0 which is unreleased). I believe SPF tests are also enabled by default, but won't do quite the right thing unless you're inserting the Received-SPF header at your MTA. Also, where can I look to verify the tests/rules currently in place on the server? (per-user rules are not implemented) I looked in /usr/share/spamassassin and there are a few files with spf and dkim in their names. Does that mean those tests are active? Using the official Debian / Ubuntu packages, that directory contains the rules installed by the spamassassin package, which are only used if you do not run sa-update. Which would obviously be sub-optimal. ls *spf* -rw-r--r-- 1 root root 3100 Mar 15 2010 25_spf.cf -rw-r--r-- 1 root root 3584 Mar 15 2010 60_whitelist_spf.cf ls *dkim* -rw-r--r-- 1 root root 4407 Mar 15 2010 25_dkim.cf -rw-r--r-- 1 root root 9288 Mar 15 2010 60_adsp_override_dkim.cf -rw-r--r-- 1 root root 6455 Mar 15 2010 60_whitelist_dkim.cf Those are related, although their presence doesn't indicate anything about defaults. None of the SPF or DKIM rules are particularly highly ranked in spamassassin rule QA, so I wouldn't actually expect significant improvements in accuracy from it: http://ruleqa.spamassassin.org/?daterev=20120204 They both have some substantial flaws. -- Every man, woman and child on the face of this earth is at the mercy of chaos. - a maxwell smart movie http://www.ChaosReigns.com
SPF and DKIM tests by default?
Hello, I have a server where I never customized any of the SA rules/tests (SA v.3.3.1). The server does run sa-update every day. Is this the right place to look to know what tests the server should be running? https://spamassassin.apache.org/tests_3_0_x.html From that page, it seems that SPF checks are normal but DKIM is not. Is this right? Contrary to that, this page suggests that DKIM test are enabled by default in version 3.3: https://wiki.apache.org/spamassassin/Plugin/DKIM Also, where can I look to verify the tests/rules currently in place on the server? (per-user rules are not implemented) I looked in /usr/share/spamassassin and there are a few files with spf and dkim in their names. Does that mean those tests are active? ls *spf* -rw-r--r-- 1 root root 3100 Mar 15 2010 25_spf.cf -rw-r--r-- 1 root root 3584 Mar 15 2010 60_whitelist_spf.cf ls *dkim* -rw-r--r-- 1 root root 4407 Mar 15 2010 25_dkim.cf -rw-r--r-- 1 root root 9288 Mar 15 2010 60_adsp_override_dkim.cf -rw-r--r-- 1 root root 6455 Mar 15 2010 60_whitelist_dkim.cf
