Re: SPF should always hit? SOLVED
Am 11.07.2016 um 21:02 schrieb David B Funk: On Mon, 11 Jul 2016, Reindl Harald wrote: SA has also a weakness or design mistake here "envelope_sender_header X-Local-Envelope-From" while that header comes from postfix with customized configuration because we use it in own rules has no fallback __ By default, various MTAs will use different headers, such as the following: X-Envelope-From Envelope-Sender X-Sender Return-Path __ well, in case of "envelope_sender_header" present in the configuration and that header is missing for whatever reason there is *no fallback* while for most cases it would be better to use "envelope_sender_header" as prefered one instead the only one that it is not the case can you see when "add_header all Status _YESNO_, score=_SCORE_, tag-level=_REQD_, block-level=8.0, envelope=_SENDERDOMAIN_, from=_AUTHORDOMAIN_, _TOKENSUMMARY_" ends with SENDERDOMAIN_ in your headers The SA Conf man page seems to indicate that it -should- fall back to its heuristic if the envelope_sender_header is missing: To avoid this heuristic failure, the "envelope_sender_header" setting may be helpful. Name the header that your MTA or MDA adds to messages containing the address used at the MAIL FROM step of the SMTP transaction. If the header in question contains "<" or ">" characters at the start and end of the email address in the right-hand side, as in the SMTP transaction, these will be stripped. If the header is not found in a message, or if it's value does not contain an "@" sign, SpamAssassin will issue a warning in the logs and fall back to its default heuristics. It doesn't look like that fall-back is working. If you completely omit the envelope_sender_header config setting, the heuristic works. Maybe you should file a bug-report. looks so One additional question, if you're setting the envelope_sender_header configwhy aren't you actually supplying it? because i have *no idea* from where it comes that postfix sometimes ignores check_sender_access proxy:pcre:/etc/postfix/x_envelope_from.cf check_recipient_access proxy:pcre:/etc/postfix/x_envelope_to.cf If you cannot depend upon your system to actually supply the header you list in your envelope_sender_header config, then don't set that parameter well, the idea is to add a own heaer in the MTA instead rely on heuristic which hopefully don't use a randm but wrong header (if that would be impossible the other problem also won't exist) signature.asc Description: OpenPGP digital signature
Re: SPF should always hit? SOLVED
On Mon, 11 Jul 2016, Reindl Harald wrote:
Am 11.07.2016 um 19:30 schrieb RW:
[snip..]
It sounds like SA is not able to parse the envelope sender out of the
headers.
See the description for envelope_sender_header in
man Mail::SpamAssassin::Conf
SA has also a weakness or design mistake here
"envelope_sender_header X-Local-Envelope-From" while that header comes from
postfix with customized configuration because we use it in own rules has no
fallback
__
By default, various MTAs will use different headers, such as the following:
X-Envelope-From
Envelope-Sender
X-Sender
Return-Path
__
well, in case of "envelope_sender_header" present in the configuration and
that header is missing for whatever reason there is *no fallback* while for
most cases it would be better to use "envelope_sender_header" as prefered one
instead the only one
that it is not the case can you see when "add_header all Status _YESNO_,
score=_SCORE_, tag-level=_REQD_, block-level=8.0, envelope=_SENDERDOMAIN_,
from=_AUTHORDOMAIN_, _TOKENSUMMARY_" ends with SENDERDOMAIN_ in your headers
The SA Conf man page seems to indicate that it -should- fall back to its
heuristic if the envelope_sender_header is missing:
To avoid this heuristic failure, the "envelope_sender_header" setting may be
helpful. Name
the header that your MTA or MDA adds to messages containing the address used
at the MAIL
FROM step of the SMTP transaction.
If the header in question contains "<" or ">" characters at the start and
end of the email
address in the right-hand side, as in the SMTP transaction, these will be
stripped.
If the header is not found in a message, or if it's value does not contain an
"@" sign,
SpamAssassin will issue a warning in the logs and fall back to its default
heuristics.
It doesn't look like that fall-back is working. If you completely omit the
envelope_sender_header config setting, the heuristic works.
Maybe you should file a bug-report.
One additional question, if you're setting the envelope_sender_header config
why aren't you actually supplying it?
If you cannot depend upon your system to actually supply the header you list
in your envelope_sender_header config, then don't set that parameter.
--
Dave Funk University of Iowa
College of Engineering
319/335-5751 FAX: 319/384-0549 1256 Seamans Center
Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527
#include
Better is not better, 'standard' is better. B{
smime.p7s
Description: S/MIME Cryptographic Signature
Re: SPF should always hit? SOLVED
Am 11.07.2016 um 19:30 schrieb RW: On Mon, 11 Jul 2016 12:49:04 -0400 Robert Fitzpatrick wrote: I finally was able to get SPF checks to be more reliable by making sure Postfix SPF policies were in place. Here is a good read https://github.com/mail-in-a-box/mailinabox/issues/698 Excerpt: It's worth noting that lack of postfix's spf checker renders spamassassin's flagging impaired because without it spamassassin in my case is only adding helo_pass and that's all regarding spfs. It sounds like SA is not able to parse the envelope sender out of the headers. See the description for envelope_sender_header in man Mail::SpamAssassin::Conf SA has also a weakness or design mistake here "envelope_sender_header X-Local-Envelope-From" while that header comes from postfix with customized configuration because we use it in own rules has no fallback __ By default, various MTAs will use different headers, such as the following: X-Envelope-From Envelope-Sender X-Sender Return-Path __ well, in case of "envelope_sender_header" present in the configuration and that header is missing for whatever reason there is *no fallback* while for most cases it would be better to use "envelope_sender_header" as prefered one instead the only one that it is not the case can you see when "add_header all Status _YESNO_, score=_SCORE_, tag-level=_REQD_, block-level=8.0, envelope=_SENDERDOMAIN_, from=_AUTHORDOMAIN_, _TOKENSUMMARY_" ends with SENDERDOMAIN_ in your headers signature.asc Description: OpenPGP digital signature
Re: SPF should always hit? SOLVED
On Mon, 11 Jul 2016 12:49:04 -0400 Robert Fitzpatrick wrote: > I finally was able to get SPF checks to be more reliable by making > sure Postfix SPF policies were in place. Here is a good read > > https://github.com/mail-in-a-box/mailinabox/issues/698 > Excerpt: It's worth noting that lack of postfix's spf checker renders > spamassassin's flagging impaired because without it spamassassin in > my case is only adding helo_pass and that's all regarding spfs. It sounds like SA is not able to parse the envelope sender out of the headers. See the description for envelope_sender_header in man Mail::SpamAssassin::Conf
Re: SPF should always hit? SOLVED
Robert Fitzpatrick wrote: Joe Quinn wrote: On 6/9/2016 11:23 AM, Robert Fitzpatrick wrote: Excuse me if this is too lame a question, but I have the SPF plugin enabled and it hits a lot. Should SPF_ something hit on every message if the domain has an SPF record in DNS? Furthermore, a message found as Google phishing did not get a hit on a email address where the domain has SPF setup. Not sure if it would fail anyway if the envelope from is the culprit? In a perfect world, every message you scan will hit one of the following: SPF_HELO_NONE SPF_HELO_NEUTRAL SPF_HELO_PASS SPF_HELO_FAIL SPF_HELO_SOFTFAIL T_SPF_HELO_PERMERROR T_SPF_HELO_TEMPERROR And additionally one of the following: SPF_NONE SPF_NEUTRAL SPF_PASS SPF_FAIL SPF_SOFTFAIL T_SPF_PERMERROR T_SPF_TEMPERROR I finally was able to get SPF checks to be more reliable by making sure Postfix SPF policies were in place. Here is a good read https://github.com/mail-in-a-box/mailinabox/issues/698 Excerpt: It's worth noting that lack of postfix's spf checker renders spamassassin's flagging impaired because without it spamassassin in my case is only adding helo_pass and that's all regarding spfs. Once we got Postfix SPF checks setup using the Python version and disabling rejects in the config, we now have headers we can be sure are handled by our custom rules in addition to any SA checks. -- Robert
Re: SPF should always hit?
Joe Quinn wrote: On 6/9/2016 11:23 AM, Robert Fitzpatrick wrote: Excuse me if this is too lame a question, but I have the SPF plugin enabled and it hits a lot. Should SPF_ something hit on every message if the domain has an SPF record in DNS? Furthermore, a message found as Google phishing did not get a hit on a email address where the domain has SPF setup. Not sure if it would fail anyway if the envelope from is the culprit? In a perfect world, every message you scan will hit one of the following: SPF_HELO_NONE SPF_HELO_NEUTRAL SPF_HELO_PASS SPF_HELO_FAIL SPF_HELO_SOFTFAIL T_SPF_HELO_PERMERROR T_SPF_HELO_TEMPERROR And additionally one of the following: SPF_NONE SPF_NEUTRAL SPF_PASS SPF_FAIL SPF_SOFTFAIL T_SPF_PERMERROR T_SPF_TEMPERROR In practice, there's almost certainly a few edge cases where messages can avoid getting one in either category. For purposes of writing your own metas against these, the rules that matter most for measuring spamminess are the none, pass, and fail/softfail results. The rest are for total coverage of the results that an SPF query can yield, for debugging and documentation purposes. Also, none of these will hit at all if you disable network tests. Yes, network tests are on. I have lots of messages hitting, it is harder to find one that doesn't have hits as you suggested. However, I can find several out of our database of 280K messages cached which do not hit any of these rules. So, what would be a reason they didn't hit? The only custom rule I have with SPF_* is with SPF_FAIL combined without DKIM to give higher score: meta WT_FORGED_SENDER (SPF_FAIL && !DKIM_VALID) describe WT_FORGED_SENDER To score high when SPF fails without valid DKIM scoreWT_FORGED_SENDER 8.0 Here is the score for this particular example: 2.095 FREEMAIL_FORGED_REPLYTO Freemail in Reply-To, but not From 1.000 XPRIO_SHORT_SUBJ(No description provided) 0.250 FREEMAIL_REPLYTO_END_DIGIT Reply-To freemail username ends in digit 0.001 HTML_MESSAGEHTML included in message 0.001 HEADER_FROM_DIFFERENT_DOMAINS (No description provided) 0.000 RCVD_IN_DNSWL_NONE Sender listed at http://www.dnswl.org/, low trust -1.900 BAYES_00Bayesian spam probability is 0 to 1% -5.000 RCVD_IN_JMF_W (No description provided) -- Robert
Re: SPF should always hit?
On 6/9/2016 11:23 AM, Robert Fitzpatrick wrote: Excuse me if this is too lame a question, but I have the SPF plugin enabled and it hits a lot. Should SPF_ something hit on every message if the domain has an SPF record in DNS? Furthermore, a message found as Google phishing did not get a hit on a email address where the domain has SPF setup. Not sure if it would fail anyway if the envelope from is the culprit? In a perfect world, every message you scan will hit one of the following: SPF_HELO_NONE SPF_HELO_NEUTRAL SPF_HELO_PASS SPF_HELO_FAIL SPF_HELO_SOFTFAIL T_SPF_HELO_PERMERROR T_SPF_HELO_TEMPERROR And additionally one of the following: SPF_NONE SPF_NEUTRAL SPF_PASS SPF_FAIL SPF_SOFTFAIL T_SPF_PERMERROR T_SPF_TEMPERROR In practice, there's almost certainly a few edge cases where messages can avoid getting one in either category. For purposes of writing your own metas against these, the rules that matter most for measuring spamminess are the none, pass, and fail/softfail results. The rest are for total coverage of the results that an SPF query can yield, for debugging and documentation purposes. Also, none of these will hit at all if you disable network tests.
Re: SPF should always hit?
Am 09.06.2016 um 17:23 schrieb Robert Fitzpatrick: Excuse me if this is too lame a question, but I have the SPF plugin enabled and it hits a lot. Should SPF_ something hit on every message if the domain has an SPF record in DNS? and if it's SPF_NONE Furthermore, a message found as Google phishing did not get a hit on a email address where the domain has SPF setup. Not sure if it would fail anyway if the envelope from is the culprit? SPF is always about envelopes (no trolling about sender-id needed this time) and with the infos you provided what do you expect for answers? just read https://en.wikipedia.org/wiki/Sender_Policy_Framework SPF is more than just "fail" T_SPF_PERMERROR T_SPF_TEMPERROR T_SPF_HELO_PERMERROR T_SPF_HELO_TEMPERROR SPF_FAIL SPF_HELO_FAIL SPF_HELO_NEUTRAL SPF_HELO_NONE SPF_HELO_PASS SPF_HELO_SOFTFAIL SPF_NEUTRAL SPF_NONE SPF_PASS SPF_SOFTFAIL signature.asc Description: OpenPGP digital signature
SPF should always hit?
Excuse me if this is too lame a question, but I have the SPF plugin enabled and it hits a lot. Should SPF_ something hit on every message if the domain has an SPF record in DNS? Furthermore, a message found as Google phishing did not get a hit on a email address where the domain has SPF setup. Not sure if it would fail anyway if the envelope from is the culprit? -- Robert
