Please read CVE-2019-20790, and tell me you’ve found a way to tell
PyPolicyd not to trust the SMTP HELO to generate a passing AR
header.
This is possibly off-topic for the Postfix list... maybe not.
…with tongue in cheek, I’ll take all the discussion about mail
filtering over any about political correctness that’s happened this
year. That said, if people feel this is off topic, nudge me
privately and I’ll take a hint.
Ditto - perhaps some of these comments are better placed at
https://github.com/trusteddomainproject/OpenDMARC/issues/49
Re the risk from the CVE to my setup...
I run policyd-spf as a postfix check_policy with reject on fail (on
both HELO and MAILFROM), and assuming a mail gets past that (99%
do), I then use that Pass along with OpenDKIM and OpenDMARC to
contribute to Spamassassin scoring. My DMARC Spamassassin rules do
not apply a -ve on a pass, only a +ve on a fail.
The impact of a false pass from DMARC (particularly as our Webmail
system - Horde - does not show any added trust for a dmarc pass)
would be failure to apply a negative Spamassassin score from a the
missed DMARC fail. With my mail setup and users the impact of that
is probably manageable... but agreed it is not ideal.
What concerns me as much as anything is that each of the apps' devs
in question (pypolicyd-spf and opendmarc) are each pointing at each
other :-/
https://bugs.launchpad.net/pypolicyd-spf/+bug/1838816/comments/4:
"my belief that the issue here is a DMARC processor failing to do
alignment validation correctly"
https://github.com/trusteddomainproject/OpenDMARC/issues/49#issuecomment-802242666
"Honestly, this CVE seems more to belong to py-policyd than OpenDMARC."
What would be nice would be if the two of you worked together on
coming up with a fix...
The upcoming 1.4.1 will have a more detailed explanation of the CVE
in question as we go over the decisions that we made in taking
action on this.
My logic in writing up the CVE summary was: OpenDMARC is following
the protocols as designed. If an earlier milter in the chain says
SPF passed, it doesn’t feel right to pick that apart and say “but
did it really?”
This is (IMO) flawed reasoning. Hear me out... :)
The *sole* scope of pypolicyd-spf is *SPF* validation at the SMTP
transaction point. RFC7208 recommends that SPF verifiers check both
HELO and MAIL FROM (2.3), and pypolicyd-spf provides users with the
ability to choose whether to do MAIL FROM only or also HELO. Out of
the box it aligns with RFC 7208's recommended approach, checking both.
There is nothing in the SPF RFC which says *for the purposes of SPF*
that either HELO or MAIL FROM validation is more or less valid than
the other. As defined in SPF, both are valid. We know SPF is not
perfect, which is why it has been extended and added to by things like
DMARC... but remember this is at the moment *solely in context of an
SPF policy agent*.
Pypolicyd-spf then tags what has driven the result for later use:
E.g.
Apr 3 11:19:23 emp87 policyd-spf[1336326]: prepend
Authentication-Results: mail.simonandkate.net; spf=pass (mailfrom)
Apr 2 12:32:51 emp87 policyd-spf[1255235]: prepend
Authentication-Results: mail.simonandkate.net; spf=pass (helo)
Whether or not MTAs upstream have enforced matches of hostname to
EHLO, Pypolicyd-spf is meeting its defined brief. Whether or not you
think "trusting HELO is insane" it is clearly a valid interpretation
of the SPF RFC for an SPY policy agent to assign a pass based on HELO.
It is *not* "a bad authentication results header" if pypolicyd-spf
tags as such (quotes from
https://github.com/trusteddomainproject/OpenDMARC/issues/49#issuecomment-803634646,
https://github.com/trusteddomainproject/OpenDMARC/issues/49#issuecomment-800047724).
1. Setting pypolicyd to HELO_Reject = No_Check. RFC7208 indicates
that MAILFROM and HELO are both equally valid for an SPF result,
*BUT* the vast majority of my SPF checks trigger pass or fail on
MAILFROM (I've had 3 spf HELO passes in 10 days on HELO, thousands
on MAILFROM). Testing pypolicyd-spf with HELO_reject = No_Check
results in spf=none for those. Removing the HELO check means that
OpenDMARC can assume any spf result it is given is based on MAILFROM.
RFC7489 Section 4.1: "DMARC uses the result of SPF authentication
of the MAIL FROM identity.”
If we’re processing SPF ourselves, this is true. (The SPF RFC calls
HELO checking optional). If we’re parsing an earlier added header
(perhaps added by a border MX), we prioritize spf=pass over other
decisions. (Perhaps this is wrong to do).
With regard to OpenDMARC, we *could* add a check on an earlier SPF
header that checks *ONLY* the mail-from identity in a received-spf
header. This might be a better response to the CVE for future code,
but at the very least right now a workaround and a statement is
warranted.
So now we come to DMARC.
The DMARC RFC7489 says at 3.1 about SPF:
"while [SPF] can authenticate either the domain that appears in the
RFC5321.MailFrom (MAIL FROM) portion of [SMTP] or the RFC5321.EHLO/
HELO domain, or both"
...acknowledging that a valid SPF pass can be based on HELO.
It then says (4.1) that DMARC *must only use SPF MAIL FROM* authentication:
"DMARC uses the result of SPF authentication of the MAIL FROM identity"
pypolicyd-spf is giving OpenDMARC the necessary information to check
whether this is achievable. Look at my example log entries as noted
above with the prepend headers:
spf=pass (mailfrom)
spf=pass (helo)
OpenDMARC has the information needed to make an informed decision on
whether the SPF pass which has been made earlier can actually be used
in alignment with the DMARC RFC (i.e. based on MAIL FROM) or should be
ignored (i.e. based on HELO). If OpenDMARC stops parsing at "spf=pass"
how is that an issue with anything other than OpenDMARC? The DMARC RFC
*explicitly* says that DMARC must only use an SPF authentication of
MAIL FROM - It does not say that DMARC should trust an spf=pass.
Your comment: "we *could* add a check on an earlier SPF header that
checks *ONLY* the mail-from identity in a received-spf header" strikes
me as being the sensible fix. In fact, if OpenDMARC is *NOT* checking
against only an SPF MAILFROM pass, perhaps you can enlighten me on how
it is compliant with RFC7489 4.1?
In turn, that could lead users to point at an SPF=pass in a previous
A/R or received-spf header, and complain that we’re still
rejecting. It could also lead OpenDMARC to add its own SPF A/R
header with a different result than a previous one, which could
greater complicate things down the chain. Which header does a
downstream filter (say, Spamd) believe?
Again - spf=pass is not the issue.
Rejecting based on an SPF policy such as -all is carefully cautioned
when used with DMARC (RFC7489 10.1). Since SPF is broken by pretty
much every mailing list, and mailing lists are not SPF aware (but
some ARE dmarc aware), using py-policyd to reject is probably not a
great idea anyway, since as far as I know py-policyd doesn’t know
how to check if a _dmarc record also exists.
Not true.
I currently reject SPF on fail, am on dozens of mailing lists without
issue (I should say without *SPF* issues... ignoring issues like the
OpenDMARC mailing list's use of deprecated TLS protocols). I usually
get spf=none from those. I currently have only 1 dumb domain in my SPF
skip-check whitelist (I've contacted them, no change). On my small
server this results in (after extensive postscreen activity dropping
about 90% of connection attempts) typically 2 or 3 SPF rejects a day
(out of 300 to 500 emails. Of those, maybe 1 every three or four weeks
is a false positive. Yes I could accept and pass further down the line
for evaluation and e.g. DMARC tagging - I choose not to.
Your interpretation of RFC7489 10.1 is also flawed (again IMO)...
"Some receiver architectures might implement SPF in advance of any
DMARC operations. This means that a "-" prefix on a sender's SPF
mechanism, such as "-all", could cause that rejection to go into
effect early in handling, causing message rejection before any DMARC
processing takes place. Operators choosing to use "-all" should be
aware of this."
That places the onus on domain owners, not receivers. It acknowledges
that some receivers do SPF before DMARC and that may result in
rejection before DMARC - it then cautions Operators to be aware of
that risk. It is entirely valid under the SPF RFC for a receiver to
reject messages based on an SPF Fail (RFC7208 G.2)... and I choose to
do so before I get to DMARC.
Our advice to the administrators is to carefully evaluate the
behavior of this software — be aware that this is a failure mode,
and act accordingly.
I don’t use py-policyd, so I don’t know what else it can be used to
do (for example, if it’s a general purpose wrapper like Amavis is).
I don’t feel that we should be writing documentation or suggesting
specific configuration knobs for software we didn’t put out.
The CVE response is visible in the SECURITY folder in our github on
the “develop” branch. This specific callout will probably also be
in the RELEASE NOTES, which we’re still writing.
Fundamentally I agree with Scott Kitterman here
(https://bugs.launchpad.net/pypolicyd-spf/+bug/1838816/comments/3):
"If you look at https://tools.ietf.org/html/rfc7208#section-4.1 you can see
that both HELO and Mail From are treated as first class inputs to
check_host().
From a specification perspective, the result needs to consider what
identity it
relates to, which is why I think ***a DMARC processor that assumes any
reported
SPF result relates to the Mail From of the message is buggy***.
Scott K" (***emphasis added***)
Simon
--
Simon Wilson
M: 0400 12 11 16
