Re: python-policyd-spf doesn't check mail from my own domain

[Dominic Raferd]

On 31 January 2018 at 09:48, li...@lazygranch.com  wrote:
>
> I'm at a loss on the HELO_reject = False. Why wouldn't you use the
> default "fail".

I use opendmarc to pass or fail emails, I never fail an email based
only on SPF. That can lead to lots of false positives IMO. All I want
the spf test for is as input to the dmarc test. If using policyd-spf,
all that is wanted is the header that will then be read by opendmarc.

>
> Reading some chatter on the opendmarc forum, they suggested
> HEADER_Type = AR as you have done, but I don't understand the
> Authserv_Id field.
>
> I don't think opendmarc has an issue reading the spf format.

You may be right, in which case the header type shouldn't matter.
However the Authserv_Id field is critical - opendmarc should only
trust spf (and dkim) headers (as added by policyd.spf and opendkim)
which have an authserv_id field matching the TrustedAuthServIDs field
in /etc/opendmarc.conf (which defaults to opendmarc's AuthservID
field) - this is so that they can't be faked by incoming mails. (Even
if they are faked, the headers added by your opendkim and policyd-spf
should be more recent and therefore take priority.) There is no
default for this in policyd-spf, and I'm not sure whether/how it shows
it in the added header if you don't specify AR
(Authentication-Results) type-header. The default in opendkim and
opendmarc is the hostname.
>
> I did re-read the policyd-spf.conf and decided to do whitelisting here
> rather than in opendmarc. I get email from pobox.com, which always
> fails.

I don't know whether policyd-spf adds a header if it whitelists an
incoming mail. If not, and the email is not whitelisted by opendmarc,
opendmarc can only validate the email with 'SPFSelfValidate true' in
/etc/opendmarc.conf (when it will do its own SPF analysis). But once
you are relying on opendmarc's internal SPF analysis for some emails
you have made policyd-spf redundant and complicated your life; if you
want to use policyd-spf I think you should leave SPFSelfValidate and
SPFIgnoreResults unset (i.e. default false).

Re: python-policyd-spf doesn't check mail from my own domain

[li...@lazygranch.com]

On Wed, 31 Jan 2018 07:43:17 + (UTC)
Dominic Raferd  wrote:

> On 31 January 2018 at 03:44, li...@lazygranch.com
>  wrote:
> > On Tue, 30 Jan 2018 10:50:18 +
> > Dominic Raferd  wrote:
> >  
> >> On 30 January 2018 at 10:11, li...@lazygranch.com
> >>  wrote:  
> >> > I've installed the opendmarc milter. I'm not rejecting mail from
> >> > it at the moment. I've noticed that if I send myself a message,
> >> > the policyd-spf milter isn't run. That in turn causes mail I send
> >> > myself to fail in opendmarc. Any ideas?
> >> >
> >> > The various email verifiers do show that my email passes spf.
> >> >
> >> > It is easy enough just to whitelist your own domains from
> >> > opendmarc, but that would allow spoofed email to get through.  
> >>
> >> Which version of opendmarc? (opendmarc -V) If you have 1.3.2+ you
> >> can use opendmarc's own spf instead (SPFSelfValidate True) - not
> >> reliable for earlier versions though.
> >>
> >> Anyway, in general:
> >>
> >> /etc/opendmarc.conf:
> >> ...
> >> IgnoreAuthenticatedClients true
> >> IgnoreHosts /etc/postfix/opendmarc-ignorehosts.txt
> >> ...
> >>
> >> /etc/opendkim.conf:
> >> ...
> >> InternalHosts /etc/postfix/opendmarc-ignorehosts.txt
> >> ...
> >>
> >> /etc/postfix/opendmarc-ignorehosts.txt
> >> # emails from localhost are not authenticated but should be signed
> >> by opendkim and not tested by opendmarc
> >> 127.0.0.1
> >> # similarly any ips from which we accept unauthenticated
> >> originating emails (e.g. lan, or none)  
> >
> >
> > opendmarc: OpenDMARC Filter v1.3.2
> > SMFI_VERSION 0x101
> > libmilter version 1.0.1
> > Active code options:
> > WITH_SPF
> > WITH_SPF2
> >
> > I suppose it is dumb to check spf if authenticated, but then again
> > dkim is checked.
> >
> > I will work on the bypasses as suggested. I kind of like the
> > python-policyd-spf since...well...it is working. (Something that
> > works is something I don't like to change.)
> >
> > Still I wonder what part of the email food chain determines that spf
> > wasn't needed. I commented out the local reference in
> > pythod-policyd-spf, but that didn't change anything.
> >
> > Lots of spam gets marked as fail in opendmarc. I can't wait to start
> > "trusting" it.  
> 
> It shouldn't be a problem to continue using python-policyd-spf. You
> would expect it to give a fail when testing mail from authenticated
> clients. Opendkim needs to run in such cases not to test them but to
> add the dkim header.
> 
> I use opendmarc (obvs) but I have to say I don't see it blocking many
> emails. Looking at my records over a few months: 38000 mails came
> through of which 50 were rejected by opendmarc and 30 quarantined. Of
> those 80, 34 appear to have come via mailing lists (including
> postfix.org) so may just reflect senders using the mailing list but
> with incompatible dmarc settings on their domain. The reality is that
> comparatively few domains are set up with dmarc and with p=reject (or
> p=quarantine). If you see a large number of opendmarc fails (in
> opendmarc log: action!=2) then I fear there is something wrong with
> your setup.
> 
> Here is my entire opendmarc.conf:
> 
> PidFile /var/run/opendmarc/opendmarc.pid
> RejectFailures true
> Syslog true
> UMask 0002
> UserID opendmarc:opendmarc
> PublicSuffixList /usr/share/publicsuffix/public_suffix_list.dat
> IgnoreAuthenticatedClients true
> AuthservID  myauthserv.tld
> AuthservIDWithJobID yes
> IgnoreHosts /etc/postfix/opendmarc-ignorehosts.txt
> Socket inet:8893@localhost
> HistoryFile /var/tmp/opendmarc.log
> RecordAllMessages True
> # ignore any external spf results
> SPFIgnoreResults True
> # use internal spf checker
> SPFSelfValidate True
> 
> and the matching /etc/opendkim.conf:
> 
> Syslog yes
> SyslogSuccess yes
> UMask 0002
> Canonicalization relaxed/relaxed
> OversignHeaders From
> InternalHosts /etc/postfix/opendmarc-ignorehosts.txt
> Domain mydomain1.tld,mydomain2.tld,mydomain3.tld
> KeyFile /etc/mail/dkim.key
> Selector mail
> Statistics /tmp/dkim-stats
> AuthservID myauthserv.tld
> AlwaysAddARHeader yes
> 
> I used postfix-policyd-spf-python until recently and these were my
> settings in /etc/postfix-policyd-spf-python/policyd-spf.conf:
> defaultSeedOnly = 1
> HELO_reject = False
> Mail_From_reject = False
> skip_addresses = 127.0.0.0/8,:::127.0.0.0/104,::1
> # whitelist allows lan clients
> Whitelist = 192.168.100.0/23
> # not sure if Header_Type needs to be AR but it makes headers more
> consistent (default is SPF)
> Header_Type = AR
> # authserv_id must match the setting in opendmarc.conf
> Authserv_Id = myauthserv.tld

I'm at a loss on the HELO_reject = False. Why wouldn't you use the
default "fail". 

Reading some chatter on the opendmarc forum, they suggested 
HEADER_Type = AR as you have done, but I don't understand the
Authserv_Id field. 

I don't think 

Re: python-policyd-spf doesn't check mail from my own domain

[Dominic Raferd]

On 31 January 2018 at 03:44, li...@lazygranch.com  wrote:
> On Tue, 30 Jan 2018 10:50:18 +
> Dominic Raferd  wrote:
>
>> On 30 January 2018 at 10:11, li...@lazygranch.com
>>  wrote:
>> > I've installed the opendmarc milter. I'm not rejecting mail from it
>> > at the moment. I've noticed that if I send myself a message, the
>> > policyd-spf milter isn't run. That in turn causes mail I send
>> > myself to fail in opendmarc. Any ideas?
>> >
>> > The various email verifiers do show that my email passes spf.
>> >
>> > It is easy enough just to whitelist your own domains from opendmarc,
>> > but that would allow spoofed email to get through.
>>
>> Which version of opendmarc? (opendmarc -V) If you have 1.3.2+ you can
>> use opendmarc's own spf instead (SPFSelfValidate True) - not reliable
>> for earlier versions though.
>>
>> Anyway, in general:
>>
>> /etc/opendmarc.conf:
>> ...
>> IgnoreAuthenticatedClients true
>> IgnoreHosts /etc/postfix/opendmarc-ignorehosts.txt
>> ...
>>
>> /etc/opendkim.conf:
>> ...
>> InternalHosts /etc/postfix/opendmarc-ignorehosts.txt
>> ...
>>
>> /etc/postfix/opendmarc-ignorehosts.txt
>> # emails from localhost are not authenticated but should be signed by
>> opendkim and not tested by opendmarc
>> 127.0.0.1
>> # similarly any ips from which we accept unauthenticated originating
>> emails (e.g. lan, or none)
>
>
> opendmarc: OpenDMARC Filter v1.3.2
> SMFI_VERSION 0x101
> libmilter version 1.0.1
> Active code options:
> WITH_SPF
> WITH_SPF2
>
> I suppose it is dumb to check spf if authenticated, but then again dkim
> is checked.
>
> I will work on the bypasses as suggested. I kind of like the
> python-policyd-spf since...well...it is working. (Something that works
> is something I don't like to change.)
>
> Still I wonder what part of the email food chain determines that spf
> wasn't needed. I commented out the local reference in
> pythod-policyd-spf, but that didn't change anything.
>
> Lots of spam gets marked as fail in opendmarc. I can't wait to start
> "trusting" it.

It shouldn't be a problem to continue using python-policyd-spf. You
would expect it to give a fail when testing mail from authenticated
clients. Opendkim needs to run in such cases not to test them but to
add the dkim header.

I use opendmarc (obvs) but I have to say I don't see it blocking many
emails. Looking at my records over a few months: 38000 mails came
through of which 50 were rejected by opendmarc and 30 quarantined. Of
those 80, 34 appear to have come via mailing lists (including
postfix.org) so may just reflect senders using the mailing list but
with incompatible dmarc settings on their domain. The reality is that
comparatively few domains are set up with dmarc and with p=reject (or
p=quarantine). If you see a large number of opendmarc fails (in
opendmarc log: action!=2) then I fear there is something wrong with
your setup.

Here is my entire opendmarc.conf:

PidFile /var/run/opendmarc/opendmarc.pid
RejectFailures true
Syslog true
UMask 0002
UserID opendmarc:opendmarc
PublicSuffixList /usr/share/publicsuffix/public_suffix_list.dat
IgnoreAuthenticatedClients true
AuthservID  myauthserv.tld
AuthservIDWithJobID yes
IgnoreHosts /etc/postfix/opendmarc-ignorehosts.txt
Socket inet:8893@localhost
HistoryFile /var/tmp/opendmarc.log
RecordAllMessages True
# ignore any external spf results
SPFIgnoreResults True
# use internal spf checker
SPFSelfValidate True

and the matching /etc/opendkim.conf:

Syslog yes
SyslogSuccess yes
UMask 0002
Canonicalization relaxed/relaxed
OversignHeaders From
InternalHosts /etc/postfix/opendmarc-ignorehosts.txt
Domain mydomain1.tld,mydomain2.tld,mydomain3.tld
KeyFile /etc/mail/dkim.key
Selector mail
Statistics /tmp/dkim-stats
AuthservID myauthserv.tld
AlwaysAddARHeader yes

I used postfix-policyd-spf-python until recently and these were my
settings in /etc/postfix-policyd-spf-python/policyd-spf.conf:
defaultSeedOnly = 1
HELO_reject = False
Mail_From_reject = False
skip_addresses = 127.0.0.0/8,:::127.0.0.0/104,::1
# whitelist allows lan clients
Whitelist = 192.168.100.0/23
# not sure if Header_Type needs to be AR but it makes headers more
consistent (default is SPF)
Header_Type = AR
# authserv_id must match the setting in opendmarc.conf
Authserv_Id = myauthserv.tld

Re: python-policyd-spf doesn't check mail from my own domain

[li...@lazygranch.com]

On Tue, 30 Jan 2018 10:50:18 +
Dominic Raferd  wrote:

> On 30 January 2018 at 10:11, li...@lazygranch.com
>  wrote:
> > I've installed the opendmarc milter. I'm not rejecting mail from it
> > at the moment. I've noticed that if I send myself a message, the
> > policyd-spf milter isn't run. That in turn causes mail I send
> > myself to fail in opendmarc. Any ideas?
> >
> > The various email verifiers do show that my email passes spf.
> >
> > It is easy enough just to whitelist your own domains from opendmarc,
> > but that would allow spoofed email to get through.  
> 
> Which version of opendmarc? (opendmarc -V) If you have 1.3.2+ you can
> use opendmarc's own spf instead (SPFSelfValidate True) - not reliable
> for earlier versions though.
> 
> Anyway, in general:
> 
> /etc/opendmarc.conf:
> ...
> IgnoreAuthenticatedClients true
> IgnoreHosts /etc/postfix/opendmarc-ignorehosts.txt
> ...
> 
> /etc/opendkim.conf:
> ...
> InternalHosts /etc/postfix/opendmarc-ignorehosts.txt
> ...
> 
> /etc/postfix/opendmarc-ignorehosts.txt
> # emails from localhost are not authenticated but should be signed by
> opendkim and not tested by opendmarc
> 127.0.0.1
> # similarly any ips from which we accept unauthenticated originating
> emails (e.g. lan, or none)


opendmarc: OpenDMARC Filter v1.3.2
SMFI_VERSION 0x101
libmilter version 1.0.1
Active code options:
WITH_SPF
WITH_SPF2

I suppose it is dumb to check spf if authenticated, but then again dkim
is checked. 

I will work on the bypasses as suggested. I kind of like the
python-policyd-spf since...well...it is working. (Something that works
is something I don't like to change.)

Still I wonder what part of the email food chain determines that spf
wasn't needed. I commented out the local reference in
pythod-policyd-spf, but that didn't change anything.

Lots of spam gets marked as fail in opendmarc. I can't wait to start
"trusting" it. 

Re: python-policyd-spf doesn't check mail from my own domain

[Dominic Raferd]

On 30 January 2018 at 10:11, li...@lazygranch.com  wrote:
> I've installed the opendmarc milter. I'm not rejecting mail from it at
> the moment. I've noticed that if I send myself a message, the
> policyd-spf milter isn't run. That in turn causes mail I send myself to
> fail in opendmarc. Any ideas?
>
> The various email verifiers do show that my email passes spf.
>
> It is easy enough just to whitelist your own domains from opendmarc,
> but that would allow spoofed email to get through.

Which version of opendmarc? (opendmarc -V) If you have 1.3.2+ you can
use opendmarc's own spf instead (SPFSelfValidate True) - not reliable
for earlier versions though.

Anyway, in general:

/etc/opendmarc.conf:
...
IgnoreAuthenticatedClients true
IgnoreHosts /etc/postfix/opendmarc-ignorehosts.txt
...

/etc/opendkim.conf:
...
InternalHosts /etc/postfix/opendmarc-ignorehosts.txt
...

/etc/postfix/opendmarc-ignorehosts.txt
# emails from localhost are not authenticated but should be signed by
opendkim and not tested by opendmarc
127.0.0.1
# similarly any ips from which we accept unauthenticated originating
emails (e.g. lan, or none)

Re: python-policyd-spf doesn't check mail from my own domain

[Benny Pedersen]

li...@lazygranch.com skrev den 2018-01-30 11:11:


It is easy enough just to whitelist your own domains from opendmarc,
but that would allow spoofed email to get through.


its simple to not accept forged senders in port 25 ?

for me i just check virtual alias senders in postfixadmin, poor manns 
spf checker cant fail here


in port 465 / 587 do not accept useers that does not sasl auth, problem 
solved


in opendmarc skip sasl auth users

in policyd-spf skip localhost ip, and maybe aswell mynetworks in postfix