Re: Incoming SMTP in the year 2017 and absence of DKIM (fwd)

[John R. Levine]

In article <6134b4a7-9da8-2935-e9f6-e4374b3fd...@spamtrap.tnetconsulting.net>,
Grant Taylor via NANOG   wrote:

https://datatracker.ietf.org/doc/draft-levine-dkim-conditional/



The only way that I can think of is for the originating mail server to
DKIM sign the message twice, 1st with the classic DKIM-Signature w/o the
!fs tag, and 2nd with a DKIM-Signature that includes the !fs tag with a
value of of the recipient's domain.



Is this what you were intending?  A list of DKIM-Signatures linked via
!fs tags?


Yup, with the chain typically having no more than one or two links,
since legit forwarding of the kind that might break DKIM is pretty
rare more than two deep.


If I do understand correctly, I think that it's intriguing.  I'm not
aware of anything else that would work quite the same way.


That was the plan.  I thought it was pretty clever, but like I said, the 
large mail systems that developed ARC wanted to put the control with the 
recipients, not the senders.


R's,
John

Re: Incoming SMTP in the year 2017 and absence of DKIM

[Grant Taylor via NANOG]

On 11/30/2017 07:38 PM, John R. Levine wrote:
I did a draft of a double signing thing that let the sender say who's 
expected to sign a modified forwarded version.  The big mail systems 
weren't interested.  They want the recipient system to decide.


https://datatracker.ietf.org/doc/draft-levine-dkim-conditional/


Okay, I've now read your draft and have some questions.

How would the !fs tag enable multiple forwarders?

The only way that I can think of is for the originating mail server to 
DKIM sign the message twice, 1st with the classic DKIM-Signature w/o the 
!fs tag, and 2nd with a DKIM-Signature that includes the !fs tag with a 
value of of the recipient's domain.


I would assume that would mean that the recipient could then forward the 
message to a new recipient and that their outgoing mail server would 
also sign twice, 1st with classic DKIM-Signature w/o the !fs tag, and 
2nd with a DKIM-Signature that includes the !fs tag with a value of the 
new recipient's domain.


A1:  DKIM-Signature: ... d=domainA.example ...
A2:  DKIM-Signature: ... d=domainA.example; !fs=domainB.example ...
<1st forward>
B1:  DKIM-Signature: ... d=domainB.example ...
B2:  DKIM-Signature: ... d=domainB.example; !fs=domainC.example ...
<2nd forward>
C1:  DKIM-Signature: ... d=domainC.example ...
C2:  DKIM-Signature: ... d=domainC.example; !fs=domainD.example ...
<3rd forward>
D1:  DKIM-Signature: ... d=domainD.example ...
D2:  DKIM-Signature: ... d=domainD.example; !fs=domainE.example ...
<4th forward>
E1:  DKIM-Signature: ... d=domainE.example ...
E2:  DKIM-Signature: ... d=domainE.example; !fs=domainF.example ...

(I suppose that this pattern could go on forever.)

Is this what you were intending?  A list of DKIM-Signatures linked via 
!fs tags?


If I do understand correctly, I think that it's intriguing.  I'm not 
aware of anything else that would work quite the same way.




--
Grant. . . .
unix || die



smime.p7s
Description: S/MIME Cryptographic Signature

Re: Incoming SMTP in the year 2017 and absence of DKIM

[John R. Levine]

 Yeah, that's what ARC is intended to do.


Hum.  My understanding of ARC is that it's a way for a server to assert 
things about what it received.  -  Where as my interpretation of what we were 
discussing is the sender authorizing intermediary MTAs to send the message. 
The former is after the fact, and the latter is before hand.


I did a draft of a double signing thing that let the sender say who's 
expected to sign a modified forwarded version.  The big mail systems 
weren't interested.  They want the recipient system to decide.


https://datatracker.ietf.org/doc/draft-levine-dkim-conditional/

Regards,
John Levine, jo...@iecc.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

Re: Incoming SMTP in the year 2017 and absence of DKIM

[Grant Taylor via NANOG]

On 11/30/2017 06:47 PM, John Levine wrote:
I suppose that would make sense for the 0.1% of mailing lists run by 
people with the skill and interest to hack on their list software.


I guess I'm in the 0.1% then.


ATPS was an experiment that failed.  Nobody uses it, it didn't scale.


That's sort of what I've gathered.

I can't help but note the absence of S/MIME signatures on roughly 100% 
of all of the messages in this thread.


I believe that's because the mailing list strips non-text MIME parts, 
including the S/MIME signatures.



Yeah, that's what ARC is intended to do.


Hum.  My understanding of ARC is that it's a way for a server to assert 
things about what it received.  -  Where as my interpretation of what we 
were discussing is the sender authorizing intermediary MTAs to send the 
message.  The former is after the fact, and the latter is before hand.




--
Grant. . . .
unix || die



smime.p7s
Description: S/MIME Cryptographic Signature

Re: Incoming SMTP in the year 2017 and absence of DKIM

[John Levine]

In article <3d84c686-aa5f-8180-8a37-be77fef94...@tnetconsulting.net> you write:
>I would also configure MLMs to forward unknown bounces to the -owner. 
>Hopefully the -owner would then feed (a sanitized copy of) the unknown 
>bounce type the MLM maintainer(s) to improve said MLM.

I suppose that would make sense for the 0.1% of mailing lists run by
people with the skill and interest to hack on their list software.

>> It's a rathole, it doesn't scale, and it is not a bug that you can 
>> send mail to people who you don't already know.
>
>I wasn't aware that DKIM-ATPS necessitated needing to know who you were 
>going to send to.

ATPS was an experiment that failed.  Nobody uses it, it didn't scale.

>> If identities were a magic bullet, we'd all be signing with S/MIME.
>
>I am (and have been for years) a proponent of S/MIME.

I can't help but note the absence of S/MIME signatures on roughly 100%
of all of the messages in this thread.

>(I think we're still talking about how can an intermediate mail server 
>be authorized to be part of the SMTP end-to-end mail delivery chain. 
>Even if said intermediate mail server is downstream of the sender.)

Yeah, that's what ARC is intended to do.

R's,
John

Re: Incoming SMTP in the year 2017 and absence of DKIM

[bzs]

I'd love to hear, not here particularly, from someone very
knowledgeable about the history of postal fraud and abuse.

I suspect there are more than a few parallels and we'd find out how
much of our efforts amount to reinventing wheels once one peels away
the technical abstractions and jargon. Basically authentication for
starters.

(And if someone is about to explain the difference between paper and
electronic mail, per piece cost and all that, please spare us.)

-- 
-Barry Shein

Software Tool & Die| b...@theworld.com | http://www.TheWorld.com
Purveyors to the Trade | Voice: +1 617-STD-WRLD   | 800-THE-WRLD
The World: Since 1989  | A Public Information Utility | *oo*

Re: Incoming SMTP in the year 2017 and absence of DKIM

[John R. Levine]

It's a one way correlation.  If the rDNS is busted, you can be pretty
sure you don't want the mail.  If the rDNS is OK, you need more clues.


Pretty sure, but far from certain.

Even this one-way correlation is rather tenuous. It’s mostly harmless because
everyone knows that mail servers are filtering on this basis and legitimate
senders therefore force themselves into workarounds.


Having talked to a lot of people who run large mail systems, it's much 
simpler than that.  If you want people to accept your mail, you better 
have your DNS under control.  If it's not important enough to you to make 
your DNS work, it's not important enough to me to look at what you might 
try to send.



Fortunately for everyone’s sake, Bj0rn, while he may not like it, seems to find
a way to send his email via some mechanism that allows me to receive it from
a  host that has working rDNS.


Yeah, funny about that.


Spamassassin is as good an example as any and while it can be effective if 
you’ve
got the cycles to keep it constantly updated and fed with new information and…,
it’s a rather large PITA for a small site with an admin that needs to count on
most things running on autopilot most of the time in order to survive.


That would be me, a daily cron job to install updates does the trick. 
It's not perfect but it's good enough.



People who want to be malicious are usually less willing to do so if they know 
that
they will be identified, so actually, it does help.

i.e. rarely to bank robbers sign their names to the robbery note.


Of course not.  What it means is that now they attack the authentication 
systems.  They do so in many ways, from stealing grandma's credentials on 
botted computers to buying SIMs in bulk to defeat schemes that want to tie 
a unique phone number to each account.


Regards,
John Levine, jo...@iecc.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

Re: Incoming SMTP in the year 2017 and absence of DKIM

[Grant Taylor via NANOG]

On 11/30/2017 12:16 PM, Owen DeLong wrote:
it’s a rather large PITA for a small site with an admin that needs to count on 
most things running on autopilot most of the time in order to survive.


I have to disagree with that.

I've been running SpamAssassin for > 15 years and have found it to be 
mostly trouble free.  -  I have cron jobs update it's files and rely on 
milters to accept / tag / reject messages.  -  I spend very little time 
caring for / feeding SpamAssassin.  Probably < 5 minutes a month.)


Sure, I occasionally fiddle with things, but that's because I want to, 
not because I need to.


So, while it might be a higher-quality solution, I’d argue that it’s not completely 
“better” in that any autopilotable configuration of it involves a high degree of 
false negatives or an unacceptable level of false positives.


I've had fairly good luck with autopilot.  I also don't see many false 
negatives.  Nor do people report false positives to me.  (Granted, I tag 
at 5 and reject at 15.)



People who want to be malicious are usually less willing to do so if they know 
that
they will be identified, so actually, it does help.


Agreed.



--
Grant. . . .
unix || die



smime.p7s
Description: S/MIME Cryptographic Signature

Re: Incoming SMTP in the year 2017 and absence of DKIM

[Owen DeLong]

> On Nov 30, 2017, at 12:11 , valdis.kletni...@vt.edu wrote:
> 
> On Thu, 30 Nov 2017 11:16:09 -0800, Owen DeLong said:
>> i.e. rarely to bank robbers sign their names to the robbery note.
> 
> An amazing number of them use a deposit slip with their name on it for the 
> note.

I’m guessing that the ones that do so only do so once.

Owen

RE: Incoming SMTP in the year 2017 and absence of DKIM

[Keith Medcalf]

On Thursday, 30 November, 2017 10:55, Bjørn Mork , wrote:

>Steve Atkins  writes:

>>> On Nov 30, 2017, at 1:22 AM, Bjørn Mork  wrote:

>>> "John Levine"  writes:

>> It tells you something about the competence of the operator and
>> whether the host is intended by the owners to send email.

>No.  It only tells you something about the administrative split
>between IP address management and host management.

>There is no way my laptop is going to be able to update the rDNS for
>all addresses it will use in different networks.  This does in no way
>affect its MTA configuration.

Your Laptop should not be an MTA.  Perhaps it is a authenticated submission 
agent sending to MTA, but without properly configured forward/reverse DNS it is 
not an MTA.  Many systems will not accept SMTP from it unless it can 
authenticate.

>> Or, for a more empirical way to look at it, there's reasonable
>> correlation between having missing, generic or incorrect reverse
>> DNS and the host being a source of unwanted or malicious email.

>Really?  Where did you get those numbers?  This is a myth.  Spam
>sources are average Internet hosts.  The split between working and non-
>working rDNS is mostly between IPv4 and IPv6, not between ham and spam.

You are incorrect.  If DNS is not configured correctly then the spam to ham 
ratio is pretty much 100% spam with no ham.

>And if there is some correlation there, then I'd say that an IPv4 host is
>more likely to be a spam source than a dual stack or IPv6 only host.

Actually, you are incorrect again.  In order of "Spaminess" (most spammy first) 
you have the following order:

IPv4 with incorrectly configured DNS.
IPv6 without regard for DNS configuration.
IPv4 with correctly configured DNS.

Re: Incoming SMTP in the year 2017 and absence of DKIM

[Grant Taylor via NANOG]

On 11/30/2017 11:30 AM, John Levine wrote:
If you look at the bounce handling in packages like sympa and mailman, 
they have lots of heuristics to try to figure out what bounces mean. 
 They work OK but I agree they are far from perfect.


I never have.  Further, I think I'd like to not go insane.

I naively would expect that one would look for the most common bounce 
format (likely standard DSN), then the next most common, ... rinse, 
lather, repeat.


I'd bet that between three and eight formats in, you would have a VERY 
LARGE portion of bounces covered.


I would also configure MLMs to forward unknown bounces to the -owner. 
Hopefully the -owner would then feed (a sanitized copy of) the unknown 
bounce type the MLM maintainer(s) to improve said MLM.


It's a rathole, it doesn't scale, and it is not a bug that you can 
send mail to people who you don't already know.


I wasn't aware that DKIM-ATPS necessitated needing to know who you were 
going to send to.


I thought DKIM-ATPS was meant to allow a 3rd party that you contract to 
be an "Authorized Third (party) Sender" of email for your domain.


Though, that doesn't do anything for recipients forwarding to their new 
mailbox.



If identities were a magic bullet, we'd all be signing with S/MIME.


I am (and have been for years) a proponent of S/MIME.  Though I don't 
think that it really does anything to help with this paradigm.  Unless 
you are able to filter incoming messages with the intention that all 
incoming messages MUST be signed and reject (or otherwise filter) 
unsigned messages.


(I think we're still talking about how can an intermediate mail server 
be authorized to be part of the SMTP end-to-end mail delivery chain. 
Even if said intermediate mail server is downstream of the sender.)




--
Grant. . . .
unix || die



smime.p7s
Description: S/MIME Cryptographic Signature

Re: Incoming SMTP in the year 2017 and absence of DKIM

[valdis . kletnieks]

On Thu, 30 Nov 2017 11:16:09 -0800, Owen DeLong said:
> i.e. rarely to bank robbers sign their names to the robbery note.

An amazing number of them use a deposit slip with their name on it for the note.


pgpLt6XbYQz1w.pgp
Description: PGP signature

Re: Incoming SMTP in the year 2017 and absence of DKIM

[Rich Kulawiec]

On Thu, Nov 30, 2017 at 10:22:40AM +0100, Bj??rn Mork wrote:
> rDNS is not a host attribute, and will therefore tell you exactly
> nothing about the host.

The lack of rDNS disqualifies a system from being a legitimate mail host.
The lack of FCrDNS does the same.  (Note that it's usually prudent to
tempfail some of these cases in order to allow for the circumstance that
something is temporarily wonky with DNS.  Well-run mail services that
are experiencing transient issues will correct those, DNS will once
again be working, and queued mail will eventually make it through.)

The content of rDNS provides additional information, and some of
it's enormously useful: e.g., blah.dynamic.example.com is not a valid
mailhost, and immediate rejection is highly advisable.  Same for
blah.dsl.example.com and blah.unassigned.example.com and many other
patterns.  And of course depending on the expected mix of spam/nonspam
traffic at a particular mail server, there may be no need to accept any
mail traffic from blah.example.TLD for many values of "TLD". [1]

I just checked on a particular mail server that I'm working on, and in
November 2017, 62% of all messages that were rejected were disposed of
thusly because they failed rDNS/DNS-related checks.  (Which includes things
like the above as well as checking HELO, MX validity, etc.)  That means
that roughly 2/3 of the messages didn't need to be checked against a
DNSBL or anything else, reducing the load on valuable shared resources.

rDNS/DNS checks are an efficient, reliable, scalable, first-line MTA
defense -- and they're quite robust in the face of attempts to game them.

---rsk

[1] Or, alternatively, to only accept it at certain MX's designated
for the task -- ones which presumably apply higher scrutiny to such
traffic than would otherwise be employed.  This works for various
geoblocking tactics as well.

Re: Incoming SMTP in the year 2017 and absence of DKIM

[Owen DeLong]

> On Nov 30, 2017, at 10:28 , John Levine  wrote:
> 
> In article  you write:
>>> Or, for a more empirical way to look at it, there's reasonable correlation
>>> between having missing, generic or incorrect reverse DNS and the host
>>> being a source of unwanted or malicious email.
>> 
>> I’m not so sure about that.
> 
> It's a one way correlation.  If the rDNS is busted, you can be pretty
> sure you don't want the mail.  If the rDNS is OK, you need more clues.

Pretty sure, but far from certain.

Even this one-way correlation is rather tenuous. It’s mostly harmless because
everyone knows that mail servers are filtering on this basis and legitimate
senders therefore force themselves into workarounds.

In an ideal world, I wouldn’t mind accepting email from Bj0rn’s laptop directly,
but today, the price of doing so in SPAM is just too high, so I don’t.

Fortunately for everyone’s sake, Bj0rn, while he may not like it, seems to find
a way to send his email via some mechanism that allows me to receive it from
a  host that has working rDNS.

>> Unfortunately, until we get widespread deployment of something better than 
>> IP reputation based
>> systems, ...
> 
> You might take a look at how current spam filters work.  Spamassassin
> is as good an example as any.  It does dynamic weigthted scoring of a
> lot of factors, of which IP reputation is only one.  I find that I can
> use conservatively run IP blacklists as a cheap prepass to avoid
> sending the mail to spamassassin at all, but there's a lot more than
> IP by the time the mail does or does not get delivered.  DKIM is
> useful if have opinions about the reputations of the signing domains,
> not purely by whether there's a signature.

Spamassassin is as good an example as any and while it can be effective if 
you’ve
got the cycles to keep it constantly updated and fed with new information and…,
it’s a rather large PITA for a small site with an admin that needs to count on
most things running on autopilot most of the time in order to survive.

So, while it might be a higher-quality solution, I’d argue that it’s not 
completely
“better” in that any autopilotable configuration of it involves a high degree of
false negatives or an unacceptable level of false positives.

>> Perhaps this is simply the inherent cost of maintaining an open 
>> communications infrastructure with
>> a low barrier to entry and the potential for anonymous communications which 
>> I believe has value
>> to society and should be preserved. Perhaps someone smarter than I will some 
>> day develop a better
>> solution.
> 
> It seems to be an axiom that any community large enough to be
> interesting is large enough to contain people who are malicious, so
> even requiring that people be identified won't help.

People who want to be malicious are usually less willing to do so if they know 
that
they will be identified, so actually, it does help.

i.e. rarely to bank robbers sign their names to the robbery note.

Owen

Re: Incoming SMTP in the year 2017 and absence of DKIM

[John Levine]

In article  you write:
>> Without something like VERP to encode the original recipient in the return 
>> address, the percentage of bounces your list successfully processes each 
>> month will slowly but steadily decline.
>
>I think it's entirely possible to teach MLMs about the most common forms 
>of bounces (DSNs).  But it will quickly get into a game of diminishing 
>returns.  Especially if the bounce (because it's not going to be the 
>well known DNS format) goes out of it's way to hide something.  In that 
>case, the only thing that you could count on (that I'm aware of) is 
>something like VERP.

If you look at the bounce handling in packages like sympa and mailman,
they have lots of heuristics to try to figure out what bounces mean.
They work OK but I agree they are far from perfect.

>  -  I think that SPF and DKIM-ATPS can (at least partially) address the 
>latter.  With the latter assuming some sort of established business 
>relationship between the originating and intermediary parties.

It's a rathole, it doesn't scale, and it is not a bug that you can
send mail to people who you don't already know.  If identities were a
magic bullet, we'd all be signing with S/MIME.

R's,
John

Re: Incoming SMTP in the year 2017 and absence of DKIM

[John Levine]

In article  you write:
>> Or, for a more empirical way to look at it, there's reasonable correlation
>> between having missing, generic or incorrect reverse DNS and the host
>> being a source of unwanted or malicious email.
>
>I’m not so sure about that.

It's a one way correlation.  If the rDNS is busted, you can be pretty
sure you don't want the mail.  If the rDNS is OK, you need more clues.

>Unfortunately, until we get widespread deployment of something better than IP 
>reputation based
>systems, ...

You might take a look at how current spam filters work.  Spamassassin
is as good an example as any.  It does dynamic weigthted scoring of a
lot of factors, of which IP reputation is only one.  I find that I can
use conservatively run IP blacklists as a cheap prepass to avoid
sending the mail to spamassassin at all, but there's a lot more than
IP by the time the mail does or does not get delivered.  DKIM is
useful if have opinions about the reputations of the signing domains,
not purely by whether there's a signature.

>Perhaps this is simply the inherent cost of maintaining an open communications 
>infrastructure with
>a low barrier to entry and the potential for anonymous communications which I 
>believe has value
>to society and should be preserved. Perhaps someone smarter than I will some 
>day develop a better
>solution.

It seems to be an axiom that any community large enough to be
interesting is large enough to contain people who are malicious, so
even requiring that people be identified won't help.

R's,
John

Re: Incoming SMTP in the year 2017 and absence of DKIM

[Grant Taylor via NANOG]

On 11/30/2017 01:53 AM, Benoit Panizzon wrote:
DKIM is not widely used and DKIM does break a lot of mailinglists and 
sometimes also SRS compliant forwarding.


How does DKIM break SRS compliant forwarding?  (Assuming that only the 
message envelope is modified.)


Or are you referring to DMARC's interactions there in?



--
Grant. . . .
unix || die



smime.p7s
Description: S/MIME Cryptographic Signature

Re: Incoming SMTP in the year 2017 and absence of DKIM

[Owen DeLong]

> On Nov 30, 2017, at 09:55 , Bjørn Mork  wrote:
> 
> Steve Atkins  writes:
> 
>>> On Nov 30, 2017, at 1:22 AM, Bjørn Mork  wrote:
>>> 
>>> "John Levine"  writes:
>>> 
 Broken rDNS is just broken, since there's approximately no reason ever
 to send from a host that doesn't know its own name.
>>> 
>>> rDNS is not a host attribute, and will therefore tell you exactly
>>> nothing about the host.
>> 
>> It tells you something about the competence of the operator and
>> whether the host is intended by the owners to send email.
> 
> No.  It only tells you something about the administrative split between
> IP address management and host management.
> 
> There is no way my laptop is going to be able to update the rDNS for all
> addresses it will use in different networks.  This does in no way affect
> its MTA configuration.

Perhaps a better way to word it is “It tells us something about whether the
machine is likely to possess properties which make it generally undesirable
for us to accept messages from it directly.”

I, for one, have no interest in accepting messages into my mail server directly
from your laptop, even if they are legitimately from you to me. I’m perfectly
happy to insist that you go via an MTA hosted in a more permanent location on
your side first in order to avoid receiving messages directly from the much
larger quantity of incompetently administered mailservers, many of which I 
suspect
are not intended by their owners (distinct from their pwn3rs) to be mail servers
at all.

>> Or, for a more empirical way to look at it, there's reasonable correlation
>> between having missing, generic or incorrect reverse DNS and the host
>> being a source of unwanted or malicious email.
> 
> Really?  Where did you get those numbers?  This is a myth.  Spam sources
> are average Internet hosts.  The split between working and non-working
> rDNS is mostly between IPv4 and IPv6, not between ham and spam.  And if
> there is some correlation there, then I'd say that an IPv4 host is more
> likely to be a spam source than a dual stack or IPv6 only host.

Really? Most of my hosts have working rDNS for both v4 and v6.

As to an IPv4 host being a more likely source of SPAM, I’m not convinced about
that, either given the amount of SPAM that hits my mailserver via IPv6.

Owen

Re: Incoming SMTP in the year 2017 and absence of DKIM

[Owen DeLong]

> On Nov 30, 2017, at 09:03 , Steve Atkins  wrote:
> 
> 
>> On Nov 30, 2017, at 1:22 AM, Bjørn Mork  wrote:
>> 
>> "John Levine"  writes:
>> 
>>> Broken rDNS is just broken, since there's approximately no reason ever
>>> to send from a host that doesn't know its own name.
>> 
>> rDNS is not a host attribute, and will therefore tell you exactly
>> nothing about the host.
> 
> It tells you something about the competence of the operator and
> whether the host is intended by the owners to send email.
> 
> Or, for a more empirical way to look at it, there's reasonable correlation
> between having missing, generic or incorrect reverse DNS and the host
> being a source of unwanted or malicious email.

I’m not so sure about that.

Lots of hosts that send unwanted/malicious email have missing, generic, or 
obviously incorrect rDNS.
Lots of hosts that send unwanted/malicious email have valid non-generic 
possibly correct rDNS.

I don’t accept email from the former, but I still get plenty of SPAM from the 
latter.

Unfortunately, until we get widespread deployment of something better than IP 
reputation based
systems, SPAM continues to be a low-cost to the sender side with a high burden 
on the delivery side
and therefore remains a very profitable industry.

DKIM certainly could help (though I’m not convinced it’s a 100% effective 
solution, nor am I
particularly convinced we’ve found any particularly effective solutions as yet.

Perhaps this is simply the inherent cost of maintaining an open communications 
infrastructure with
a low barrier to entry and the potential for anonymous communications which I 
believe has value
to society and should be preserved. Perhaps someone smarter than I will some 
day develop a better
solution.

Owen

Re: Incoming SMTP in the year 2017 and absence of DKIM

[Bjørn Mork]

Steve Atkins  writes:

>> On Nov 30, 2017, at 1:22 AM, Bjørn Mork  wrote:
>> 
>> "John Levine"  writes:
>> 
>>> Broken rDNS is just broken, since there's approximately no reason ever
>>> to send from a host that doesn't know its own name.
>> 
>> rDNS is not a host attribute, and will therefore tell you exactly
>> nothing about the host.
>
> It tells you something about the competence of the operator and
> whether the host is intended by the owners to send email.

No.  It only tells you something about the administrative split between
IP address management and host management.

There is no way my laptop is going to be able to update the rDNS for all
addresses it will use in different networks.  This does in no way affect
its MTA configuration.

> Or, for a more empirical way to look at it, there's reasonable correlation
> between having missing, generic or incorrect reverse DNS and the host
> being a source of unwanted or malicious email.

Really?  Where did you get those numbers?  This is a myth.  Spam sources
are average Internet hosts.  The split between working and non-working
rDNS is mostly between IPv4 and IPv6, not between ham and spam.  And if
there is some correlation there, then I'd say that an IPv4 host is more
likely to be a spam source than a dual stack or IPv6 only host.



Bjørn

Re: Incoming SMTP in the year 2017 and absence of DKIM

[Steve Atkins]

> On Nov 30, 2017, at 1:22 AM, Bjørn Mork  wrote:
> 
> "John Levine"  writes:
> 
>> Broken rDNS is just broken, since there's approximately no reason ever
>> to send from a host that doesn't know its own name.
> 
> rDNS is not a host attribute, and will therefore tell you exactly
> nothing about the host.

It tells you something about the competence of the operator and
whether the host is intended by the owners to send email.

Or, for a more empirical way to look at it, there's reasonable correlation
between having missing, generic or incorrect reverse DNS and the host
being a source of unwanted or malicious email.

Cheers,
  Steve

Re: Incoming SMTP in the year 2017 and absence of DKIM

[Bjørn Mork]

"John Levine"  writes:

> Broken rDNS is just broken, since there's approximately no reason ever
> to send from a host that doesn't know its own name.

rDNS is not a host attribute, and will therefore tell you exactly
nothing about the host.


Bjørn

Re: Incoming SMTP in the year 2017 and absence of DKIM

[Benoit Panizzon]

Hi

> For those who operate public facing SMTPd that receive a large volume
> of incoming traffic, and accordingly, a lot of spam...
> 
> How much weight do you put on an incoming message, in terms of adding
> additional score towards a possible value of spam, for total absence
> of DKIM signature?

No DKIM = not scored.

DKIM is not widely used and DKIM does break a lot of mailinglists and
sometimes also SRS compliant forwarding.

We do score some points if a DKIM header with invalid signature is
present.

-Benoît Panizzon-
-- 
I m p r o W a r e   A G-Leiter Commerce Kunden
__

Zurlindenstrasse 29 Tel  +41 61 826 93 00
CH-4133 PrattelnFax  +41 61 826 93 01
Schweiz Web  http://www.imp.ch
__

Re: Incoming SMTP in the year 2017 and absence of DKIM

[Grant Taylor via NANOG]

On 11/29/2017 07:16 PM, William Herrin wrote:
There's no "must" standard for the format of bounce message, deferred 
bounces are still a thing and mail gets auto-forwarded to addresses which 
bounce (that is, bounce from an address different than the one you sent to).


Agreed.  I wish that more software would use the well defined Delivery 
Status Notification and Message Disposition Notifications.  (RFC 6553)


Without something like VERP to encode the original recipient in the return 
address, the percentage of bounces your list successfully processes each 
month will slowly but steadily decline.


I think it's entirely possible to teach MLMs about the most common forms 
of bounces (DSNs).  But it will quickly get into a game of diminishing 
returns.  Especially if the bounce (because it's not going to be the 
well known DNS format) goes out of it's way to hide something.  In that 
case, the only thing that you could count on (that I'm aware of) is 
something like VERP.


I wonder if SMTP's ORCPT parameter to the RCPT command would cross 
forwarders.  (I'm not holding my breath.)


Aside:  I'm quite interested in discussing the the following reply, but 
I suspect it's going to be a bit of a rabbit whole.  Is such a 
discussion appropriate for NANOG?  (I'll assume that a lack of reply 
indicates that the discussion is better had elsewhere.)


I could not disagree with you more. It's relatively easy to design and 
implement a system which allows the originating MUA and MTA to offer proof 
of authority to act on behalf of a given email address. Though possible, 
systems which would allow intermediate mail handlers to generate proof of 
authority to handle a message alleged to originate from a particular 
address don't especially exist and would be much more complex. Systemic and 
computational complexity is a very practical difference between the two 
situations.


I feel like SPF & DKIM (at least partially) address the former scenario. 
 -  I think that SPF and DKIM-ATPS can (at least partially) address the 
latter.  With the latter assuming some sort of established business 
relationship between the originating and intermediary parties.




--
Grant. . . .
unix || die

Re: Incoming SMTP in the year 2017 and absence of DKIM

[William Herrin]

On Wed, Nov 29, 2017 at 5:50 PM, John Levine  wrote:
>
> In article <3677d101-3874-b8e4-87b3-37e4dd870...@tnetconsulting.net> you
write:
> >> Normal lists put their own bounce address in the
> >> envelope so they can handle the bounces, so their own SPF applies.
> >
> >Yep.  V.E.R.P. is a very powerful thing.  (B.A.T.V. is an interesting
> >alternative, but I never messed with it.)
>
> VERP helps identify the bouncing party, but list bounce handling works
> fine without it.

Not so much, no.

There's no "must" standard for the format of bounce message, deferred
bounces are still a thing and mail gets auto-forwarded to addresses which
bounce (that is, bounce from an address different than the one you sent to).

Without something like VERP to encode the original recipient in the return
address, the percentage of bounces your list successfully processes each
month will slowly but steadily decline.

Broken rDNS is just broken, since there's approximately no reason ever
> to send from a host that doesn't know its own name.  Broken SPF may or
> may not be an issue since there are lots of legit ways to send mail
> that SPF can't describe.
>

+1


>P.S.  I'm strongly of the opinion that if a MLM alters the message in
> >ANY capacity, that it is actually generating a new message.  Thus the
> >MLM is the new author.  It's just using content strongly based on emails
> >that came into it.  -  But that's a different discussion that lasted
> >days on the mailman mailing list.
>
> It's an interesting theological argument but it makes little practical
> difference.
>

I could not disagree with you more. It's relatively easy to design and
implement a system which allows the originating MUA and MTA to offer proof
of authority to act on behalf of a given email address. Though possible,
systems which would allow intermediate mail handlers to generate proof of
authority to handle a message alleged to originate from a particular
address don't especially exist and would be much more complex. Systemic and
computational complexity is a very practical difference between the two
situations.

Regards,
Bill Herrin


-- 
William Herrin  her...@dirtside.com  b...@herrin.us
Dirtside Systems . Web: 

Re: Incoming SMTP in the year 2017 and absence of DKIM

[John Levine]

In article <11e9c18dac053c4bb91b95a4993c1...@mail.dessus.com> you write:
>
>Not old enough to have had an Executive Secretary processing your incoming 
>snail-mail before it gets to you?

Probably about the same age as you, but I hope that after 50 years of
e-mail we have figured out that the parallels with paper mail are
imperfect.  The e-mail envelope is a metaphor, you know.

R's,
John

Re: Incoming SMTP in the year 2017 and absence of DKIM

[John Levine]

In article <3677d101-3874-b8e4-87b3-37e4dd870...@tnetconsulting.net> you write:
>> Normal lists put their own bounce address in the 
>> envelope so they can handle the bounces, so their own SPF applies.
>
>Yep.  V.E.R.P. is a very powerful thing.  (B.A.T.V. is an interesting 
>alternative, but I never messed with it.)

VERP helps identify the bouncing party, but list bounce handling works
fine without it.  What matters is that it's the list's address in the
envelope, not the message author.  BATV works OK (I should know, I
invented it) but it has its false positives.

>I'm saying that I've heard arguments over the last 15 years from people 
>that (FC)rDNS and SPF (independently) are things that will break some 
>portion of email.

Broken rDNS is just broken, since there's approximately no reason ever
to send from a host that doesn't know its own name.  Broken SPF may or
may not be an issue since there are lots of legit ways to send mail
that SPF can't describe.

R's,
John

>P.S.  I'm strongly of the opinion that if a MLM alters the message in 
>ANY capacity, that it is actually generating a new message.  Thus the 
>MLM is the new author.  It's just using content strongly based on emails 
>that came into it.  -  But that's a different discussion that lasted 
>days on the mailman mailing list.

It's an interesting theological argument but it makes little practical
difference.

RE: Incoming SMTP in the year 2017 and absence of DKIM

[Keith Medcalf]

Not old enough to have had an Executive Secretary processing your incoming 
snail-mail before it gets to you?

The "envelope" in which a letter arrived is just as important as the letter 
itself and contains valuable information that is duplicated in e-mail -- the 
postmark (received headers), the return address (mail from); and, the delivery 
address (mail to).

It was an offense to discard the envelope in which correspondence arrived since 
it is used to determine the validity of the snail mail.

Current e-mail clients are comparable to having a secretary that throws out the 
envelope and snips off most of the inside addressing information and delivers 
only the heavily redacted letter so that no determination of its validity is 
possible.

---
The fact that there's a Highway to Hell but only a Stairway to Heaven says a 
lot about anticipated traffic volume.


>-Original Message-
>From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of John Levine
>Sent: Wednesday, 29 November, 2017 14:28
>To: nanog@nanog.org
>Subject: Re: Incoming SMTP in the year 2017 and absence of DKIM
>
>In article <20171129183535.gb18...@ucsd.edu> you write:
>>As I see it, the problem isn't with DKIM, it's with the
>>implementation of DMARC and other such filters.  Almost all
>>of them TEST THE WRONG FROM ADDRESS.  They compare the Author's
>>address (the header From: line) instead of the Sender's address,
>
>Sigh.  I have my differences with the people who designed DMARC but
>they are not stupid and they really do understand the relevant RFCs.
>Some of them even wrote some of those RFCs.
>
>The reason they look at the From: line is that's the one recipients
>see.  The Sender: header was a nice idea but in practice, it's not
>useful.
>
>R's,
>John

Re: Incoming SMTP in the year 2017 and absence of DKIM

[Grant Taylor via NANOG]

On 11/29/2017 02:13 PM, John Levine wrote:
A mailing list sending with bad rDNS or bad SPF is a pretty cruddy 
mailing list.


s/mailing list sending/sending server/

Agreed.

Normal lists put their own bounce address in the 
envelope so they can handle the bounces, so their own SPF applies.


Yep.  V.E.R.P. is a very powerful thing.  (B.A.T.V. is an interesting 
alternative, but I never messed with it.)


No idea why you think rDNS for a list's MTA is any harder than anyone 
else's MTA.


I don't.

I'm saying that I've heard arguments over the last 15 years from people 
that (FC)rDNS and SPF (independently) are things that will break some 
portion of email.  -  I believe that these are simply technologies that 
the email industry has adopted and now considers to be Best Practice, if 
not actual requirements that MUST be done.


IMHO, Mailing List Managers are simply a different form of MUA that 
utilizes the same email infrastructure (MTAs.)  Thus, MLMs are subject 
tot he same requirements as "individual email" (as referred to earlier.)




--
Grant. . . .
unix || die

P.S.  I'm strongly of the opinion that if a MLM alters the message in 
ANY capacity, that it is actually generating a new message.  Thus the 
MLM is the new author.  It's just using content strongly based on emails 
that came into it.  -  But that's a different discussion that lasted 
days on the mailman mailing list.




smime.p7s
Description: S/MIME Cryptographic Signature

Re: Incoming SMTP in the year 2017 and absence of DKIM

[John Levine]

In article <20171129183535.gb18...@ucsd.edu> you write:
>As I see it, the problem isn't with DKIM, it's with the
>implementation of DMARC and other such filters.  Almost all
>of them TEST THE WRONG FROM ADDRESS.  They compare the Author's
>address (the header From: line) instead of the Sender's address,

Sigh.  I have my differences with the people who designed DMARC but
they are not stupid and they really do understand the relevant RFCs.
Some of them even wrote some of those RFCs.

The reason they look at the From: line is that's the one recipients
see.  The Sender: header was a nice idea but in practice, it's not
useful.

R's,
John

Re: Incoming SMTP in the year 2017 and absence of DKIM

[Grant Taylor via NANOG]

On 11/29/2017 11:35 AM, Brian Kantor wrote:

As I see it, the problem isn't with DKIM,


I don't think DKIM is (the source of) /the/ problem per say.  Rather I 
think it's a complication of other things (DMARC) that interact with DKIM.


it's with the 
implementation of DMARC and other such filters.  Almost all 
of them TEST THE WRONG FROM ADDRESS.  They compare the Author's 
address (the header From: line) instead of the Sender's address, 
(the SMTP Mail From: transaction or Sender: header line).


I believe it's more than just the implementation.  The DMARC 
specification specifically calls out the RFC 5322 From: header.


Further, RFC 7489, Appendix A, § 3 speaks directly to this.

If the filter checked the Sender address of mail instead of the 
Author address, mailing lists wouldn't be broken!


Perhaps.  However I fear we would be facing an entirely new type of spam 
that used spoofed From: headers and perfectly legitimate Sender: headers 
(that also match the RFC 5321 SMTP FROM address.)  See RFC 7489 § A.3.1




--
Grant. . . .
unix || die



smime.p7s
Description: S/MIME Cryptographic Signature

Re: Incoming SMTP in the year 2017 and absence of DKIM

[John Levine]

In article <85393a12-a51f-6722-4171-118919fcc...@mtcc.com> you write:
>The real problem with large enterprise that we found, however, is that 
>it was really hard to track down every 25 year
>old 386 sitting in dusty corners that was sending mail directly instead 
>of through corpro servers to make certain
>that everything was signed that should be signed. Maybe that's gotten 
>better in the last 15 years, but I'm not too hopeful.

No kidding.  That's why you publish a DMARC policy record that says
don't treat my mail any differently, but please send me summary
reports about it.  This lets you see where mail with your From: domain
is coming from, to track down all those dusty servers.  Once you've
found them all, then you can decide whether publishing a policy is
likely make things better or worse.

You'll also find a whole lot of Chinese botnets that send out spam
with random return addresses including yours, but they're not hard to
tell apart.

R's,
John

Re: Incoming SMTP in the year 2017 and absence of DKIM

[John Levine]

In article <88a1ae22-a5c1-dc46-caa7-cca813109...@tnetconsulting.net> you write:
>  - Requiring Reverse DNS
>  - SPF
>
>I'm not commenting about the viability of these things, just that they 
>are fairly well accepted and that they can trivially break mailing lists.

A mailing list sending with bad rDNS or bad SPF is a pretty cruddy
mailing list.  Normal lists put their own bounce address in the
envelope so they can handle the bounces, so their own SPF applies.

No idea why you think rDNS for a list's MTA is any harder than anyone
else's MTA.

R's,
John

Re: Incoming SMTP in the year 2017 and absence of DKIM

[Brian Kantor]

As I see it, the problem isn't with DKIM, it's with the
implementation of DMARC and other such filters.  Almost all
of them TEST THE WRONG FROM ADDRESS.  They compare the Author's
address (the header From: line) instead of the Sender's address,
(the SMTP Mail From: transaction or Sender: header line).

For personal mail, these are almost always the same, but for
properly-functioning mailing lists, the Author address is the
email address of the person submitting the posting to the mailing
list, and the Sender address is the error-return ("bounce") address
of the mailing list.

If the filter checked the Sender address of mail instead of the
Author address, mailing lists wouldn't be broken!
- Brian


On Wed, Nov 29, 2017 at 10:12:05AM -0800, Michael Thomas wrote:
> I've been saying for years that it should be possible to create the concept
> of DKIM-friendly mailing lists. In such
> a case, you could have your nines. Until then, the best you can hope for is
> the list re-signing the mail and blaming
> the list owner instead.
> 
> Mike

RE: Incoming SMTP in the year 2017 and absence of DKIM

[Keith Medcalf]

In which case neither will they be RFC compliant.

(1) The "inaddr-arpa" ptr from the incoming connection, when resolved, MUST 
result in a set of IP Addresses which includes the original IP Address.

(2) The "name" specified in the HELO/EHLO MUST resolve to an MTA that meets the 
above reverse/forward resolution requirement.

(3) The domain name specified in the envelope-from MUST be resolvable to an MTA 
that meets the above requirement (1) or be empty.

(4) The SPF checking, if done, MUST NOT fail.

(5) The connecting MTA MUST NOT speak when not spoken to (that is, it MUST NOT 
not violate the SMTP chat protocol).

If you dump all connections that are do not meet these requirements, you will 
have eliminated 99% or more of all spam.

DKIM signatures do not really add much at all except prove that the message was 
sent through a server that could calculate a DKIM signature.  It says nothing 
about whether the message is SPAM or not.  99% (or more) of all spam will have 
violated one or more of rules (1) through (5) long before the message contents 
are accepted so that the signature can be verified.

---
The fact that there's a Highway to Hell but only a Stairway to Heaven says a 
lot about anticipated traffic volume.


>-Original Message-
>From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of Eric Kuhnke
>Sent: Wednesday, 29 November, 2017 11:19
>To: nanog@nanog.org list
>Subject: Re: Incoming SMTP in the year 2017 and absence of DKIM
>
>Anecdotal experience. I'm subscribed to a lot of mailing lists. Some
>pass
>through DKIM correctly. Others re-sign the message with DKIM from
>their own
>server.
>
>>98% of the spam that gets through my filters, which comes from an IP
>not
>in any of the major RBLs, has no DKIM signature for the domain. My
>theory
>is that it does introduce somewhat of a barrier to spam senders
>because
>they are frequently not in control of the mail server (which may be
>some
>ignorant third party's open relay), nor do they have access to the
>zonefile
>for the domain the mail server belongs to for the purpose of adding
>any
>sort of DKIM record.
>
>
>
>On Wed, Nov 29, 2017 at 10:12 AM, Michael Thomas 
>wrote:
>
>> On 11/29/2017 10:03 AM, valdis.kletni...@vt.edu wrote:
>>
>>> On Wed, 29 Nov 2017 09:32:27 -0800, Michael Thomas said:
>>>
>>> There are quite a few things you can do to get the mailing list
 traversal rate > 90%, iirc.

>>> Only 90% should be considered horribly broken.  Anything that
>makes
>>> it difficult to run a simple mailing list with less that at least
>2 or 3
>>> 9's
>>> is unacceptable.
>>>
>>
>> I've been saying for years that it should be possible to create the
>> concept of DKIM-friendly mailing lists. In such
>> a case, you could have your nines. Until then, the best you can
>hope for
>> is the list re-signing the mail and blaming
>> the list owner instead.
>>
>> Mike
>>
>>

Re: Incoming SMTP in the year 2017 and absence of DKIM

[Grant Taylor via NANOG]

On 11/29/2017 01:35 PM, Blake Hudson wrote:
Where DKIM/SPF really help is when there's a failure that indicates a 
message has been spoofed.


There are other legitimate things that can break DKIM signatures.  I 
have personally seen changes in content type encoding break a DKIM 
signature.


The message was perfectly valid, and only failed DKIM signature validation.

This is a good indication of phishing and is a 
justified reason to reject or quarantine a message in the interest of 
your employees or subscribers.


As much as I would like to be able to safely reject on DKIM Signature 
validation failure, I don't think that it is safe to do so.


Sometimes these will be config errors, 
but I feel confident telling the sender to take config issues up with 
their service provider.


Hopefully this will bring the perceived problem to someone's attention 
who can hypothetically do something to correct it.




--
Grant. . . .
unix || die



smime.p7s
Description: S/MIME Cryptographic Signature

Re: Incoming SMTP in the year 2017 and absence of DKIM

[Grant Taylor via NANOG]

On 11/29/2017 01:17 PM, Michael Thomas wrote:
Remember: if you treat a broken signature better than lack of signature, 
spammers will just insert phony signatures to game you.


So they really are the same.


Yes, they are /effectively/ the same.  However it is possible to 
distinguish between a broken DKIM signature and the lack of a DKIM 
signature.


What you do with that information is up to you.  -  Guidelines suggest 
that you treat them the same.  (Thus them being /effectively/ the same.)


The real problem with large enterprise that we found, however, is that 
it was really hard to track down every 25 year 
old 386 sitting in dusty corners that was sending mail directly instead 
of through corpro servers to make certain 
that everything was signed that should be signed. Maybe that's gotten 
better in the last 15 years, but I'm not too hopeful.


I hear you, and I don't disagree with your sentiments about the 
difficult of the matter.  However, I find it highly suspect that such 
systems ancient are still in use.  There may very well be replacements 
for said systems that are < 20 years old.


Either way, they would still run afoul of things like SPF (unless you 
allow your entire IP space to send email).


There are other security / vulnerability implications of such 
infrastructures.  -  I'd argue that they are motivation enough to 
wrangle these rogue systems.




--
Grant. . . .
unix || die



smime.p7s
Description: S/MIME Cryptographic Signature

Re: Incoming SMTP in the year 2017 and absence of DKIM

[Chuck Anderson]

On Wed, Nov 29, 2017 at 12:17:57PM -0800, Michael Thomas wrote:
> The real problem with large enterprise that we found, however, is
> that it was really hard to track down every 25 year
> old 386 sitting in dusty corners that was sending mail directly
> instead of through corpro servers to make certain
> that everything was signed that should be signed. Maybe that's
> gotten better in the last 15 years, but I'm not too hopeful.

15 years ago we blocked outbound port 25 except from our campus mail
servers.  That should be SOP by now.  It is fairly easy to look at
firewall logs to find these.

Re: Incoming SMTP in the year 2017 and absence of DKIM

[Blake Hudson]

Eric Kuhnke wrote on 11/29/2017 11:03 AM:

For those who operate public facing SMTPd that receive a large volume of
incoming traffic, and accordingly, a lot of spam...

How much weight do you put on an incoming message, in terms of adding
additional score towards a possible value of spam, for total absence of
DKIM signature?


Spammers can:
    A) Establish domains that use SPF and DKIM as well as anyone else
    B) Use the stolen credentials of legitimate accounts on legitimate 
servers to relay SPAM messages.


So the presence of SPF/DKIM does not reliably indicate whether the 
message is spam or not - only that the sender is "authenticated". The 
lack of optional tech like SPF and DKIM might be used as a heuristic, 
but it's not reliable enough to use in practice in my opinion. I 
wouldn't quarantine or reject messages that are missing these optional 
technology because the take rate isn't high enough.


Where DKIM/SPF really help is when there's a failure that indicates a 
message has been spoofed. This is a good indication of phishing and is a 
justified reason to reject or quarantine a message in the interest of 
your employees or subscribers. Sometimes these will be config errors, 
but I feel confident telling the sender to take config issues up with 
their service provider.

Re: Incoming SMTP in the year 2017 and absence of DKIM

[Michael Thomas]

On 11/29/2017 11:53 AM, Grant Taylor via NANOG wrote:

On 11/29/2017 11:33 AM, Michael Thomas wrote:
A broken DKIM signature is indistinguishable from a lack of a 
signature header.


I'll argue that it's possible to distinguish between the two. 
*However* the DKIM standard states that you should treat a broken DKIM 
signature the same as no DKIM signature.


Remember: if you treat a broken signature better than lack of signature, 
spammers will just insert phony signatures to game you.


So they really are the same.

Not being able to tell if DKIM is in use has been a long standing 
annoyance of mine.


That being said, I think it could be trivial to query for DMARC 
records and deduce things from the existence of the "adkim" option.  
If it's there and set to reject, then there really should be 
DKIM-Signature header for the message. 


I haven't really kept up with dmarc, but its progenitor ssp could give 
you that indication, iirc.


The real problem with large enterprise that we found, however, is that 
it was really hard to track down every 25 year
old 386 sitting in dusty corners that was sending mail directly instead 
of through corpro servers to make certain
that everything was signed that should be signed. Maybe that's gotten 
better in the last 15 years, but I'm not too hopeful.


Mike

Re: Incoming SMTP in the year 2017 and absence of DKIM

[Grant Taylor via NANOG]

On 11/29/2017 11:03 AM, valdis.kletni...@vt.edu wrote:
Only 90% should be considered horribly broken.  Anything that makes it 
difficult to run a simple mailing list with less that at least 2 or 3 
9's is unacceptable.


There have been a number of things that fall into that category, two of 
which immediately come to mind are:


 - Requiring Reverse DNS
 - SPF

I'm not commenting about the viability of these things, just that they 
are fairly well accepted and that they can trivially break mailing lists.


IMHO, DKIM and DMARC are just the recent additions to that list.



--
Grant. . . .
unix || die



smime.p7s
Description: S/MIME Cryptographic Signature

Re: Incoming SMTP in the year 2017 and absence of DKIM

[Grant Taylor via NANOG]

On 11/29/2017 11:33 AM, Michael Thomas wrote:
A broken DKIM signature is indistinguishable from a lack of a signature 
header.


I'll argue that it's possible to distinguish between the two.  *However* 
the DKIM standard states that you should treat a broken DKIM signature 
the same as no DKIM signature.


I've come to understand DKIM to be proof /positive/, as in trust 
something when there is a DKIM signature -and- it validates.  Everything 
else defaults to neutral, NOT /negative/.


It's possible that as a heuristic you might be able to divine something 
from lack of signature and the existence of selectors for a domain, but 
afaik there isn't an easy way to query for all of the dkim selectors for 
a domain, and even if there were it would be a pretty sketchy heuristic, 
is my bet.


Not being able to tell if DKIM is in use has been a long standing 
annoyance of mine.


That being said, I think it could be trivial to query for DMARC records 
and deduce things from the existence of the "adkim" option.  If it's 
there and set to reject, then there really should be DKIM-Signature 
header for the message.




--
Grant. . . .
unix || die



smime.p7s
Description: S/MIME Cryptographic Signature

Re: Incoming SMTP in the year 2017 and absence of DKIM

[Michael Thomas]

A broken DKIM signature is indistinguishable from a lack of a signature 
header. It's possible that as a heuristic
you might be able to divine something from lack of signature and the 
existence of selectors for a domain, but
afaik there isn't an easy way to query for all of the dkim selectors for 
a domain, and even if there were it would

be a pretty sketchy heuristic, is my bet.

Mike

On 11/29/2017 10:18 AM, Eric Kuhnke wrote:

Anecdotal experience. I'm subscribed to a lot of mailing lists. Some pass
through DKIM correctly. Others re-sign the message with DKIM from their own
server.


98% of the spam that gets through my filters, which comes from an IP not

in any of the major RBLs, has no DKIM signature for the domain. My theory
is that it does introduce somewhat of a barrier to spam senders because
they are frequently not in control of the mail server (which may be some
ignorant third party's open relay), nor do they have access to the zonefile
for the domain the mail server belongs to for the purpose of adding any
sort of DKIM record.



On Wed, Nov 29, 2017 at 10:12 AM, Michael Thomas  wrote:


On 11/29/2017 10:03 AM, valdis.kletni...@vt.edu wrote:


On Wed, 29 Nov 2017 09:32:27 -0800, Michael Thomas said:

There are quite a few things you can do to get the mailing list

traversal rate > 90%, iirc.


Only 90% should be considered horribly broken.  Anything that makes
it difficult to run a simple mailing list with less that at least 2 or 3
9's
is unacceptable.


I've been saying for years that it should be possible to create the
concept of DKIM-friendly mailing lists. In such
a case, you could have your nines. Until then, the best you can hope for
is the list re-signing the mail and blaming
the list owner instead.

Mike

Re: Incoming SMTP in the year 2017 and absence of DKIM

[Eric Kuhnke]

Anecdotal experience. I'm subscribed to a lot of mailing lists. Some pass
through DKIM correctly. Others re-sign the message with DKIM from their own
server.

>98% of the spam that gets through my filters, which comes from an IP not
in any of the major RBLs, has no DKIM signature for the domain. My theory
is that it does introduce somewhat of a barrier to spam senders because
they are frequently not in control of the mail server (which may be some
ignorant third party's open relay), nor do they have access to the zonefile
for the domain the mail server belongs to for the purpose of adding any
sort of DKIM record.



On Wed, Nov 29, 2017 at 10:12 AM, Michael Thomas  wrote:

> On 11/29/2017 10:03 AM, valdis.kletni...@vt.edu wrote:
>
>> On Wed, 29 Nov 2017 09:32:27 -0800, Michael Thomas said:
>>
>> There are quite a few things you can do to get the mailing list
>>> traversal rate > 90%, iirc.
>>>
>> Only 90% should be considered horribly broken.  Anything that makes
>> it difficult to run a simple mailing list with less that at least 2 or 3
>> 9's
>> is unacceptable.
>>
>
> I've been saying for years that it should be possible to create the
> concept of DKIM-friendly mailing lists. In such
> a case, you could have your nines. Until then, the best you can hope for
> is the list re-signing the mail and blaming
> the list owner instead.
>
> Mike
>
>

Re: Incoming SMTP in the year 2017 and absence of DKIM

[Michael Thomas]

On 11/29/2017 10:03 AM, valdis.kletni...@vt.edu wrote:

On Wed, 29 Nov 2017 09:32:27 -0800, Michael Thomas said:


There are quite a few things you can do to get the mailing list
traversal rate > 90%, iirc.

Only 90% should be considered horribly broken.  Anything that makes
it difficult to run a simple mailing list with less that at least 2 or 3 9's
is unacceptable.


I've been saying for years that it should be possible to create the 
concept of DKIM-friendly mailing lists. In such
a case, you could have your nines. Until then, the best you can hope for 
is the list re-signing the mail and blaming

the list owner instead.

Mike

Re: Incoming SMTP in the year 2017 and absence of DKIM

[valdis . kletnieks]

On Wed, 29 Nov 2017 09:32:27 -0800, Michael Thomas said:

> There are quite a few things you can do to get the mailing list
> traversal rate > 90%, iirc.

Only 90% should be considered horribly broken.  Anything that makes
it difficult to run a simple mailing list with less that at least 2 or 3 9's
is unacceptable.


pgpIvpNci6U2A.pgp
Description: PGP signature

Re: Incoming SMTP in the year 2017 and absence of DKIM

[Ken O'Driscoll]

On Wed, 2017-11-29 at 12:24 -0500, William Herrin wrote:
> Alright, so "horribly broken design" overstates the case but there are
> enough problems that weighting the absence of DKIM at something other
> than zero will surely do more harm than good.

+1. A DKIM signature by itself means nothing more than someone had the
ability to configure DKIM on an email server.

The signing domain (d=) is what matters as the signer needs access to the
zone in order to be able to publish the key, which may be interpreted as an
indication of trust.

DMARC requires the signing domain to be either exactly the same or share
the same organisational unit with the From address for this reason.

Even without DMARC, a receiver *could*, depending on the signing domain,
choose to interpret it as a positive signal. This is marginally better than
treating any DKIM signature or the absence thereof as a signal of any kind.

Personally, unless an author domain is publishing a DMARC policy of reject
or quarantine, I don't think recipients should be scoring based on DKIM at
all, perhaps with the exception of signing with a revoked key.

Ken.

-- 
Ken O'Driscoll / We Monitor Email
t: +353 1 254 9400 | w: www.wemonitoremail.com

Need to understand deliverability? Now there's a book:
www.wemonitoremail.com/book

Re: Incoming SMTP in the year 2017 and absence of DKIM

[Michael Thomas]

On 11/29/2017 09:24 AM, William Herrin wrote:

On Wed, Nov 29, 2017 at 12:17 PM, Stephen Frost  wrote:


* William Herrin (b...@herrin.us) wrote:

On Wed, Nov 29, 2017 at 12:03 PM, Eric Kuhnke 

wrote:

How much weight do you put on an incoming message, in terms of adding
additional score towards a possible value of spam, for total absence of
DKIM signature?

Zero. DKIM for mailing lists is a horribly broken design and legitimate
mailing lists are second only to spam in quantity of SMTP transactions.

Eh, that's really not accurate, imv, and some folks who run mailing
lists have put in serious effort to make sure to *not* break DKIM
signatures (which is certainly possible to do).


Alright, so "horribly broken design" overstates the case but there are
enough problems that weighting the absence of DKIM at something other than
zero will surely do more harm than good.



There are quite a few things you can do to get the mailing list 
traversal rate > 90%, iirc. For average mailman-like
lists like nanog it's very high. Of course while a "badly" behaving 
mailing list can trivially defeat any DKIM signature,
it doesn't really take too much effort to not behave "badly". Whether 
that false positive rate is too high is another

question.

Mike

Re: Incoming SMTP in the year 2017 and absence of DKIM

[William Herrin]

On Wed, Nov 29, 2017 at 12:17 PM, Stephen Frost  wrote:

> * William Herrin (b...@herrin.us) wrote:
> > On Wed, Nov 29, 2017 at 12:03 PM, Eric Kuhnke 
> wrote:
> > > How much weight do you put on an incoming message, in terms of adding
> > > additional score towards a possible value of spam, for total absence of
> > > DKIM signature?
> >
> > Zero. DKIM for mailing lists is a horribly broken design and legitimate
> > mailing lists are second only to spam in quantity of SMTP transactions.
>
> Eh, that's really not accurate, imv, and some folks who run mailing
> lists have put in serious effort to make sure to *not* break DKIM
> signatures (which is certainly possible to do).


Alright, so "horribly broken design" overstates the case but there are
enough problems that weighting the absence of DKIM at something other than
zero will surely do more harm than good.

Regards,
Bill Herrin


-- 
William Herrin  her...@dirtside.com  b...@herrin.us
Dirtside Systems . Web: 

Re: Incoming SMTP in the year 2017 and absence of DKIM

[Stephen Frost]

Greetings,

* William Herrin (b...@herrin.us) wrote:
> On Wed, Nov 29, 2017 at 12:03 PM, Eric Kuhnke  wrote:
> 
> > For those who operate public facing SMTPd that receive a large volume of
> > incoming traffic, and accordingly, a lot of spam...
> >
> > How much weight do you put on an incoming message, in terms of adding
> > additional score towards a possible value of spam, for total absence of
> > DKIM signature?
> 
> Zero. DKIM for mailing lists is a horribly broken design and legitimate
> mailing lists are second only to spam in quantity of SMTP transactions.

Eh, that's really not accurate, imv, and some folks who run mailing
lists have put in serious effort to make sure to *not* break DKIM
signatures (which is certainly possible to do).  The combination of
making DKIM signatures work and DMARC allows messages to go through that
would otherwise end up getting bounced, from what I've seen.

What's annoying are the systems that appear to assume a DMARC policy
that says "bounce it if it's not from a server in our SPF list" when
there's no DMARC policy in place, but there is an SPF list.  Not
everyone really wants to put in the effort to set up DKIM, but they're
fine putting up an SPF record, but there seem to be a number of servers
out there that bounce mailing list traffic in those cases (seems to
specifically be MS Exchange systems, from the google'ing that I've
done).

Thanks!

Stephen


signature.asc
Description: Digital signature

Re: Incoming SMTP in the year 2017 and absence of DKIM

[William Herrin]

On Wed, Nov 29, 2017 at 12:03 PM, Eric Kuhnke  wrote:

> For those who operate public facing SMTPd that receive a large volume of
> incoming traffic, and accordingly, a lot of spam...
>
> How much weight do you put on an incoming message, in terms of adding
> additional score towards a possible value of spam, for total absence of
> DKIM signature?
>

Zero. DKIM for mailing lists is a horribly broken design and legitimate
mailing lists are second only to spam in quantity of SMTP transactions.

Regards,
Bill Herrin


-- 
William Herrin  her...@dirtside.com  b...@herrin.us
Dirtside Systems . Web: