Re: [dmarc-ietf] Support for DMARC p=reject
Hi Doug, At 17:07 23-05-2014, Douglas Otis wrote: I have been asked nearly the same question by others. A section will be added to explain the how and why and its impact on privacy. Senders and receivers desire a scheme that improves protection of the From header field. The business aspects of this became extremely clear to PayPal by effects on their customers then obligated to sift through massive amounts of email abuse. The benefit of timely out-of-band notification instead caused an exodus of customers. To staunch customer loss, direct appeals to major ESPs to reject non-aligned PayPal messages helped, and helped everyone. Unfortunately, direct appeal does not scale. Hence we have a limited stop-gap scheme. I was not thinking about privacy in that comment. Some time back I looked into why abuse occurred in different systems. I was not trying to understand the matter instead of trying to resolve it. Note that I have read the relevant papers. Anyway, you did not answer the question I asked. :-) PayPal is the not really a good case in my opinion. I do not get money by solving PayPal's problem. I might put it some effort as encouraging abuse is not a good idea. A problem, which you identified, is that a scheme would have to scale or else be workable at some scale. As has become extremely clear, although there was early evidence, DMARC did not fully consider important corner cases. It seemed okay to leave these issues for receivers to sort out. Now AOL and Yahoo! have made it evident receiver cooperation is in jeopardy. If you are like most others, you have had malicious and socially engineered messages from someone you know prompting you to click a link or call a number, etc. Resorting to the only functional policy scheme able to reject messages is understandable, but this came at a dear cost. As John Levine describes, a death by a thousand cuts. If you push things too strongly I (as a receiver) would be no longer be inclined to be cooperative. Asserting whether all messages must have From/Source domain alignment is somewhat analogous to the type of federation supporting single-sign-on schemes. A federation has the purpose of protecting identifiers. In other words using laissez faire protocols where the strategy of block on event is inverted to become accept on event. In the case of DMARC + TPA, accept on event is derived from DMARC feedback/log review accurately determining which domains require exception. Further review can even characterize how domains authenticate and which header elements confirm or narrow authorization. I'll say ok to the above. Cute comic. It is not really the information protected by a federation scheme. It is not even the exclusion of bad stuff. It is protection of federated identities by receivers implementing DMARC + TPA. By using this scheme, associated identities can be protected while also avoiding disruption of legitimate messages. Even Eliot Lear's mother becomes safer. (I hope this is the right example as it was discussed many years ago.) The problem might be one of trust. I won't even try to analyze that as it is going to end up being a lot of work. Unfortunately, that approach now only works for authenticated domains. :^( Dane anyone? You'll have other problems then. :-( Regards, S. Moonesamy ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] Support for DMARC p=reject
Hi Doug, At 14:32 22-05-2014, Douglas Otis wrote: The goal of federated services is to prevent inclusion of unknown servers. It does not in itself exclude bad stuff. When bad stuff is noted, federation enables blocking future exchanges. Direct tweets are possible, although messaging constrained to 140 characters is difficult to take seriously since it is nearly impossible to know whether a tweet is being auto-generated. Much of it is. The way this would relate to DMARC is in the handling of non-aligned messages. When sources are identified, and feedback is generated for all known exceptions, it should not be difficult to then invert block logic into accept logic. The question I asked was: is federation effective in excluding bad stuff? The reason I asked that question is to be able to form an opinion about the idea of an email federation. The above explains that the email federation is about preventing the inclusion of unknown servers. That does not help me in forming an opinion about the idea (see the question I asked). I used tweeter as an example of a closed system. The argument was that bad stuff can happen in a system in which the owner has full control. Imagine your domain supporting millions of users had their accounts exposed along with their address-books. This can easily get ugly in respect to the harm this would allow. I read that information security and customer data protection are of paramount importance. My interpretation of that is that the importance is ranked higher than money. My interpretation would be incorrect if my domain had millions of accounts exposed. Someone might also point me to http://www.dilbert.com/strips/comic/2014-05-19/ I supported a system that reported on _all_ identified bad IPv4 addresses. To do this, it required careful exclusion of those randomly assigned addresses. We would then apply about 15 million updates every 5 minutes for the entire IPv4 address space orchestrated by two redundant servers. This information then supported several very large ISPs. The larger ISPs wanted to do zone transfers, but a DMARC exception rate for an Author Domain will be several orders of magnitude lower in scale. I'll admit this was all done using C code that avoided SQL or Hadoop. Judy is your friend. :^). I'll make this a little complicated for you. Could that system also be used for IPv6 addresses? Regards, S. Moonesamy ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] Support for DMARC p=reject
Dear SM, Thank you for your thoughtful questions. See comments inline: On May 22, 2014, at 11:19 PM, S Moonesamy sm+i...@elandsys.com wrote: Hi Doug, At 14:32 22-05-2014, Douglas Otis wrote: The goal of federated services is to prevent inclusion of unknown servers. It does not in itself exclude bad stuff. When bad stuff is noted, federation enables blocking future exchanges. Direct tweets are possible, although messaging constrained to 140 characters is difficult to take seriously since it is nearly impossible to know whether a tweet is being auto-generated. Much of it is. The way this would relate to DMARC is in the handling of non-aligned messages. When sources are identified, and feedback is generated for all known exceptions, it should not be difficult to then invert block logic into accept logic. The question I asked was: is federation effective in excluding bad stuff? The reason I asked that question is to be able to form an opinion about the idea of an email federation. The above explains that the email federation is about preventing the inclusion of unknown servers. That does not help me in forming an opinion about the idea (see the question I asked). I used tweeter as an example of a closed system. The argument was that bad stuff can happen in a system in which the owner has full control. I have been asked nearly the same question by others. A section will be added to explain the how and why and its impact on privacy. Senders and receivers desire a scheme that improves protection of the From header field. The business aspects of this became extremely clear to PayPal by effects on their customers then obligated to sift through massive amounts of email abuse. The benefit of timely out-of-band notification instead caused an exodus of customers. To staunch customer loss, direct appeals to major ESPs to reject non-aligned PayPal messages helped, and helped everyone. Unfortunately, direct appeal does not scale. Hence we have a limited stop-gap scheme. As has become extremely clear, although there was early evidence, DMARC did not fully consider important corner cases. It seemed okay to leave these issues for receivers to sort out. Now AOL and Yahoo! have made it evident receiver cooperation is in jeopardy. If you are like most others, you have had malicious and socially engineered messages from someone you know prompting you to click a link or call a number, etc. Resorting to the only functional policy scheme able to reject messages is understandable, but this came at a dear cost. As John Levine describes, a death by a thousand cuts. Asserting whether all messages must have From/Source domain alignment is somewhat analogous to the type of federation supporting single-sign-on schemes. A federation has the purpose of protecting identifiers. In other words using laissez faire protocols where the strategy of block on event is inverted to become accept on event. In the case of DMARC + TPA, accept on event is derived from DMARC feedback/log review accurately determining which domains require exception. Further review can even characterize how domains authenticate and which header elements confirm or narrow authorization. TPA uses DNS to signal trust based on hash labels. DNS is used because it is ubiquitously relied upon by MTAs, offers low latency and low overhead from legitimate domains. DNS is also able to handily support this scheme at massive scales. Imagine your domain supporting millions of users had their accounts exposed along with their address-books. This can easily get ugly in respect to the harm this would allow. I read that information security and customer data protection are of paramount importance. My interpretation of that is that the importance is ranked higher than money. My interpretation would be incorrect if my domain had millions of accounts exposed. Someone might also point me to http://www.dilbert.com/strips/comic/2014-05-19/ Cute comic. It is not really the information protected by a federation scheme. It is not even the exclusion of bad stuff. It is protection of federated identities by receivers implementing DMARC + TPA. By using this scheme, associated identities can be protected while also avoiding disruption of legitimate messages. Even Eliot Lear's mother becomes safer. (I hope this is the right example as it was discussed many years ago.) I supported a system that reported on _all_ identified bad IPv4 addresses. To do this, it required careful exclusion of those randomly assigned addresses. We would then apply about 15 million updates every 5 minutes for the entire IPv4 address space orchestrated by two redundant servers. This information then supported several very large ISPs. The larger ISPs wanted to do zone transfers, but a DMARC exception rate for an Author Domain will be several orders of magnitude lower in scale. I'll admit
Re: [dmarc-ietf] Support for DMARC p=reject
Hi, Doug, On 05/22/2014 10:21 PM, Douglas Otis wrote: Dear Brandon, See comments inline: On May 12, 2014, at 12:30 PM, Brandon Long bl...@google.com mailto:bl...@google.com wrote: On Mon, May 12, 2014 at 12:16 PM, Douglas Otis doug.mtv...@gmail.com mailto:doug.mtv...@gmail.com wrote: On May 11, 2014, at 12:47 PM, Gabriel Iovino giov...@people.ops-trust.net mailto:giov...@people.ops-trust.net wrote: Greetings, Last week I was having a conversation with a familiar person on this mailing list and I was expressing my disappointment with the negativity towards Yahoo[1] and AOL[2] for breaking email. I was encouraged to share these thoughts on this list. I believe email is already broken[3][4][5][6] and DMARC p=reject moves us towards a position where email is less broken. Will there be some bumps[7] along the road? Sure but a few bumps are no reason to leave email in it's current state. I applaud Yahoo and AOL for taking the first few punches and I look forward to the day when Google and Microsoft follow their lead. Thank you for all the hard work you have done to improve the state of email! Gabriel Iovino Dear Gabriel, While email is generally abused, DMARC's intent was to better protect transactional email which Yahoo may put in jeopardy. There will be a forthcoming draft to allow Author-Domains a means to request restrictive policies against normal user email accounts without disrupting very legitimate communications. The draft places the burden of mitigating disruption on those making the requests. Otherwise, it won't be too much longer before even DMARC is ignored when misapplied against user accounts. Where can we learn more about this? An update is pending recovery of xml.resource.org/public/rfc/bibxml/ http://xml.resource.org/public/rfc/bibxml/. You don't miss it until it is gone. :^( I should have been more proactive about transferring reference content. Yahoo has suffered from a lack of security permitting millions of their users accounts to be compromised. A better approach would not use DMARC, but would federate third-party services they can see their customers employ. The federation of email, much like that of XMPP, would be an effective means to exclude bad actors without breaking mailing-list and other third-party email services. As it is now, it seems Yahoo only protects their own mailing-list operations which really does not warrant a basis for applauding such efforts. I feel that there is a theory that has gone around as to why AOL and Yahoo! have done this, but I don't know as there has been any proof of that or acknowledgement. For one, the level of hijacking we see, and the level of spam I personally receive that has at least a From name of someone I know, lead me to question that theory. I have notified several friends that their accounts were compromised. Most did not like having to change their password. Also, unless you know otherwise, my understanding was that Yahoo Groups didn't have any mitigation of DMARC policies until recently, and they implemented the same (and only currently useful) mitigation of re-writing the From header, and did so well after yahoo.com http://yahoo.com/ went to REJECT. Rewriting the From header field in itself is disruptive. This prevents review of prior conversations from an individual. Often you might remember who said something without recalling some of the details. Also... federation across millions of servers? That seems... unlikely. Federation simply means sending servers authenticate their domain and allow receivers to decide whether they wish to disallow messaging from unknown domains. That feature is sorely missing from SMTP. In this case, it only comes into play for third-party servers used by users of the Author-Domain asserting a DMARC policy request. The scale of this is likely to be in the area of about 30K. This number of 30K has been first mentioned by Yahoo! and after that it has been mentioned a couple of times by various people, but I have yet to see any proof that this figure is correct. Apart from this, quoting your own mail, you mention [...] tens of thousands of legitimate services that might be sending on behalf of their client [...]. Although I think TPA may have its use for specific author/sender combinations [1], it definitely is not the answer to the current problems, introduced by Yahoo! and AOL, when they activated 'p=reject'. It simply will not scale enough and it remains to be seen that the too-big-to-ignore ESPs will spend time and money on the use of TPA, as they have their own mailing-list-like fora, which provide them revenues. Not to mention the privacy aspects of TPA... /rolf [1] My company DKIM-signs mail on behalf of some customers, a proper TPA
Re: [dmarc-ietf] Support for DMARC p=reject
On May 12, 2014, at 1:38 PM, S Moonesamy sm+i...@elandsys.com wrote: Hi Doug, At 12:16 12-05-2014, Douglas Otis wrote: be compromised. A better approach would not use DMARC, but would federate third-party services they can see their customers employ. The federation of email, much like that of XMPP, would be an effective means to exclude bad actors without breaking mailing-list and other third-party email services. As it is now, it seems Yahoo only protects their own Is federation effective in excluding bad stuff? Bad stuff also occurs in closed systems (see https://support.twitter.com/groups/56-policies-violations/topics/238-report-a-violation/articles/64986-reporting-spam-on-twitter ). XMPP has been mentioned previously. There wasn't any information provided to assess whether it does exclude bad stuff. Dear SM, The goal of federated services is to prevent inclusion of unknown servers. It does not in itself exclude bad stuff. When bad stuff is noted, federation enables blocking future exchanges. Direct tweets are possible, although messaging constrained to 140 characters is difficult to take seriously since it is nearly impossible to know whether a tweet is being auto-generated. Much of it is. The way this would relate to DMARC is in the handling of non-aligned messages. When sources are identified, and feedback is generated for all known exceptions, it should not be difficult to then invert block logic into accept logic. Imagine your domain supporting millions of users had their accounts exposed along with their address-books. This can easily get ugly in respect to the harm this would allow. Yahoo likely considered DMARC the closest approximation of server federation that could be enabled. After all, there are no other email policy layers reliably acted upon. What is missing is a minor amount of feedback needed to communicate what the Author-Domain knows to be legitimate sources. Doing so should represent a negligible amount of effort. I supported a system that reported on _all_ identified bad IPv4 addresses. To do this, it required careful exclusion of those randomly assigned addresses. We would then apply about 15 million updates every 5 minutes for the entire IPv4 address space orchestrated by two redundant servers. This information then supported several very large ISPs. The larger ISPs wanted to do zone transfers, but a DMARC exception rate for an Author Domain will be several orders of magnitude lower in scale. I'll admit this was all done using C code that avoided SQL or Hadoop. Judy is your friend. :^). Regards, Douglas Otis ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] Support for DMARC p=reject
On May 22, 2014, at 2:03 PM, Rolf E. Sonneveld r.e.sonnev...@sonnection.nl wrote: Hi, Doug, On 05/22/2014 10:21 PM, Douglas Otis wrote: Dear Brandon, See comments inline: On May 12, 2014, at 12:30 PM, Brandon Long bl...@google.com wrote: On Mon, May 12, 2014 at 12:16 PM, Douglas Otis doug.mtv...@gmail.com wrote: On May 11, 2014, at 12:47 PM, Gabriel Iovino giov...@people.ops-trust.net wrote: Greetings, Last week I was having a conversation with a familiar person on this mailing list and I was expressing my disappointment with the negativity towards Yahoo[1] and AOL[2] for breaking email. I was encouraged to share these thoughts on this list. I believe email is already broken[3][4][5][6] and DMARC p=reject moves us towards a position where email is less broken. Will there be some bumps[7] along the road? Sure but a few bumps are no reason to leave email in it's current state. I applaud Yahoo and AOL for taking the first few punches and I look forward to the day when Google and Microsoft follow their lead. Thank you for all the hard work you have done to improve the state of email! Gabriel Iovino Dear Gabriel, While email is generally abused, DMARC's intent was to better protect transactional email which Yahoo may put in jeopardy. There will be a forthcoming draft to allow Author-Domains a means to request restrictive policies against normal user email accounts without disrupting very legitimate communications. The draft places the burden of mitigating disruption on those making the requests. Otherwise, it won't be too much longer before even DMARC is ignored when misapplied against user accounts. Where can we learn more about this? An update is pending recovery of xml.resource.org/public/rfc/bibxml/. You don't miss it until it is gone. :^( I should have been more proactive about transferring reference content. Yahoo has suffered from a lack of security permitting millions of their users accounts to be compromised. A better approach would not use DMARC, but would federate third-party services they can see their customers employ. The federation of email, much like that of XMPP, would be an effective means to exclude bad actors without breaking mailing-list and other third-party email services. As it is now, it seems Yahoo only protects their own mailing-list operations which really does not warrant a basis for applauding such efforts. I feel that there is a theory that has gone around as to why AOL and Yahoo! have done this, but I don't know as there has been any proof of that or acknowledgement. For one, the level of hijacking we see, and the level of spam I personally receive that has at least a From name of someone I know, lead me to question that theory. I have notified several friends that their accounts were compromised. Most did not like having to change their password. Also, unless you know otherwise, my understanding was that Yahoo Groups didn't have any mitigation of DMARC policies until recently, and they implemented the same (and only currently useful) mitigation of re-writing the From header, and did so well after yahoo.com went to REJECT. Rewriting the From header field in itself is disruptive. This prevents review of prior conversations from an individual. Often you might remember who said something without recalling some of the details. Also... federation across millions of servers? That seems... unlikely. Federation simply means sending servers authenticate their domain and allow receivers to decide whether they wish to disallow messaging from unknown domains. That feature is sorely missing from SMTP. In this case, it only comes into play for third-party servers used by users of the Author-Domain asserting a DMARC policy request. The scale of this is likely to be in the area of about 30K. This number of 30K has been first mentioned by Yahoo! and after that it has been mentioned a couple of times by various people, but I have yet to see any proof that this figure is correct. Apart from this, quoting your own mail, you mention [...] tens of thousands of legitimate services that might be sending on behalf of their client [...]. Although I think TPA may have its use for specific author/sender combinations [1], it definitely is not the answer to the current problems, introduced by Yahoo! and AOL, when they activated 'p=reject'. It simply will not scale enough and it remains to be seen that the too-big-to-ignore ESPs will spend time and money on the use of TPA, as they have their own mailing-list-like fora, which provide them revenues. Not to mention the privacy aspects of TPA... Dear Rolf, I strongly disagree with the assumption TPA does not scale to levels needed by AOL or Yahoo. Not having TPA declare the exceptions needed, both receiving and then offering feedback will likely to involve greater
Re: [dmarc-ietf] Support for DMARC p=reject
Hi, Doug, On 05/22/2014 11:56 PM, Douglas Otis wrote: [...] This number of 30K has been first mentioned by Yahoo! and after that it has been mentioned a couple of times by various people, but I have yet to see any proof that this figure is correct. Apart from this, quoting your own mail, you mention [...] tens of thousands of legitimate services that might be sending on behalf of their client [...]. Although I think TPA may have its use for specific author/sender combinations [1], it definitely is not the answer to the current problems, introduced by Yahoo! and AOL, when they activated 'p=reject'. It simply will not scale enough and it remains to be seen that the too-big-to-ignore ESPs will spend time and money on the use of TPA, as they have their own mailing-list-like fora, which provide them revenues. Not to mention the privacy aspects of TPA... Dear Rolf, I strongly disagree with the assumption TPA does not scale to levels needed by AOL or Yahoo. Not having TPA declare the exceptions needed, both receiving and then offering feedback will likely to involve greater levels of network resources. TPA is structured to ensure it can be supported by a single DNS transaction. This allows for effective caching so perhaps only one out of 10 queries sees an authoritative response. Can that be said for any other email protocol? I'm not talking about that type of scaling. What I mean is, that it will take a massive amount of work to gather the right information about who is subscribed where to what list, to update this information on a continuous basis and to get this (changing information) in DNS. But maybe I misunderstand what will be in the draft, I'd better wait for the draft. /rolf ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] Support for DMARC p=reject
Hi Doug, At 12:16 12-05-2014, Douglas Otis wrote: be compromised. A better approach would not use DMARC, but would federate third-party services they can see their customers employ. The federation of email, much like that of XMPP, would be an effective means to exclude bad actors without breaking mailing-list and other third-party email services. As it is now, it seems Yahoo only protects their own Is federation effective in excluding bad stuff? Bad stuff also occurs in closed systems (see https://support.twitter.com/groups/56-policies-violations/topics/238-report-a-violation/articles/64986-reporting-spam-on-twitter ). XMPP has been mentioned previously. There wasn't any information provided to assess whether it does exclude bad stuff. Regards, S. Moonesamy ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
[dmarc-ietf] Support for DMARC p=reject
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Greetings, Last week I was having a conversation with a familiar person on this mailing list and I was expressing my disappointment with the negativity towards Yahoo[1] and AOL[2] for breaking email. I was encouraged to share these thoughts on this list. I believe email is already broken[3][4][5][6] and DMARC p=reject moves us towards a position where email is less broken. Will there be some bumps[7] along the road? Sure but a few bumps are no reason to leave email in it's current state. I applaud Yahoo and AOL for taking the first few punches and I look forward to the day when Google and Microsoft follow their lead. Thank you for all the hard work you have done to improve the state of email! Gabriel Iovino [1] Yahoo DMARC policy https://help.yahoo.com/kb/postmaster/yahoo-dmarc-policy-sln24050.html [2] AOL Mail updates DMARC policy to 'reject' http://postmaster-blog.aol.com/2014/04/22/aol-mail-updates-dmarc-policy-to-reject/ [3] Identifying fraudulent phishing email http://support.apple.com/kb/ht4933 [4] How to Find Out Where a Link Will Take You in iPhone Mail http://email.about.com/od/iphonemailtips/qt/et_see_url.htm [5] Obtaining Email Headers for Spam/Phishing Notification https://community.shaw.ca/docs/DOC-1132 [6] Check It Before You Click It - Phishing, Malicious Links Spoofed Headers http://grok.lsu.edu/article.aspx?articleId=17107 [7] Mailman April 2014 Archives by thread http://lists.dmarc.org/pipermail/dmarc-discuss/2014-April/thread.html -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.22 (Darwin) Comment: GPGTools - https://gpgtools.org iEYEARECAAYFAlNv0+cACgkQwqygxIz+pTuVqQCeNYB4Uhtgd7Q8YqQKQTfcFkDX a20An0rQ6xmBzGP6tcHAjcs/Mzv1JWWx =0UWT -END PGP SIGNATURE- ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
