Re: [Mailman-Users] Executive summary of DMARC issues
Gary Algier writes: I ran some tests this morning. I created an Exchange distribution list here and added myself five ways on the list: 1. On our Exchange server (how I receive internal emails) 2. On a local Linux server running sendmail and dovecot (how I receive real mail) 3. A Yahoo address. 4. A Gmail address. 5. An iCloud address. I then sent an email to the list and to my work sendmail address. Where did you send the mail from, what address was in From, and what host did the DKIM signing? Does the domain listed in From have a DMARC record? The DKIM checks seem to be good. So, it seems that nothing has changed in the content or checked header. It must be SPf. It could be SPF, but if it is it has nothing to do with DMARC. DMARC accepts either SPF or DKIM as evidence of authenticity. That is, either may fail as long as at least one succeeds. If it is indeed SPF, then it doesn't matter what you use. The problem is that the host where the distribution list or mailing list is hosted is not SPF-authorized, and almost certainly not which MTA or MLM you use. I'm not sure if you care about DMARC, or just whether it gets through. But if the latter, I'm not at all clear on exactly what you're trying to test. % dig +short TXT _spf.mail.yahoo.com v=spf1 ptr:yahoo.com ptr:yahoo.net ip4:206.108.40.0/27 ip4:199.16.139.0/26 ?all This is mostly unrelated to Yahoo's behavior when receiving messages. Amusingly enough, RFC 7208 deprecates the ptr mechanism strongly. -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Executive summary of DMARC issues
On May 16, 2014, at 5:49, Stephen J. Turnbull step...@xemacs.org wrote: Gary Algier writes: I ran some tests this morning. I created an Exchange distribution list here and added myself five ways on the list: 1. On our Exchange server (how I receive internal emails) 2. On a local Linux server running sendmail and dovecot (how I receive real mail) 3. A Yahoo address. 4. A Gmail address. 5. An iCloud address. I then sent an email to the list and to my work sendmail address. Where did you send the mail from, what address was in From, and what host did the DKIM signing? Does the domain listed in From have a DMARC record? Sorry, I forgot to mention that I sent from yahoo for these tests. The From: header said x...@yahoo.com. The DKIM checks seem to be good. So, it seems that nothing has changed in the content or checked header. It must be SPf. It could be SPF, but if it is it has nothing to do with DMARC. DMARC accepts either SPF or DKIM as evidence of authenticity. That is, either may fail as long as at least one succeeds. If it is indeed SPF, then it doesn't matter what you use. The problem is that the host where the distribution list or mailing list is hosted is not SPF-authorized, and almost certainly not which MTA or MLM you use. I'm not sure if you care about DMARC, or just whether it gets through. But if the latter, I'm not at all clear on exactly what you're trying to test. My management wants to replace our in-house email with an Exchange Online (aka MS365) solution. Several high priced consultants have told them that Exchange can do it all. I am trying to prove otherwise. Actually to be fare, two consulting firms have said: 1. Exchange can't do what Mailman can. 2. MS365 can't work with Mailman. 3. MS365 will cost more, don't do it. They are being ignored because they did not supply the answers wanted. % dig +short TXT _spf.mail.yahoo.com v=spf1 ptr:yahoo.com ptr:yahoo.net ip4:206.108.40.0/27 ip4:199.16.139.0/26 ?all This is mostly unrelated to Yahoo's behavior when receiving messages. Amusingly enough, RFC 7208 deprecates the ptr mechanism strongly. -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Executive summary of DMARC issues
Around a year ago we started using Exchange Online. In my experience, the undesired responses are correct. On May 16, 2014, at 09:13 AM, mail.ulticom.com g...@ulticom.com wrote: My management wants to replace our in-house email with an Exchange Online (aka MS365) solution. Several high priced consultants have told them that Exchange can do it all. I am trying to prove otherwise. Actually to be fare, two consulting firms have said: 1. Exchange can't do what Mailman can. It has fancy forwards, but besides rudimentary moderation, very little in the way of traditional mailing list features. 2. MS365 can't work with Mailman. I migrated my lists to a dedicated subdomain to work around this, and some of the details surrounding that are documented in the list archives. 3. MS365 will cost more, don't do it. This may or may not be true. It’s a subscription service, so costs will change. It seems like these services are trending to go down in price as competition goes up and more users sign on, but not always. (We qualified for nonprofit pricing, which has gone from $2/user/mo to FREE, which is nice. However, we’ve been told that we no longer qualify for the nonprofit program.) Feel free to contact me directly for more details. -- Matthew Needham mneed...@hdfgroup.org 217-531-6110 The HDF Group 1800 South Oak Street, Suite 203 Champaign, IL 61820 -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Executive summary of DMARC issues
On May 14, 2014, at 11:47 PM, Stephen J. Turnbull step...@xemacs.org wrote: Peter Shute writes: When MS365 forwards the mails sent to the distribution list, should that make the DMARC authentication fail? I thought that only happened if you made changes like adding a prefix to the subject line like Mailman does. If it forwards verbatim *and* the sending domain signs the mail with DKIM (the common case), DMARC validation will succeed. Without DKIM, DMARC validation is guaranteed to fail. However, even in the sender uses DKIM, *any* change *whatsoever* to the body will cause validation to fail, and there are several changes to the header that could cause it to fail. Furthermore, which parts of the header are protected by the DKIM signature are determined by the sender, not by DMARC AFAIK. If distribution lists are pure forwards, MS365 will be OK. But I find it hard to believe that that level of functionality is popular with users -- there's a reason why all popular MLMs implement subject prefixes, body headers and body footers, and it isn't because it's the Microsoft way. Especially as legally mailing lists are required to add unsubscribe instructions in the footer. best regards, Larry -- Larry Finch finc...@portadmiral.org -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Executive summary of DMARC issues
On 05/14/14 23:47, Stephen J. Turnbull wrote: Peter Shute writes: When MS365 forwards the mails sent to the distribution list, should that make the DMARC authentication fail? I thought that only happened if you made changes like adding a prefix to the subject line like Mailman does. If it forwards verbatim *and* the sending domain signs the mail with DKIM (the common case), DMARC validation will succeed. Without DKIM, DMARC validation is guaranteed to fail. However, even in the sender uses DKIM, *any* change *whatsoever* to the body will cause validation to fail, and there are several changes to the header that could cause it to fail. Furthermore, which parts of the header are protected by the DKIM signature are determined by the sender, not by DMARC AFAIK. If distribution lists are pure forwards, MS365 will be OK. But I find it hard to believe that that level of functionality is popular with users -- there's a reason why all popular MLMs implement subject prefixes, body headers and body footers, and it isn't because it's the Microsoft way. I ran some tests this morning. I created an Exchange distribution list here and added myself five ways on the list: 1. On our Exchange server (how I receive internal emails) 2. On a local Linux server running sendmail and dovecot (how I receive real mail) 3. A Yahoo address. 4. A Gmail address. 5. An iCloud address. I then sent an email to the list and to my work sendmail address. It was delivered to both work addresses and the iCloud address. Gmail put it in my Spam folder with the warning: --- Be careful with this message. Our systems couldn't verify that this message was really sent by yahoo.com. You might want to avoid clicking links or replying with personal information. --- There is also a link to their Why messages are marked as Spam page. On Yahoo I found the bounce in my Spam folder with the following: --- This is an automated message from the Extensible Content Security at host wg.ulticom.com. The message returned below could not be delivered to its intended destinations. For further assistance, please send mail to ...@wg.ulticom.com. If you do so, please include this problem report. You can delete your own text from the message returned below. Reason: x...@yahoo.com: host mta7.am0.yahoodns.net[98.138.112.34] said: 554 5.7.9 Message not accepted for policy reasons. See http://postmaster.yahoo.com/errors/postmaster-28.html (in reply to end of DATA command) --- The server wg.ulticom.com is our WatchGuard Anti-Spam appliance. It had no trouble accepting it when it came in the first time (it does not do DMARC checks), but when it tried to delivery the email to Yahoo, they rejected it. Of course, the reject went to Yahoo anyway, but with a MAILER-DAEMON sender address. I saved the two copies from my sendmail address and compared them: % h=$(sed -n -e 'y/:/|/' -e '/DKIM-Signature|/s/.* h=\([^;]*\).*/\1/p' direct.eml) % diff -s -u (egrep -i ^($h): direct.eml) (egrep -i ^($h) list.eml) Files /dev/fd/4 and /dev/fd/5 are identical % diff -s -u (sed '1,/^$/d' direct.eml) (sed '1,/^$/d' list.eml) Files /dev/fd/4 and /dev/fd/5 are identical The first diff compares only the headers in the DKIM Signature. The second diff compares the body. The DKIM checks seem to be good. So, it seems that nothing has changed in the content or checked header. It must be SPf. % dig +short TXT _spf.mail.yahoo.com v=spf1 ptr:yahoo.com ptr:yahoo.net ip4:206.108.40.0/27 ip4:199.16.139.0/26 ?all It seems that in the case of a simple Exchange distribution list, the Yahoo members will fail (into their Spam folder!), some others may fail depending upon their service's SPF fussiness, and others may have to root around in their Spam folders for the content. -- Gary Algier, WB2FWZg...@ulticom.com +1 856 787 2758 Ulticom Inc., 1020 Briggs Rd, Mt. Laurel, NJ 08054 Fax:+1 856 866 2033 Nielsen's First Law of Computer Manuals: People don't read documentation voluntarily. -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Executive summary of DMARC issues
On May 15, 2014, at 10:53 AM, Gary Algier g...@ulticom.com wrote: On 05/14/14 23:47, Stephen J. Turnbull wrote: I then sent an email to the list and to my work sendmail address. It was delivered to both work addresses and the iCloud address. Gmail put it in my Spam folder with the warning: --- Be careful with this message. Our systems couldn't verify that this message was really sent by yahoo.com. You might want to avoid clicking links or replying with personal information. --- There is also a link to their Why messages are marked as Spam page. On Yahoo I found the bounce in my Spam folder with the following: --- This is an automated message from the Extensible Content Security at host wg.ulticom.com. The message returned below could not be delivered to its intended destinations. It seems that in the case of a simple Exchange distribution list, the Yahoo members will fail (into their Spam folder!), some others may fail depending upon their service's SPF fussiness, and others may have to root around in their Spam folders for the content. On the lists that I manage on listserv I’ve discovered that many ISPs honor Yahoo and AOL’s p=reject, and will not even put the message in the spam folder. Among them are: Comcast, SBCGlobal, ATT, AOL, Rogers, Earthlink, Hotmail and a few others I don’t recall. So essentially half of my list members will not get posts from Yahoo or AOL. best regards, Larry -- Larry Finch finc...@portadmiral.org -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Executive summary of DMARC issues
On 05/15/14 11:15, Larry Finch wrote: On May 15, 2014, at 10:53 AM, Gary Algier g...@ulticom.com wrote: On 05/14/14 23:47, Stephen J. Turnbull wrote: I then sent an email to the list and to my work sendmail address. It was delivered to both work addresses and the iCloud address. Gmail put it in my Spam folder with the warning: --- Be careful with this message. Our systems couldn't verify that this message was really sent by yahoo.com. You might want to avoid clicking links or replying with personal information. --- There is also a link to their Why messages are marked as Spam page. On Yahoo I found the bounce in my Spam folder with the following: --- This is an automated message from the Extensible Content Security at host wg.ulticom.com. The message returned below could not be delivered to its intended destinations. It seems that in the case of a simple Exchange distribution list, the Yahoo members will fail (into their Spam folder!), some others may fail depending upon their service's SPF fussiness, and others may have to root around in their Spam folders for the content. On the lists that I manage on listserv I’ve discovered that many ISPs honor Yahoo and AOL’s p=reject, and will not even put the message in the spam folder. Among them are: Comcast, SBCGlobal, ATT, AOL, Rogers, Earthlink, Hotmail and a few others I don’t recall. So essentially half of my list members will not get posts from Yahoo or AOL. best regards, Larry -- Larry Finch finc...@portadmiral.org Apparently, simple Exchange distribution lists do not rewrite headers or touch the body so DKIM passes. However, the distribution lists also do not change the envelope sender so the SPF fails. In order to get through DKIM, the internal author address (From: ) and a bunch of other headers must stay the same, which Exchange does. Most mailing list software rewrites something, which makes DKIM fail. However, the mailing list software will use an envelope address from the list so SPF should not fail. Summary: Can't use Exchange distribution lists: SPF will fail. Can't use mailing list software without changing the author, etc.: DKIM will fail. Time for sendmail aliases? Or perhaps, SPF will fail? -- Gary Algier, WB2FWZg...@ulticom.com +1 856 787 2758 Ulticom Inc., 1020 Briggs Rd, Mt. Laurel, NJ 08054 Fax:+1 856 866 2033 Nielsen's First Law of Computer Manuals: People don't read documentation voluntarily. -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Executive summary of DMARC issues
On May 14, 2014, at 22:47 PM, Stephen J. Turnbull step...@xemacs.orgmailto:step...@xemacs.org wrote: If distribution lists are pure forwards, MS365 will be OK. But I find it hard to believe that that level of functionality is popular with users -- there's a reason why all popular MLMs implement subject prefixes, body headers and body footers, and it isn't because it's the Microsoft way. Exchange Online (standalone or as part of Office 365) really does lack most of the basic mailing list functionality. It does have moderation, but no subject tagging, list footer, etc. I believe the main reason distribution groups get used is because of the integration with Outlook and compatibility with Exchange Calendaring (inviting a Mailman list to a meeting is bad). It is limited, but in many cases is “good enough”, and many people don’t know what they’re missing. Here’s a table comparing select features of the two: [cid:2EBFB349-BF05-4E11-BED3-DCE3885C4D60@hdfgroup.uiuc.edu] -- Matthew Needham mneed...@hdfgroup.orgmailto:mneed...@hdfgroup.org 217-531-6110 The HDF Group 1800 South Oak Street, Suite 203 Champaign, IL 61820 -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Executive summary of DMARC issues
On 05/15/2014 08:35 AM, Gary Algier wrote: However, the mailing list software will use an envelope address from the list so SPF should not fail. SPF won't fail, but for DMARC purposes, the domain of the Envelope sender that passes SPF will not align with the From: domain, so the fact that SPF is OK doesn't help. -- Mark Sapiro m...@msapiro.netThe highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
[Mailman-Users] Executive summary of DMARC issues
Hello, I have been following the discussion of the DMARC issues and Mailman's attempts to live with it. I was wondering if anyone has an Executive Summary of the DMARC issue in a general sense. The information on the wiki talks about the impact on Mailman, but I need a generic explanation that can be presented to our CFO. Our management wants to move our email from an in-house Exchange environment (with Mailman on the side for mailing lists) to a totally MS365 solution. We have been told that everything we do with Mailman can be done with Exchange distribution lists, etc. I know all the reasons this is really a poor statement, but distribution lists have the same DMARC problem. I created a test distribution list here. I created local contacts that forward to my personal gmail.com and icloud.com addresses. I added these and my work address to the list. Email from gmail and icloud works fine, however the author address (From:) carries the original address. When I sent an email from my yahoo.com account, it was not delivered to gmail. I never saw a bounce, though, so I don't know who, if anyone, gets notified. I am hoping that I can make this issue plain at an executive level so we can get them to stay with a Mailman solution as we go to the cloud. Of course if someone says that the current MS365 implementation has addressed this, then that's a different (unfortunate) story. -- Gary Algier, WB2FWZg...@ulticom.com +1 856 787 2758 Ulticom Inc., 1020 Briggs Rd, Mt. Laurel, NJ 08054 Fax:+1 856 866 2033 Nielsen's First Law of Computer Manuals: People don't read documentation voluntarily. -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
[Mailman-Users] Executive summary of DMARC issues
Gary Algier writes: I have been following the discussion of the DMARC issues and Mailman's attempts to live with it. I was wondering if anyone has an Executive Summary of the DMARC issue in a general sense. How about the following: DMARC is a set of protocols for Internet mail that are used to by participating hosts to exchange information about mail traffic. This information is primarily useful for fighting email fraud, including spam and phishing. The basic procedure is that recipient hosts which participate in the protocol check whether the apparent sending domain (the domain that appears in the address in From header field) participates, using a query to the DNS defined by the DMARC protocol. If so, the recipient host will send to that domain some information about mail which purports to be from the sending host, but fails to validate. The validation refers specifically to, and only to, the domain in the From header field. The DMARC protocol itself does not validate the mailbox (user), although some originating hosts may do so. Nor can it guarantee that the user's account hasn't been used fraudulently. For participants in the DMARC protocol, valid mail provides a strong guarantee that the domain of the address in the From header field is authentic, with some strong restrictions required to ensure success of validation. These restrictions include: (1) When authentic mail is delivered directly from the sending host to the recipient, validation will succeed. (2) When authentic mail is forwarded verbatim by all intervening Internet hosts (including mailing lists and user mailbox forwarding), validation will succeed. Authentic mail may fail to validate in many common circumstances, where a forwarding host alters certain headers or any part of the body of the message. This may occur for forwarding hosts which add advertisements to the footer or the like, or if a forwarding host reformats the body (it shouldn't do that, but some do). There are also restrictions on some parts of the mail. For example, the From header field must contain exactly one address (multiple author addresses are permitted by the mail standards and are occasionally useful, though rarely seen nowadays). Failure to validate is also frequent on mailing lists. Common transformations that *will* cause mail distributed by a mailing list to fail validation include (but are not limited to) - Adding a list tag or sequence number to the Subject header field. - Adding a footer, typically containing unsubscribe information for public discussion lists or legal disclaimers for corporate lists. - Removal of banned content, such as executable file attachments or excessively large attachments. Because many common practices with Internet mail can cause authentic mail to fail DMARC validation, although mail which successfully validates may be trusted to be from the domain in the address in the From field, failure to validate in general *does not* imply lack of authenticity. Under some circumstances, a sending domain may be willing to severely restrict its users' use of their addresses. For example, they may not be allowed to post to mailing lists, and recipients may be advised to *always* keep their addresses updated to the domain where they actually read their mail. This is common practice with banks and other financial institutions. Such organizations can be reasonably sure that mail which fails to validate is fraudulent (although the intent may be merely mischievious rather than truly felonious). They may take advantage of a further feature of the DMARC protocol, the policy, which is the action that the sending domain suggests that recipients take with mail that appears to be from that domain but fails validation. The normal policy is none, that is, the recipient should deliver the mail as though the apparent sending domain did not participate in DMARC. (Of course, if the mail fails spam or virus checks, it would be quarantined or discarded as usual.) The second level is quarantine, which means that the mail should not be delivered directly to the user's mailbox in the ordinary way, for fear of fraudulent use of the sending address (typically phishing). The third level is reject, which means that the sender recommends that the mail not be delivered at all, to completely prevent fraudulent misuse of their domain. Unfortunately, unless the sending domain has truly draconian policies about the use of addresses in that domain, use of quarantine or reject makes it virtually certain that some authentic mail will not be read in a timely fashion or (in the case of reject) never delivered at all. The reject policy may also result in denial of services to third parties (one common case is having delivery disabled from a mailing list, or even being unsubscribed), due to rejected mail being bounced back to the forwarding host. At present we know of *no* hosts which distinguish DMARC bounces from other
Re: [Mailman-Users] Executive summary of DMARC issues
On 5/14/2014 2:49 PM, Stephen J. Turnbull wrote: Gary Algier writes: I have been following the discussion of the DMARC issues and Mailman's attempts to live with it. I was wondering if anyone has an Executive Summary of the DMARC issue in a general sense. How about the following: DMARC is a set of protocols for Internet mail that are used to by participating hosts to exchange information about mail traffic. This information is primarily useful for fighting email fraud, including spam and phishing. The basic procedure is that recipient hosts which participate in the protocol check whether the apparent sending domain (the domain that appears in the address in From header field) participates, using a query to the DNS defined by the DMARC protocol. If so, the recipient host will send to that domain some information about mail which purports to be from the sending host, but fails to validate. The validation refers specifically to, and only to, the domain in the From header field. The DMARC protocol itself does not validate the mailbox (user), although some originating hosts may do so. Nor can it guarantee that the user's account hasn't been used fraudulently. For participants in the DMARC protocol, valid mail provides a strong guarantee that the domain of the address in the From header field is authentic, with some strong restrictions required to ensure success of validation. These restrictions include: (1) When authentic mail is delivered directly from the sending host to the recipient, validation will succeed. (2) When authentic mail is forwarded verbatim by all intervening Internet hosts (including mailing lists and user mailbox forwarding), validation will succeed. Authentic mail may fail to validate in many common circumstances, where a forwarding host alters certain headers or any part of the body of the message. This may occur for forwarding hosts which add advertisements to the footer or the like, or if a forwarding host reformats the body (it shouldn't do that, but some do). There are also restrictions on some parts of the mail. For example, the From header field must contain exactly one address (multiple author addresses are permitted by the mail standards and are occasionally useful, though rarely seen nowadays). Failure to validate is also frequent on mailing lists. Common transformations that *will* cause mail distributed by a mailing list to fail validation include (but are not limited to) - Adding a list tag or sequence number to the Subject header field. - Adding a footer, typically containing unsubscribe information for public discussion lists or legal disclaimers for corporate lists. - Removal of banned content, such as executable file attachments or excessively large attachments. Because many common practices with Internet mail can cause authentic mail to fail DMARC validation, although mail which successfully validates may be trusted to be from the domain in the address in the From field, failure to validate in general *does not* imply lack of authenticity. Under some circumstances, a sending domain may be willing to severely restrict its users' use of their addresses. For example, they may not be allowed to post to mailing lists, and recipients may be advised to *always* keep their addresses updated to the domain where they actually read their mail. This is common practice with banks and other financial institutions. Such organizations can be reasonably sure that mail which fails to validate is fraudulent (although the intent may be merely mischievious rather than truly felonious). They may take advantage of a further feature of the DMARC protocol, the policy, which is the action that the sending domain suggests that recipients take with mail that appears to be from that domain but fails validation. The normal policy is none, that is, the recipient should deliver the mail as though the apparent sending domain did not participate in DMARC. (Of course, if the mail fails spam or virus checks, it would be quarantined or discarded as usual.) The second level is quarantine, which means that the mail should not be delivered directly to the user's mailbox in the ordinary way, for fear of fraudulent use of the sending address (typically phishing). The third level is reject, which means that the sender recommends that the mail not be delivered at all, to completely prevent fraudulent misuse of their domain. Unfortunately, unless the sending domain has truly draconian policies about the use of addresses in that domain, use of quarantine or reject makes it virtually certain that some authentic mail will not be read in a timely fashion or (in the case of reject) never delivered at all. The reject policy may also result in denial of services to third parties (one common case is having delivery disabled from a mailing list, or even being unsubscribed), due to rejected mail being bounced back to the forwarding host. At present we know of
Re: [Mailman-Users] Executive summary of DMARC issues
On May 14, 2014, at 4:24 PM, Barry S. Finkel bsfin...@att.net wrote: Is this also true? Users from DMARC-reject domains send mail to mailing lists, and the resulting mail from the mailing list is rejected. Enough rejections can cause the mailing list possibly to be blacklisted for sending lots of spam mail. I would expect it to be true, however I cannot confirm as I have set all AOL and Yahoo accounts to moderated so the problem doesn’t arise. We then resend the message as a forward from one of the moderators. And suggest (strongly) that the user switch from AOL or Yahoo to a list-friendly email provider. best regards, Larry Note: We do not currently use mailman. We use listserv, but are considering switching. -- Larry Finch finc...@portadmiral.org -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Executive summary of DMARC issues
At Wed, 14 May 2014 15:24:32 -0500 Barry S. Finkel bsfin...@att.net wrote: On 5/14/2014 2:49 PM, Stephen J. Turnbull wrote: Gary Algier writes: I have been following the discussion of the DMARC issues and Mailman's attempts to live with it. I was wondering if anyone has an Executive Summary of the DMARC issue in a general sense. How about the following: DMARC is a set of protocols for Internet mail that are used to by participating hosts to exchange information about mail traffic. This information is primarily useful for fighting email fraud, including spam and phishing. The basic procedure is that recipient hosts which participate in the protocol check whether the apparent sending domain (the domain that appears in the address in From header field) participates, using a query to the DNS defined by the DMARC protocol. If so, the recipient host will send to that domain some information about mail which purports to be from the sending host, but fails to validate. The validation refers specifically to, and only to, the domain in the From header field. The DMARC protocol itself does not validate the mailbox (user), although some originating hosts may do so. Nor can it guarantee that the user's account hasn't been used fraudulently. For participants in the DMARC protocol, valid mail provides a strong guarantee that the domain of the address in the From header field is authentic, with some strong restrictions required to ensure success of validation. These restrictions include: (1) When authentic mail is delivered directly from the sending host to the recipient, validation will succeed. (2) When authentic mail is forwarded verbatim by all intervening Internet hosts (including mailing lists and user mailbox forwarding), validation will succeed. Authentic mail may fail to validate in many common circumstances, where a forwarding host alters certain headers or any part of the body of the message. This may occur for forwarding hosts which add advertisements to the footer or the like, or if a forwarding host reformats the body (it shouldn't do that, but some do). There are also restrictions on some parts of the mail. For example, the From header field must contain exactly one address (multiple author addresses are permitted by the mail standards and are occasionally useful, though rarely seen nowadays). Failure to validate is also frequent on mailing lists. Common transformations that *will* cause mail distributed by a mailing list to fail validation include (but are not limited to) - Adding a list tag or sequence number to the Subject header field. - Adding a footer, typically containing unsubscribe information for public discussion lists or legal disclaimers for corporate lists. - Removal of banned content, such as executable file attachments or excessively large attachments. Because many common practices with Internet mail can cause authentic mail to fail DMARC validation, although mail which successfully validates may be trusted to be from the domain in the address in the From field, failure to validate in general *does not* imply lack of authenticity. Under some circumstances, a sending domain may be willing to severely restrict its users' use of their addresses. For example, they may not be allowed to post to mailing lists, and recipients may be advised to *always* keep their addresses updated to the domain where they actually read their mail. This is common practice with banks and other financial institutions. Such organizations can be reasonably sure that mail which fails to validate is fraudulent (although the intent may be merely mischievious rather than truly felonious). They may take advantage of a further feature of the DMARC protocol, the policy, which is the action that the sending domain suggests that recipients take with mail that appears to be from that domain but fails validation. The normal policy is none, that is, the recipient should deliver the mail as though the apparent sending domain did not participate in DMARC. (Of course, if the mail fails spam or virus checks, it would be quarantined or discarded as usual.) The second level is quarantine, which means that the mail should not be delivered directly to the user's mailbox in the ordinary way, for fear of fraudulent use of the sending address (typically phishing). The third level is reject, which means that the sender recommends that the mail not be delivered at all, to completely prevent fraudulent misuse of their domain. Unfortunately, unless the sending domain has truly draconian policies about the use of addresses in that domain, use of quarantine or reject makes it virtually certain that some authentic mail will not be read in a timely fashion or (in the case of reject) never delivered at all. The
Re: [Mailman-Users] Executive summary of DMARC issues
Gary Algier wrote: I created a test distribution list here. I created local contacts that forward to my personal gmail.com and icloud.com addresses. I added these and my work address to the list. Email from gmail and icloud works fine, however the author address (From:) carries the original address. When I sent an email from my yahoo.com account, it was not delivered to gmail. I never saw a bounce, though, so I don't know who, if anyone, gets notified. I am hoping that I can make this issue plain at an executive level so we can get them to stay with a Mailman solution as we go to the cloud. Of course if someone says that the current MS365 implementation has addressed this, then that's a different (unfortunate) story. When MS365 forwards the mails sent to the distribution list, should that make the DMARC authentication fail? I thought that only happened if you made changes like adding a prefix to the subject line like Mailman does. Peter Shute -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Executive summary of DMARC issues
On 05/14/2014 01:24 PM, Barry S. Finkel wrote: Is this also true? Users from DMARC-reject domains send mail to mailing lists, and the resulting mail from the mailing list is rejected. Enough rejections can cause the mailing list possibly to be blacklisted for sending lots of spam mail. We don't know, and can't know until Mailman servers start getting blacklisted for this reason. Actually, From: domains can request reports even if DMARC p=none. It is unclear what might be done with these reports, but given what some domains have done with DMARC already, I for one would not be surprised if this information was used to color the reputation of the sending server. Note that currently, Yahoo.com only requests aggregate reports which *I think* do not identify the sending server, but AOL.com requests failure reports as well which are intended to identify servers and actual senders. -- Mark Sapiro m...@msapiro.netThe highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Executive summary of DMARC issues
Barry S. Finkel writes: Is this also true? Users from DMARC-reject domains send mail to mailing lists, and the resulting mail from the mailing list is rejected. Enough rejections can cause the mailing list possibly to be blacklisted for sending lots of spam mail. The rejections themselves will be received by the mailing list (assuming RFC-conforming recipients), so I think this is unlikely in theory. I haven't heard of this happening, and Mark says he hasn't either. The failure notices received by DMARC senders are another matter. These are not supposed to be used for blacklisting, but as we already know, Yahoo! and AOL are desperate. OTOH, they're already known to be hostile to mailing lists; getting blacklisted by them is not news. I suppose we could worry that they would publish their blacklists to be used by others, but it was pointed out to me by Murray Kucherawy that MAPS was shut down by lawsuits, and it seems likely that a public whitelist or blacklist would attract them too. My estimate is that this is not something to worry about because it will be mitigated by any of the options already implemented, and it's not something to complain about because there's no evidence it actually happens. IMO, we need to avoid crying wolf; there are way too many people (including a number of Mailman list operators) with a vested interest in painting Yahoo! and AOL as well-intentioned but overwhelmed by circumstances beyond their control. Steve -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Executive summary of DMARC issues
Actually, From: domains can request reports even if DMARC p=none. It is unclear what might be done with these reports, but given what some domains have done with DMARC already, I for one would not be surprised if this information was used to color the reputation of the sending server. Note that currently, Yahoo.com only requests aggregate reports which *I think* do not identify the sending server, but AOL.com requests failure reports as well which are intended to identify servers and actual senders. I've been collecting DMARC aggregate reports for over two years and have over 40,000 of them. I use some scripts that decompress and parse the reports and put the interesting bits into a mysql database. I also have 22,000 failure reports (fewer providers send them.) The aggregate reports do indeed identify the sending server and are pretty interesting. For some of the larger mail systems, it's clear from the tags in the reports that they have a pretty good idea where the mailing lists are, which makes me wonder why they don't use that info to whitelist around the DMARC damage. I don't see any evidence that DMARC failures alone are likely to get a server blacklisted, although I wouldn't be surprised if it were a factor along with user complaints and spam filter statistics. R's, John PS: The scripts are at http://www.taugh.com/rddmarc/ if you want to play along on your own system. You can (and should) collect DMARC stats without publishing any DMARC policies. -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Executive summary of DMARC issues
Peter Shute writes: When MS365 forwards the mails sent to the distribution list, should that make the DMARC authentication fail? I thought that only happened if you made changes like adding a prefix to the subject line like Mailman does. If it forwards verbatim *and* the sending domain signs the mail with DKIM (the common case), DMARC validation will succeed. Without DKIM, DMARC validation is guaranteed to fail. However, even in the sender uses DKIM, *any* change *whatsoever* to the body will cause validation to fail, and there are several changes to the header that could cause it to fail. Furthermore, which parts of the header are protected by the DKIM signature are determined by the sender, not by DMARC AFAIK. If distribution lists are pure forwards, MS365 will be OK. But I find it hard to believe that that level of functionality is popular with users -- there's a reason why all popular MLMs implement subject prefixes, body headers and body footers, and it isn't because it's the Microsoft way. -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
