Re: Change to Mail Infrastructure - SPF and DKIM verification will now be enforced
On Wed, Dec 2, 2015 at 10:06 PM, Ben Cooksley wrote: > Hi all, > > Just a quick announcement to inform everyone that I have now adjusted > the KDE.org mail filter configuration to: > > a) Enforce SPF rules as specified by the domain > b) Temporarily reject any mail which does not successfully pass DKIM > validation where the domain specifies it is DKIM enabled. I've had to disable DKIM validation, as Debian's infrastructure does not oversign any email which transits through their mailing lists (they don't appear to have enabled DKIM at all, which is disappointing). This means all mails to Debian mailing lists which get modified (changes to body and subject at the very least, as well as any other header specified by the DKIM signer) aren't accepted by any system performing DKIM enforcement. Email which transits through KDE.org mailing lists is always oversigned to avoid this issue. People who subscribe to Debian mailing lists using kdemail.net or kde.org addresses are advised to file tickets with Debian's Infrastructure team requesting they correct this omission. I will be re-enabling DKIM validation in one week's time - which will then break subscriptions to Debian mailing lists (as any email from anyone who has enabled DKIM which hits their lists will not be accepted by KDE email infrastructure) > > Please let sysadmin know of any issues. > > Regards, > Ben Regards, Ben
Re: Change to Mail Infrastructure - SPF and DKIM verification will now be enforced
On Thursday, 3 December 2015 07:13:07 CET, Ben Cooksley wrote: I will be re-enabling DKIM validation in one week's time - which will then break subscriptions to Debian mailing lists (as any email from anyone who has enabled DKIM which hits their lists will not be accepted by KDE email infrastructure) Ben, could you please briefly explain your idea about how a complying mailing list service should behave? Suppose that I have an installation of mlmmj which: - mangles the Subject header, - preserves the original From header, - maybe replaces a Reply-To with the ML's address, - introduces a bunch of specific List-* headers, - otherwsie doesn't manipulate the MIME tree or the message texts. What should I do to make sure that this service continues working once you flip the switch? I would like to have more information about what you mean by "DKIM validation" -- what specific steps are you going to introduce, and how is the end result going to react to a missing or invalid DKIM signatures. Also, quoting RFC 6376, section 6.3: In general, modules that consume DKIM verification output SHOULD NOT determine message acceptability based solely on a lack of any signature or on an unverifiable signature; such rejection would cause severe interoperability problems. If an MTA does wish to reject such messages during an SMTP session (for example, when communicating with a peer who, by prior agreement, agrees to only send signed messages), and a signature is missing or does not verify, the handling MTA SHOULD use a 550/5.7.x reply code. That seems in line with what e.g. GMail is doing, only enforcing DKIM validation for notoriously faked domains like eBay and PayPal where the phishing potential is high. With kind regards, Jan -- Trojitá, a fast Qt IMAP e-mail client -- http://trojita.flaska.net/
Re: Change to Mail Infrastructure - SPF and DKIM verification will now be enforced
Am Donnerstag, 3. Dezember 2015, 11:54:43 schrieb Jan Kundrát: > On Thursday, 3 December 2015 07:13:07 CET, Ben Cooksley wrote: > > I will be re-enabling DKIM validation in one week's time - which will > > then break subscriptions to Debian mailing lists (as any email from > > anyone who has enabled DKIM which hits their lists will not be > > accepted by KDE email infrastructure) > > Ben, could you please briefly explain your idea about how a complying > mailing list service should behave? Suppose that I have an installation of > mlmmj which: > > - mangles the Subject header, > - preserves the original From header, > - maybe replaces a Reply-To with the ML's address, > - introduces a bunch of specific List-* headers, > - otherwsie doesn't manipulate the MIME tree or the message texts. > > What should I do to make sure that this service continues working once you > flip the switch? This is actually a flaw in those standards: they think that everyone you communicate with will comply. If you as a mailing list service do not then everyone that sends to your list will eventually get in trouble if you do not rewrite the message or resigns it. Think of SPF: I sent an email to a kde.org email address only some weeks ago. My domain sets a SPF policy. The KDE server accepts this (it's actually correct), and then sends the mail on (unaltered). Now the next server also checks SPF and will reject the mail, because the KDE server is not allowed to send mail for my domain. Now you have 2 ways out: either the KDE server rewrites the "mail from" header (what you will later find as Return-Path in the mail header), or the final destination says allows the user to say "hey, I use those kde.org server as a forwarder to me, so whatever SPF says, mails from that host are fine". Both ways work, both are fine, but both require some sort of action somewhere on the path. That part is simple. For DKIM stuff get's more complicated because you sometimes _have_ to modify the body, e.g. when you need to base64- or qp- recode parts of the mail because the receiving mail server does not support 8bit-transfer (which is an issue by itself, but still sadly legal). So with DKIM you are actually screwed at this point. The only good way it is again to permit your users to ignore DKIM signatures from certain hosts (e.g. if you subscribe to a Debian list, then simply ignore DKIM for the Debian servers). Finding out those itself is not an easy task either. So all in all one can enable DKIM for list services, but for user accounts it should be opt-in with an easy way to whitelist certain hosts for relaying. Everything else is just asking for endless bounces. > I would like to have more information about what you mean by "DKIM > validation" -- what specific steps are you going to introduce, and how is > the end result going to react to a missing or invalid DKIM signatures. > > Also, quoting RFC 6376, section 6.3: > >In general, modules that consume DKIM verification output SHOULD NOT >determine message acceptability based solely on a lack of any >signature or on an unverifiable signature; such rejection would cause >severe interoperability problems. If an MTA does wish to reject such >messages during an SMTP session (for example, when communicating with >a peer who, by prior agreement, agrees to only send signed messages), >and a signature is missing or does not verify, the handling MTA >SHOULD use a 550/5.7.x reply code. > > That seems in line with what e.g. GMail is doing, only enforcing DKIM > validation for notoriously faked domains like eBay and PayPal where the > phishing potential is high. No, this means that a mail should not be rejected if there is _no_ signature, or a malformed one. If a domain publishs that it does DKIM signing and mails are expected to be correctly signed, then rejecting on a invalid signature is actually fine (and the sole purpose of this RfC). Greetings, Eike signature.asc Description: This is a digitally signed message part.
Re: Change to Mail Infrastructure - SPF and DKIM verification will now be enforced
On Thu, Dec 3, 2015 at 11:54 PM, Jan Kundrát wrote: > On Thursday, 3 December 2015 07:13:07 CET, Ben Cooksley wrote: >> >> I will be re-enabling DKIM validation in one week's time - which will >> then break subscriptions to Debian mailing lists (as any email from >> anyone who has enabled DKIM which hits their lists will not be >> accepted by KDE email infrastructure) > > > Ben, could you please briefly explain your idea about how a complying > mailing list service should behave? Suppose that I have an installation of > mlmmj which: > > - mangles the Subject header, > - preserves the original From header, > - maybe replaces a Reply-To with the ML's address, > - introduces a bunch of specific List-* headers, > - otherwsie doesn't manipulate the MIME tree or the message texts. > > What should I do to make sure that this service continues working once you > flip the switch? > > I would like to have more information about what you mean by "DKIM > validation" -- what specific steps are you going to introduce, and how is > the end result going to react to a missing or invalid DKIM signatures. > > Also, quoting RFC 6376, section 6.3: > > In general, modules that consume DKIM verification output SHOULD NOT > determine message acceptability based solely on a lack of any > signature or on an unverifiable signature; such rejection would cause > severe interoperability problems. If an MTA does wish to reject such > messages during an SMTP session (for example, when communicating with > a peer who, by prior agreement, agrees to only send signed messages), > and a signature is missing or does not verify, the handling MTA > SHOULD use a 550/5.7.x reply code. > > That seems in line with what e.g. GMail is doing, only enforcing DKIM > validation for notoriously faked domains like eBay and PayPal where the > phishing potential is high. To be specific I will be enabling the following line: On-BadSignature tempfail within the configuration of OpenDKIM on our servers. This means you have two choices in regards to your mailing lists: 1) Resign the emails with your own signature - which will make them validate (this is what our infrastructure presently does) 2) Cease tampering with emails. In particular the body and subject will always be signed and thus shouldn't be modified. Other headers will depend on the mail sender, most are fairly sensible and thus compatible with mailing lists such as Mailman. Note that in the long run with DMARC looming you will need to switch to #2 anyway, and keeping your current behaviour will likely lead to mail from people who use Yahoo / AOL / etc ending up in the spam folder with many mailing list members. I'll be starting a discussion regarding taking this step on KDE systems at some point in the near future (switching to DMARC compatible policies). For more information, please see http://wiki.list.org/DEV/DMARC > > With kind regards, > Jan Regards, Ben > > -- > Trojitá, a fast Qt IMAP e-mail client -- http://trojita.flaska.net/
Re: Change to Mail Infrastructure - SPF and DKIM verification will now be enforced
On Fri, Dec 4, 2015 at 9:01 AM, Rolf Eike Beer wrote: > Am Donnerstag, 3. Dezember 2015, 11:54:43 schrieb Jan Kundrát: >> On Thursday, 3 December 2015 07:13:07 CET, Ben Cooksley wrote: >> > I will be re-enabling DKIM validation in one week's time - which will >> > then break subscriptions to Debian mailing lists (as any email from >> > anyone who has enabled DKIM which hits their lists will not be >> > accepted by KDE email infrastructure) >> >> Ben, could you please briefly explain your idea about how a complying >> mailing list service should behave? Suppose that I have an installation of >> mlmmj which: >> >> - mangles the Subject header, >> - preserves the original From header, >> - maybe replaces a Reply-To with the ML's address, >> - introduces a bunch of specific List-* headers, >> - otherwsie doesn't manipulate the MIME tree or the message texts. >> >> What should I do to make sure that this service continues working once you >> flip the switch? > > This is actually a flaw in those standards: they think that everyone you > communicate with will comply. If you as a mailing list service do not then > everyone that sends to your list will eventually get in trouble if you do not > rewrite the message or resigns it. > > Think of SPF: I sent an email to a kde.org email address only some weeks ago. > My domain sets a SPF policy. The KDE server accepts this (it's actually > correct), and then sends the mail on (unaltered). Now the next server also > checks SPF and will reject the mail, because the KDE server is not allowed to > send mail for my domain. Now you have 2 ways out: either the KDE server > rewrites the "mail from" header (what you will later find as Return-Path in > the > mail header), or the final destination says allows the user to say "hey, I use > those kde.org server as a forwarder to me, so whatever SPF says, mails from > that host are fine". Both ways work, both are fine, but both require some sort > of action somewhere on the path. Rewriting to workaround SPF restriction is also standardised - as a mechanism known as SRS - see http://www.openspf.org/SRS > > That part is simple. For DKIM stuff get's more complicated because you > sometimes _have_ to modify the body, e.g. when you need to base64- or qp- > recode parts of the mail because the receiving mail server does not support > 8bit-transfer (which is an issue by itself, but still sadly legal). So with > DKIM you are actually screwed at this point. The only good way it is again to > permit your users to ignore DKIM signatures from certain hosts (e.g. if you > subscribe to a Debian list, then simply ignore DKIM for the Debian servers). > Finding out those itself is not an easy task either. > > So all in all one can enable DKIM for list services, but for user accounts it > should be opt-in with an easy way to whitelist certain hosts for relaying. > Everything else is just asking for endless bounces. Note that DKIM senders and receivers are usually running on modern infrastructures, so 8bit transfer shouldn't be an issue. For user to user transmission, there is no reason why mail bodies would be modified. As for user to list to user transmission, please see http://wiki.list.org/DEV/DMARC for ways in which DKIM validity can be retained or ensured with lists. Whitelisting shouldn't be required... Note that DMARC permits enforcement of the presence of DKIM signatures. > >> I would like to have more information about what you mean by "DKIM >> validation" -- what specific steps are you going to introduce, and how is >> the end result going to react to a missing or invalid DKIM signatures. >> >> Also, quoting RFC 6376, section 6.3: >> >>In general, modules that consume DKIM verification output SHOULD NOT >>determine message acceptability based solely on a lack of any >>signature or on an unverifiable signature; such rejection would cause >>severe interoperability problems. If an MTA does wish to reject such >>messages during an SMTP session (for example, when communicating with >>a peer who, by prior agreement, agrees to only send signed messages), >>and a signature is missing or does not verify, the handling MTA >>SHOULD use a 550/5.7.x reply code. >> >> That seems in line with what e.g. GMail is doing, only enforcing DKIM >> validation for notoriously faked domains like eBay and PayPal where the >> phishing potential is high. > > No, this means that a mail should not be rejected if there is _no_ signature, > or a malformed one. If a domain publishs that it does DKIM signing and mails > are expected to be correctly signed, then rejecting on a invalid signature is > actually fine (and the sole purpose of this RfC). > > Greetings, > > Eike Cheers, Ben
Re: Change to Mail Infrastructure - SPF and DKIM verification will now be enforced
On Friday, 4 December 2015 10:56:42 CET, Ben Cooksley wrote: Note that in the long run with DMARC looming you will need to switch to #2 anyway, and keeping your current behaviour will likely lead to mail from people who use Yahoo / AOL / etc ending up in the spam folder with many mailing list members. I'll be starting a discussion regarding taking this step on KDE systems at some point in the near future (switching to DMARC compatible policies). For more information, please see http://wiki.list.org/DEV/DMARC Do I understand your plan correctly? The following projects appear to not re-sign their ML traffic, and they mangle headers at the same time. If I understand your plan correctly, this means that I won't be able to use my @kde.org addresses on mailing lists of these projects, for example: - Qt, - Debian, - Gentoo, - OpenStack, - anything hosted at SourceForge, - and many, many more, essentially anybody who were ignoring DKIM. Please, change your plans, this is obviously a huge no-go. Cheers, Jan -- Trojitá, a fast Qt IMAP e-mail client -- http://trojita.flaska.net/
Re: Change to Mail Infrastructure - SPF and DKIM verification will now be enforced
Am 04.12.2015 11:08, schrieb Ben Cooksley: On Fri, Dec 4, 2015 at 9:01 AM, Rolf Eike Beer wrote: Think of SPF: I sent an email to a kde.org email address only some weeks ago. My domain sets a SPF policy. The KDE server accepts this (it's actually correct), and then sends the mail on (unaltered). Now the next server also checks SPF and will reject the mail, because the KDE server is not allowed to send mail for my domain. Now you have 2 ways out: either the KDE server rewrites the "mail from" header (what you will later find as Return-Path in the mail header), or the final destination says allows the user to say "hey, I use those kde.org server as a forwarder to me, so whatever SPF says, mails from that host are fine". Both ways work, both are fine, but both require some sort of action somewhere on the path. Rewriting to workaround SPF restriction is also standardised - as a mechanism known as SRS - see http://www.openspf.org/SRS Has KDE implemented this in the last few weeks? Before it was not. That part is simple. For DKIM stuff get's more complicated because you sometimes _have_ to modify the body, e.g. when you need to base64- or qp- recode parts of the mail because the receiving mail server does not support 8bit-transfer (which is an issue by itself, but still sadly legal). So with DKIM you are actually screwed at this point. The only good way it is again to permit your users to ignore DKIM signatures from certain hosts (e.g. if you subscribe to a Debian list, then simply ignore DKIM for the Debian servers). Finding out those itself is not an easy task either. So all in all one can enable DKIM for list services, but for user accounts it should be opt-in with an easy way to whitelist certain hosts for relaying. Everything else is just asking for endless bounces. Note that DKIM senders and receivers are usually running on modern infrastructures, so 8bit transfer shouldn't be an issue. For user to user transmission, there is no reason why mail bodies would be modified. Well, nice try. Out of 5 mail providers I checked 3 failed: AOL, GMX.de, Web.de. Greetings, Eike
Re: Change to Mail Infrastructure - SPF and DKIM verification will now be enforced
On Fri, Dec 4, 2015 at 11:38 PM, Rolf Eike Beer wrote: > Am 04.12.2015 11:08, schrieb Ben Cooksley: >> >> On Fri, Dec 4, 2015 at 9:01 AM, Rolf Eike Beer >> wrote: > > >>> Think of SPF: I sent an email to a kde.org email address only some weeks >>> ago. >>> My domain sets a SPF policy. The KDE server accepts this (it's actually >>> correct), and then sends the mail on (unaltered). Now the next server >>> also >>> checks SPF and will reject the mail, because the KDE server is not >>> allowed to >>> send mail for my domain. Now you have 2 ways out: either the KDE server >>> rewrites the "mail from" header (what you will later find as Return-Path >>> in the >>> mail header), or the final destination says allows the user to say "hey, >>> I use >>> those kde.org server as a forwarder to me, so whatever SPF says, mails >>> from >>> that host are fine". Both ways work, both are fine, but both require some >>> sort >>> of action somewhere on the path. >> >> >> Rewriting to workaround SPF restriction is also standardised - as a >> mechanism known as SRS - see http://www.openspf.org/SRS > > > Has KDE implemented this in the last few weeks? Before it was not. We have not implemented SRS at this time, as it has not been necessary thus far. > >>> That part is simple. For DKIM stuff get's more complicated because you >>> sometimes _have_ to modify the body, e.g. when you need to base64- or qp- >>> recode parts of the mail because the receiving mail server does not >>> support >>> 8bit-transfer (which is an issue by itself, but still sadly legal). So >>> with >>> DKIM you are actually screwed at this point. The only good way it is >>> again to >>> permit your users to ignore DKIM signatures from certain hosts (e.g. if >>> you >>> subscribe to a Debian list, then simply ignore DKIM for the Debian >>> servers). >>> Finding out those itself is not an easy task either. >>> >>> So all in all one can enable DKIM for list services, but for user >>> accounts it >>> should be opt-in with an easy way to whitelist certain hosts for >>> relaying. >>> Everything else is just asking for endless bounces. >> >> >> Note that DKIM senders and receivers are usually running on modern >> infrastructures, so 8bit transfer shouldn't be an issue. >> For user to user transmission, there is no reason why mail bodies >> would be modified. > > > Well, nice try. Out of 5 mail providers I checked 3 failed: AOL, GMX.de, > Web.de. That doesn't surprise me. Somehow it must work fine for AOL however, as otherwise they would have issues corresponding with Yahoo users. > > Greetings, > > Eike Regards, Ben
Re: Change to Mail Infrastructure - SPF and DKIM verification will now be enforced
On Fri, Dec 4, 2015 at 11:28 PM, Jan Kundrát wrote: > On Friday, 4 December 2015 10:56:42 CET, Ben Cooksley wrote: >> >> Note that in the long run with DMARC looming you will need to switch >> to #2 anyway, and keeping your current behaviour will likely lead to >> mail from people who use Yahoo / AOL / etc ending up in the spam >> folder with many mailing list members. I'll be starting a discussion >> regarding taking this step on KDE systems at some point in the near >> future (switching to DMARC compatible policies). >> >> For more information, please see http://wiki.list.org/DEV/DMARC > > > Do I understand your plan correctly? The following projects appear to not > re-sign their ML traffic, and they mangle headers at the same time. If I > understand your plan correctly, this means that I won't be able to use my > @kde.org addresses on mailing lists of these projects, for example: > > - Qt, > - Debian, > - Gentoo, > - OpenStack, > - anything hosted at SourceForge, > - and many, many more, essentially anybody who were ignoring DKIM. > > Please, change your plans, this is obviously a huge no-go. *Sigh*. Debian has already committed (prior to any of this) to making their mailing lists DMARC compliant by ceasing the alteration of mail passing through their lists. It is an extreme pity these mailing list providers have demonstrated such an extreme disregard for standards which aim to eliminate forgeries and ensure people cannot be digitally misrepresented. This is why we had to change Bugzilla a while back to send as bugzilla_nore...@kde.org instead of the acting user's email address - because Yahoo's enforcement policy meant GMail always placed mail from Yahoo users in the spam folder. I'll grant an extension until 31 January, however I would like to see commitments from some of these to improve their infrastructure. > > Cheers, > Jan Regards, Ben > > > -- > Trojitá, a fast Qt IMAP e-mail client -- http://trojita.flaska.net/
Re: Change to Mail Infrastructure - SPF and DKIM verification will now be enforced
Ben Cooksley wrote: > On Fri, Dec 4, 2015 at 11:28 PM, Jan Kundrát wrote: > > On Friday, 4 December 2015 10:56:42 CET, Ben Cooksley wrote: > >> Note that in the long run with DMARC looming you will need to switch > >> to #2 anyway, and keeping your current behaviour will likely lead to > >> mail from people who use Yahoo / AOL / etc ending up in the spam > >> folder with many mailing list members. I'll be starting a discussion > >> regarding taking this step on KDE systems at some point in the near > >> future (switching to DMARC compatible policies). > >> > >> For more information, please see http://wiki.list.org/DEV/DMARC > > > > Do I understand your plan correctly? The following projects appear to not > > re-sign their ML traffic, and they mangle headers at the same time. If I > > understand your plan correctly, this means that I won't be able to use my > > @kde.org addresses on mailing lists of these projects, for example: > > > > - Qt, > > - Debian, > > - Gentoo, > > - OpenStack, > > - anything hosted at SourceForge, > > - and many, many more, essentially anybody who were ignoring DKIM. > > > > Please, change your plans, this is obviously a huge no-go. > > *Sigh*. > > Debian has already committed (prior to any of this) to making their > mailing lists DMARC compliant by ceasing the alteration of mail > passing through their lists. Which is a good idea anyway, as far as you can influence it (see the 8bit problems from the other mail). > It is an extreme pity these mailing list providers have demonstrated > such an extreme disregard for standards which aim to eliminate > forgeries and ensure people cannot be digitally misrepresented. This > is why we had to change Bugzilla a while back to send as > bugzilla_nore...@kde.org instead of the acting user's email address - > because Yahoo's enforcement policy meant GMail always placed mail from > Yahoo users in the spam folder. Huh? Of course you _must_ send with a @kde.org address. My SPF policy forbids you to send mail for my domain. And now you want to enforce SPF, but don't properly handle it yourself? > I'll grant an extension until 31 January, however I would like to see > commitments from some of these to improve their infrastructure. It wont affect me, as I ignore the whole DKIM stuff both at the sending and receiving end, but this just going to cause a lot of unnecessary trouble I think. To make it clear: I receive tons of spam per day. It has become worse in the last month, as it seems that the usual filters do not work that good anymore. You as postmaster of such a public domain are likely receiving even more of that crap. But that proposal is going to cause collateral damage. Eike signature.asc Description: This is a digitally signed message part.
Re: Change to Mail Infrastructure - SPF and DKIM verification will now be enforced
2015-12-04 16:07 GMT-03:00 Rolf Eike Beer : > Ben Cooksley wrote: >> It is an extreme pity these mailing list providers have demonstrated >> such an extreme disregard for standards which aim to eliminate >> forgeries and ensure people cannot be digitally misrepresented. This >> is why we had to change Bugzilla a while back to send as >> bugzilla_nore...@kde.org instead of the acting user's email address - >> because Yahoo's enforcement policy meant GMail always placed mail from >> Yahoo users in the spam folder. > > Huh? Of course you _must_ send with a @kde.org address. My SPF policy forbids > you to send mail for my domain. And now you want to enforce SPF, but don't > properly handle it yourself? How would that work with mailing lists? I would receive your message with From: pointing at the mailing list address instead of your own? -- Nicolás
Re: Change to Mail Infrastructure - SPF and DKIM verification will now be enforced
On Sat, Dec 5, 2015 at 8:07 AM, Rolf Eike Beer wrote: > Ben Cooksley wrote: >> On Fri, Dec 4, 2015 at 11:28 PM, Jan Kundrát wrote: >> > On Friday, 4 December 2015 10:56:42 CET, Ben Cooksley wrote: >> >> Note that in the long run with DMARC looming you will need to switch >> >> to #2 anyway, and keeping your current behaviour will likely lead to >> >> mail from people who use Yahoo / AOL / etc ending up in the spam >> >> folder with many mailing list members. I'll be starting a discussion >> >> regarding taking this step on KDE systems at some point in the near >> >> future (switching to DMARC compatible policies). >> >> >> >> For more information, please see http://wiki.list.org/DEV/DMARC >> > >> > Do I understand your plan correctly? The following projects appear to not >> > re-sign their ML traffic, and they mangle headers at the same time. If I >> > understand your plan correctly, this means that I won't be able to use my >> > @kde.org addresses on mailing lists of these projects, for example: >> > >> > - Qt, >> > - Debian, >> > - Gentoo, >> > - OpenStack, >> > - anything hosted at SourceForge, >> > - and many, many more, essentially anybody who were ignoring DKIM. >> > >> > Please, change your plans, this is obviously a huge no-go. >> >> *Sigh*. >> >> Debian has already committed (prior to any of this) to making their >> mailing lists DMARC compliant by ceasing the alteration of mail >> passing through their lists. > > Which is a good idea anyway, as far as you can influence it (see the 8bit > problems from the other mail). > >> It is an extreme pity these mailing list providers have demonstrated >> such an extreme disregard for standards which aim to eliminate >> forgeries and ensure people cannot be digitally misrepresented. This >> is why we had to change Bugzilla a while back to send as >> bugzilla_nore...@kde.org instead of the acting user's email address - >> because Yahoo's enforcement policy meant GMail always placed mail from >> Yahoo users in the spam folder. > > Huh? Of course you _must_ send with a @kde.org address. My SPF policy forbids > you to send mail for my domain. And now you want to enforce SPF, but don't > properly handle it yourself? Clarification there: SMTP From was always bugzilla_nore...@kde.org RFC 2822 From (the header) was the originating user's address. This is SPF, but not DMARC compliant. We now send emails with both SMTP and RFC 2822 headers set to @kde.org addresses. > >> I'll grant an extension until 31 January, however I would like to see >> commitments from some of these to improve their infrastructure. > > It wont affect me, as I ignore the whole DKIM stuff both at the sending and > receiving end, but this just going to cause a lot of unnecessary trouble I > think. > > To make it clear: I receive tons of spam per day. It has become worse in the > last month, as it seems that the usual filters do not work that good anymore. > You as postmaster of such a public domain are likely receiving even more of > that crap. But that proposal is going to cause collateral damage. Yes, spam has worsened, despite our efforts to block it. Unfortunately it seems that Bayes isn't able to keep up with the changing variety of rubbish - not without quite a bit of time spent daily training and training it. > > Eike Regards, Ben
Re: Change to Mail Infrastructure - SPF and DKIM verification will now be enforced
On Sat, Dec 5, 2015 at 8:14 AM, Nicolás Alvarez wrote: > 2015-12-04 16:07 GMT-03:00 Rolf Eike Beer : >> Ben Cooksley wrote: >>> It is an extreme pity these mailing list providers have demonstrated >>> such an extreme disregard for standards which aim to eliminate >>> forgeries and ensure people cannot be digitally misrepresented. This >>> is why we had to change Bugzilla a while back to send as >>> bugzilla_nore...@kde.org instead of the acting user's email address - >>> because Yahoo's enforcement policy meant GMail always placed mail from >>> Yahoo users in the spam folder. >> >> Huh? Of course you _must_ send with a @kde.org address. My SPF policy forbids >> you to send mail for my domain. And now you want to enforce SPF, but don't >> properly handle it yourself? > > How would that work with mailing lists? I would receive your message > with From: pointing at the mailing list address instead of your own? There are two "From" fields for an email. One is passed in the SMTP (RFC 5321) conversation in the "MAIL FROM" component. This is what SPF validates and controls. This will finally be placed in the Return-Path header when it is delivered into the user's mailbox. The other, which is ultimately shown in the user client as the "From" field is just a header within the email. This is what DKIM protects and validates. Mailing Lists, when resending email will often use an address such as "listname-bounces@domain" in the SMTP conversation (in order to receive bounces, and remove users who can no longer receive mail the list sends out in line with proper practice). They do not (usually) alter the email header From. > > -- > Nicolás Regards, Ben
Re: Change to Mail Infrastructure - SPF and DKIM verification will now be enforced
Am Freitag, 4. Dezember 2015, 16:14:10 schrieb Nicolás Alvarez: > 2015-12-04 16:07 GMT-03:00 Rolf Eike Beer : > > Ben Cooksley wrote: > >> It is an extreme pity these mailing list providers have demonstrated > >> such an extreme disregard for standards which aim to eliminate > >> forgeries and ensure people cannot be digitally misrepresented. This > >> is why we had to change Bugzilla a while back to send as > >> bugzilla_nore...@kde.org instead of the acting user's email address - > >> because Yahoo's enforcement policy meant GMail always placed mail from > >> Yahoo users in the spam folder. > > > > Huh? Of course you _must_ send with a @kde.org address. My SPF policy > > forbids you to send mail for my domain. And now you want to enforce SPF, > > but don't properly handle it yourself? > > How would that work with mailing lists? I would receive your message > with From: pointing at the mailing list address instead of your own? There are 2 "From" addresses. One is the one in the message body, i.e. what your mail program shows as "From:". This must not be altered. The other one is the one used between the mail servers. This can change during mail delivery. The last version you can see in the "Return-Path" header of your mail. Looking at any mail delivered to you through this list shows: Return-Path: This address should be set to a list-specific one by every mailinglist. SPF is only one part of the problem, the other one is that a delivery failure from the mailing list will end up at the original poster, not the mailing list bot. Greetings, Eike signature.asc Description: This is a digitally signed message part.
Re: Change to Mail Infrastructure - SPF and DKIM verification will now be enforced
On Friday, December 04, 2015 11:28:03 AM Jan Kundrát wrote: > On Friday, 4 December 2015 10:56:42 CET, Ben Cooksley wrote: > > Note that in the long run with DMARC looming you will need to switch > > to #2 anyway, and keeping your current behaviour will likely lead to > > mail from people who use Yahoo / AOL / etc ending up in the spam > > folder with many mailing list members. I'll be starting a discussion > > regarding taking this step on KDE systems at some point in the near > > future (switching to DMARC compatible policies). > > > > For more information, please see http://wiki.list.org/DEV/DMARC > > Do I understand your plan correctly? The following projects appear to not > re-sign their ML traffic, and they mangle headers at the same time. If I > understand your plan correctly, this means that I won't be able to use my > @kde.org addresses on mailing lists of these projects, for example: > > - Qt, > - Debian, > - Gentoo, > - OpenStack, > - anything hosted at SourceForge, > - and many, many more, essentially anybody who were ignoring DKIM. > > Please, change your plans, this is obviously a huge no-go. I cannot see how this would not hurt development. IMO, we can only enforce DMARC once all of our stakeholders support it or for those that do. We're not living in an isolated environment. We can be very strict and religious about standards and enforcing them, but in the end it's going to hurt us, and I don't think that's worth it at all. We'd simply be wasting energy we're putting into KDE, and that's not reasonable dealing with our resources. I think the lack of objections to this plan stems from almost nobody understanding what these plans actually mean for our development processes. The discussion happening around it is therefore not representative, and I think many who would have objected, haven't, for that reason. I can't say that I fully understand it myself. Please reconsider. Cheers, -- sebas http://www.kde.org | http://vizZzion.org
Re: Change to Mail Infrastructure - SPF and DKIM verification will now be enforced
On Friday, December 4, 2015 11:28:03 AM CET Jan Kundrát wrote: > On Friday, 4 December 2015 10:56:42 CET, Ben Cooksley wrote: > > Note that in the long run with DMARC looming you will need to switch > > to #2 anyway, and keeping your current behaviour will likely lead to > > mail from people who use Yahoo / AOL / etc ending up in the spam > > folder with many mailing list members. I'll be starting a discussion > > regarding taking this step on KDE systems at some point in the near > > future (switching to DMARC compatible policies). > > > > For more information, please see http://wiki.list.org/DEV/DMARC > > Do I understand your plan correctly? The following projects appear to not > re-sign their ML traffic, and they mangle headers at the same time. If I > understand your plan correctly, this means that I won't be able to use my > @kde.org addresses on mailing lists of these projects, for example: > > - Qt, > - Debian, > - Gentoo, > - OpenStack, > - anything hosted at SourceForge, > - and many, many more, essentially anybody who were ignoring DKIM. > > Please, change your plans, this is obviously a huge no-go. this looks like a huge problem. Could this be rolled out in two phases: one where a big fat warning is added in some way, so that we can inform our mailing list masters about the breakage and then a slow enforcement? Kicking out kde.org from important stakeholders doesn't look right to me. And it's not like we would notice. It might take quite some time till we notice no longer incoming mails in mailing list folders. And not everybody read this thread and understood the implications. I do not know how to verify that a mailing list sends correctly and there are important mailing lists I'm subscribed to with low traffic. So: can we do something to notify non compliant mailing lists? And what if they don't act on it? If for example freedesktop.org is slow on it the solution cannot be to effectively kick out kde from freedesktop.org. I'm not going to subscribe there with my private mail address, because it's important to be there with an @kde.org address. Cheers Martin signature.asc Description: This is a digitally signed message part.
Re: Change to Mail Infrastructure - SPF and DKIM verification will now be enforced
On Tue, Dec 8, 2015 at 2:19 AM, Martin Graesslin wrote: > On Friday, December 4, 2015 11:28:03 AM CET Jan Kundrát wrote: >> On Friday, 4 December 2015 10:56:42 CET, Ben Cooksley wrote: >> > Note that in the long run with DMARC looming you will need to switch >> > to #2 anyway, and keeping your current behaviour will likely lead to >> > mail from people who use Yahoo / AOL / etc ending up in the spam >> > folder with many mailing list members. I'll be starting a discussion >> > regarding taking this step on KDE systems at some point in the near >> > future (switching to DMARC compatible policies). >> > >> > For more information, please see http://wiki.list.org/DEV/DMARC >> >> Do I understand your plan correctly? The following projects appear to not >> re-sign their ML traffic, and they mangle headers at the same time. If I >> understand your plan correctly, this means that I won't be able to use my >> @kde.org addresses on mailing lists of these projects, for example: >> >> - Qt, >> - Debian, >> - Gentoo, >> - OpenStack, >> - anything hosted at SourceForge, >> - and many, many more, essentially anybody who were ignoring DKIM. >> >> Please, change your plans, this is obviously a huge no-go. > > this looks like a huge problem. Could this be rolled out in two phases: one > where a big fat warning is added in some way, so that we can inform our > mailing list masters about the breakage and then a slow enforcement? You can examine the "Authentication-Results" header from any mail that passes through kde.org mail infrastructure to determine if it is valid. These headers should be added by any system which is performing DKIM validation (even if it takes no action based on it) - Google at least also adds these headers. > > Kicking out kde.org from important stakeholders doesn't look right to me. And > it's not like we would notice. It might take quite some time till we notice no > longer incoming mails in mailing list folders. And not everybody read this > thread and understood the implications. I do not know how to verify that a > mailing list sends correctly and there are important mailing lists I'm > subscribed to with low traffic. You would still get the list subscription suspended message from the list, as these are generated by Mailman itself. They would only fail if they had tried to setup DKIM and made a mistake. > > So: can we do something to notify non compliant mailing lists? And what if > they don't act on it? If for example freedesktop.org is slow on it the > solution cannot be to effectively kick out kde from freedesktop.org. I'm not > going to subscribe there with my private mail address, because it's important > to be there with an @kde.org address. I would suggest mailing the list administrator or server administrator's of the mailing lists in question. Nobody else really has the power to fix it. The actual server administrators aren't required to take any action as long as you have the assistance of someone who can administrate the mailing list, at least with Mailman. You will only need the assistance of the server administrator if they've done something stupid - like enabling sitewide stripping of DKIM headers. In terms of what we should do - that will depend on their response > > Cheers > Martin Cheers, Ben
Re: Change to Mail Infrastructure - SPF and DKIM verification will now be enforced
On Tuesday, December 8, 2015 8:21:03 PM CET Ben Cooksley wrote: > On Tue, Dec 8, 2015 at 2:19 AM, Martin Graesslin wrote: > > On Friday, December 4, 2015 11:28:03 AM CET Jan Kundrát wrote: > >> On Friday, 4 December 2015 10:56:42 CET, Ben Cooksley wrote: > >> > Note that in the long run with DMARC looming you will need to switch > >> > to #2 anyway, and keeping your current behaviour will likely lead to > >> > mail from people who use Yahoo / AOL / etc ending up in the spam > >> > folder with many mailing list members. I'll be starting a discussion > >> > regarding taking this step on KDE systems at some point in the near > >> > future (switching to DMARC compatible policies). > >> > > >> > For more information, please see http://wiki.list.org/DEV/DMARC > >> > >> Do I understand your plan correctly? The following projects appear to not > >> re-sign their ML traffic, and they mangle headers at the same time. If I > >> understand your plan correctly, this means that I won't be able to use my > >> @kde.org addresses on mailing lists of these projects, for example: > >> > >> - Qt, > >> - Debian, > >> - Gentoo, > >> - OpenStack, > >> - anything hosted at SourceForge, > >> - and many, many more, essentially anybody who were ignoring DKIM. > >> > >> Please, change your plans, this is obviously a huge no-go. > > > > this looks like a huge problem. Could this be rolled out in two phases: > > one > > where a big fat warning is added in some way, so that we can inform our > > mailing list masters about the breakage and then a slow enforcement? > > You can examine the "Authentication-Results" header from any mail that > passes through kde.org mail infrastructure to determine if it is > valid. Checking the non-KDE mailing lists I'm subscribed to: * EWMH mailing list (hosted on GNOME infrastructure): Authentication-Results: postbox.kde.org; dkim=fail reason="verification failed; unprotected key" header.d=gmail.com header.i=@gmail.com header.b=qL4yX1lm; dkim-adsp=none (unprotected policy); dkim-atps=neutral * wayland: no such header * mesa: no such header, but receiving in digest form, so probably not possible to verify for me? * lxde: no such header * a private mailing list: same as with GNOME. This means GNOME and the private are wrong. What should I tell the list master now? Should we users not understanding the technical bits perform it? Wouldn't it be better if the sysadmin's do a mass filtering on the headers to figure out which mailing lists are affected and contact the list masters > You would still get the list subscription suspended message from the > list, as these are generated by Mailman itself. So kicking me out. If the system is not going to move, it means it's kicking us out. E.g. I would no longer be able to participate in EWMH discussions. > I would suggest mailing the list administrator or server > administrator's of the mailing lists in question. Nobody else really > has the power to fix it. I just don't know what to tell them. Seriously, please don't expect that we who have no clue about this stuff, will be able to contact list administrators. Cheers Martin signature.asc Description: This is a digitally signed message part.
Re: Change to Mail Infrastructure - SPF and DKIM verification will now be enforced
Hi all, The following is a guide which explains the problem, and how to correct it. SPF is a mail protocol designed to ensure a given mail server is in fact permitted to send email for a domain. It protects the return path component of an email, and helps protect against back scatter attacks, and to a certain extent makes it more difficult to falsely send mail. Please see https://en.wikipedia.org/wiki/Sender_Policy_Framework for more information. DKIM is a mail protocol designed to authenticate that the purported sender of an email actually sent the email. It does this by signing (using a public/private key mechanism) the mail body and some headers of the email and including this signature in the headers of the email, in the DKIM-Signature header. Many mail providers already implement this standard and use it to make spam filtering decisions. Technical details on how it works can be found at https://en.wikipedia.org/wiki/DomainKeys_Identified_Mail DMARC is a mail protocol designed to clarify policy enforcement and reporting around the pre-existing SPF and DKIM protocols. To date, while both of them defined what was valid and invalid, they did not state what actions should be taken if SPF or DKIM validation failed. DMARC allows domain administrators to provide this information to outside parties, and receive reports back regarding compliance - particularly in the case of failures, to aid in detecting phishing and configuration errors. Please see https://en.wikipedia.org/wiki/DMARC for more information. In regards to mailing lists, DKIM (and therefore DMARC as well) causes some problems, as rather common features - such as appending of footers and modifying subjects will break DKIM signatures. This in turn may lead to hosts which perform DKIM validation not accepting emails from the mailing list where the sender of the email signed it. Considering that Google & Yahoo among others have implemented this, a decent proportion of email landing on lists will likely be signed. To date this has not caused major issues, as DKIM was not being enforced. With the advent of DMARC however, providers are now beginning to enforce valid DKIM signatures. This requires mailing list administrators to take steps to ensure everyone is still able to subscribe and post to their mailing lists. A detailed overview of possible actions which can be taken is available at http://wiki.list.org/DEV/DMARC (for Mailman at least, the steps are broadly applicable to other software as well however). The easiest and least invasive way of correcting this problem is to stop modifying emails. This can be accomplished for Mailman lists by: a) Clearing the "subject_prefix" setting b) Clearing "msg_header" and "msg_footer" c) Disabling "scrub_nondigest" and "first_strip_reply_to" Depending on who posts to your list, you may also need to: a) Set "reply_goes_to_list" to "Poster" b) Set "include_sender_header" to "False". Note that the second set of changes should not be necessary in most cases. These changes will make your mailing list DMARC compliant, and will ensure that everyone is able to subscribe to, and respond to, postings on your list without inconveniencing the mail systems of anyone else on the list. Hope the above helps. Feel free to pass it on to any mailing list admin who needs it. Regards, Ben
Re: Change to Mail Infrastructure - SPF and DKIM verification will now be enforced
On Tuesday, December 8, 2015 9:51:50 AM CET Martin Graesslin wrote: > On Tuesday, December 8, 2015 8:21:03 PM CET Ben Cooksley wrote: > > On Tue, Dec 8, 2015 at 2:19 AM, Martin Graesslin wrote: > > > On Friday, December 4, 2015 11:28:03 AM CET Jan Kundrát wrote: > > >> On Friday, 4 December 2015 10:56:42 CET, Ben Cooksley wrote: > > >> > Note that in the long run with DMARC looming you will need to switch > > >> > to #2 anyway, and keeping your current behaviour will likely lead to > > >> > mail from people who use Yahoo / AOL / etc ending up in the spam > > >> > folder with many mailing list members. I'll be starting a discussion > > >> > regarding taking this step on KDE systems at some point in the near > > >> > future (switching to DMARC compatible policies). > > >> > > > >> > For more information, please see http://wiki.list.org/DEV/DMARC > > >> > > >> Do I understand your plan correctly? The following projects appear to > > >> not > > >> re-sign their ML traffic, and they mangle headers at the same time. If > > >> I > > >> understand your plan correctly, this means that I won't be able to use > > >> my > > >> @kde.org addresses on mailing lists of these projects, for example: > > >> > > >> - Qt, > > >> - Debian, > > >> - Gentoo, > > >> - OpenStack, > > >> - anything hosted at SourceForge, > > >> - and many, many more, essentially anybody who were ignoring DKIM. > > >> > > >> Please, change your plans, this is obviously a huge no-go. > > > > > > this looks like a huge problem. Could this be rolled out in two phases: > > > one > > > where a big fat warning is added in some way, so that we can inform our > > > mailing list masters about the breakage and then a slow enforcement? > > > > You can examine the "Authentication-Results" header from any mail that > > passes through kde.org mail infrastructure to determine if it is > > valid. > > Checking the non-KDE mailing lists I'm subscribed to: > > * EWMH mailing list (hosted on GNOME infrastructure): > > Authentication-Results: postbox.kde.org; dkim=fail > reason="verification failed; unprotected key" > header.d=gmail.com header.i=@gmail.com header.b=qL4yX1lm; > dkim-adsp=none (unprotected policy); dkim-atps=neutral > > * wayland: no such header Correction: Wayland is also affected. I didn't check a gmail mail. So given that all freedesktop.org are probably affected. Sorry Ben, that's just a no, it will be highly disruptive to KDE to turn us off from these mailing lists. Cheers Martin signature.asc Description: This is a digitally signed message part.
Re: Change to Mail Infrastructure - SPF and DKIM verification will now be enforced
On Tue, Dec 8, 2015 at 10:33 PM, Martin Graesslin wrote: > On Tuesday, December 8, 2015 9:51:50 AM CET Martin Graesslin wrote: >> On Tuesday, December 8, 2015 8:21:03 PM CET Ben Cooksley wrote: >> > On Tue, Dec 8, 2015 at 2:19 AM, Martin Graesslin > wrote: >> > > On Friday, December 4, 2015 11:28:03 AM CET Jan Kundrát wrote: >> > >> On Friday, 4 December 2015 10:56:42 CET, Ben Cooksley wrote: >> > >> > Note that in the long run with DMARC looming you will need to switch >> > >> > to #2 anyway, and keeping your current behaviour will likely lead to >> > >> > mail from people who use Yahoo / AOL / etc ending up in the spam >> > >> > folder with many mailing list members. I'll be starting a discussion >> > >> > regarding taking this step on KDE systems at some point in the near >> > >> > future (switching to DMARC compatible policies). >> > >> > >> > >> > For more information, please see http://wiki.list.org/DEV/DMARC >> > >> >> > >> Do I understand your plan correctly? The following projects appear to >> > >> not >> > >> re-sign their ML traffic, and they mangle headers at the same time. If >> > >> I >> > >> understand your plan correctly, this means that I won't be able to use >> > >> my >> > >> @kde.org addresses on mailing lists of these projects, for example: >> > >> >> > >> - Qt, >> > >> - Debian, >> > >> - Gentoo, >> > >> - OpenStack, >> > >> - anything hosted at SourceForge, >> > >> - and many, many more, essentially anybody who were ignoring DKIM. >> > >> >> > >> Please, change your plans, this is obviously a huge no-go. >> > > >> > > this looks like a huge problem. Could this be rolled out in two phases: >> > > one >> > > where a big fat warning is added in some way, so that we can inform our >> > > mailing list masters about the breakage and then a slow enforcement? >> > >> > You can examine the "Authentication-Results" header from any mail that >> > passes through kde.org mail infrastructure to determine if it is >> > valid. >> >> Checking the non-KDE mailing lists I'm subscribed to: >> >> * EWMH mailing list (hosted on GNOME infrastructure): >> >> Authentication-Results: postbox.kde.org; dkim=fail >> reason="verification failed; unprotected key" >> header.d=gmail.com header.i=@gmail.com header.b=qL4yX1lm; >> dkim-adsp=none (unprotected policy); dkim-atps=neutral >> >> * wayland: no such header > > Correction: Wayland is also affected. I didn't check a gmail mail. So given > that all freedesktop.org are probably affected. > > Sorry Ben, that's just a no, it will be highly disruptive to KDE to turn us > off from these mailing lists. Can't recall if I stated this previously, but i'd already decided to delay this until the end of January. It should not be delayed forever though. > > Cheers > Martin
Re: Change to Mail Infrastructure - SPF and DKIM verification will now be enforced
On Tuesday, 8 December 2015 10:19:51 CET, Ben Cooksley wrote: a) Clearing the "subject_prefix" setting b) Clearing "msg_header" and "msg_footer" c) Disabling "scrub_nondigest" and "first_strip_reply_to" Depending on who posts to your list, you may also need to: a) Set "reply_goes_to_list" to "Poster" b) Set "include_sender_header" to "False". So you're proposing that all mailing lists over the whole world should cease adding the "[foo-bar]" prefix into subjects, and refrain from adding/overwriting the Reply-To header (among other things). I've seen many instances of these discussions, the bikeshedding was fun, but there was never any particular outcome. It is irrelevant what our personal preference is. The fact of life is that there *are* mailing lists out there which perform these modifications, and these MLs won't change their config despite changes on our side. If we start rejecting these e-mails, well, our addresses will be unsubscribed from these MLs and we won't be able to participate in relevant technical discussions. If that happens, I'm afraid that the @kde.org e-mail addresses will no longer provide their stated value. Cheers, Jan -- Trojitá, a fast Qt IMAP e-mail client -- http://trojita.flaska.net/
Re: Change to Mail Infrastructure - SPF and DKIM verification will now be enforced
On Tue, Dec 8, 2015 at 11:11 PM, Jan Kundrát wrote: > On Tuesday, 8 December 2015 10:19:51 CET, Ben Cooksley wrote: >> >> a) Clearing the "subject_prefix" setting >> b) Clearing "msg_header" and "msg_footer" >> c) Disabling "scrub_nondigest" and "first_strip_reply_to" >> >> Depending on who posts to your list, you may also need to: >> a) Set "reply_goes_to_list" to "Poster" >> b) Set "include_sender_header" to "False". > > > So you're proposing that all mailing lists over the whole world should cease > adding the "[foo-bar]" prefix into subjects, and refrain from > adding/overwriting the Reply-To header (among other things). I've seen many > instances of these discussions, the bikeshedding was fun, but there was > never any particular outcome. It's the easiest way, and something supported by even the oldest Mailman instances. Newer ones have some other things you can try, as documented on the wiki page I pointed you to. > > It is irrelevant what our personal preference is. The fact of life is that > there *are* mailing lists out there which perform these modifications, and > these MLs won't change their config despite changes on our side. If we start > rejecting these e-mails, well, our addresses will be unsubscribed from these > MLs and we won't be able to participate in relevant technical discussions. > If that happens, I'm afraid that the @kde.org e-mail addresses will no > longer provide their stated value. Those mailing lists need to become DMARC compliant. How they achieve it, is up to them. The page I linked has many alternatives. Mailing list admins can choose their favourite. Note that GMail already uses DMARC policies to make spam filter decisions, and Yahoo hard rejects on them. By failing to comply, you risk excluding people already (and don't say you don't, something like 55% of all outbound email from kde.org goes to Google). > > Cheers, > Jan Regards, Ben > > > -- > Trojitá, a fast Qt IMAP e-mail client -- http://trojita.flaska.net/
Re: Change to Mail Infrastructure - SPF and DKIM verification will now be enforced
> It is irrelevant what our personal preference is. The fact of life is that there *are* mailing lists out there which perform these modifications, and these MLs won't change their config despite changes on our side. If we start rejecting these e-mails, well, our addresses will be unsubscribed from these MLs If this is about *incoming* mails, what would prevent "us" (i have no kde address anyway) from whitelisting known mailing lists until (if ever) they align their setups? (technically spoken, if you modify a mail, *you* wrote it, i assume that's why mailman has a dedicated wrapping feature) Cheers, Thomas
Re: Change to Mail Infrastructure - SPF and DKIM verification will now be enforced
On Tuesday, December 8, 2015 10:54:13 PM CET Ben Cooksley wrote: > On Tue, Dec 8, 2015 at 10:33 PM, Martin Graesslin wrote: > > On Tuesday, December 8, 2015 9:51:50 AM CET Martin Graesslin wrote: > >> On Tuesday, December 8, 2015 8:21:03 PM CET Ben Cooksley wrote: > >> > On Tue, Dec 8, 2015 at 2:19 AM, Martin Graesslin > > > > wrote: > >> > > On Friday, December 4, 2015 11:28:03 AM CET Jan Kundrát wrote: > >> > >> On Friday, 4 December 2015 10:56:42 CET, Ben Cooksley wrote: > >> > >> > Note that in the long run with DMARC looming you will need to > >> > >> > switch > >> > >> > to #2 anyway, and keeping your current behaviour will likely lead > >> > >> > to > >> > >> > mail from people who use Yahoo / AOL / etc ending up in the spam > >> > >> > folder with many mailing list members. I'll be starting a > >> > >> > discussion > >> > >> > regarding taking this step on KDE systems at some point in the > >> > >> > near > >> > >> > future (switching to DMARC compatible policies). > >> > >> > > >> > >> > For more information, please see http://wiki.list.org/DEV/DMARC > >> > >> > >> > >> Do I understand your plan correctly? The following projects appear > >> > >> to > >> > >> not > >> > >> re-sign their ML traffic, and they mangle headers at the same time. > >> > >> If > >> > >> I > >> > >> understand your plan correctly, this means that I won't be able to > >> > >> use > >> > >> my > >> > >> @kde.org addresses on mailing lists of these projects, for example: > >> > >> > >> > >> - Qt, > >> > >> - Debian, > >> > >> - Gentoo, > >> > >> - OpenStack, > >> > >> - anything hosted at SourceForge, > >> > >> - and many, many more, essentially anybody who were ignoring DKIM. > >> > >> > >> > >> Please, change your plans, this is obviously a huge no-go. > >> > > > >> > > this looks like a huge problem. Could this be rolled out in two > >> > > phases: > >> > > one > >> > > where a big fat warning is added in some way, so that we can inform > >> > > our > >> > > mailing list masters about the breakage and then a slow enforcement? > >> > > >> > You can examine the "Authentication-Results" header from any mail that > >> > passes through kde.org mail infrastructure to determine if it is > >> > valid. > >> > >> Checking the non-KDE mailing lists I'm subscribed to: > >> > >> * EWMH mailing list (hosted on GNOME infrastructure): > >> > >> Authentication-Results: postbox.kde.org; dkim=fail > >> > >> reason="verification failed; unprotected key" > >> header.d=gmail.com header.i=@gmail.com header.b=qL4yX1lm; > >> dkim-adsp=none (unprotected policy); dkim-atps=neutral > >> > >> * wayland: no such header > > > > Correction: Wayland is also affected. I didn't check a gmail mail. So > > given > > that all freedesktop.org are probably affected. > > > > Sorry Ben, that's just a no, it will be highly disruptive to KDE to turn > > us > > off from these mailing lists. > > Can't recall if I stated this previously, but i'd already decided to > delay this until the end of January. > It should not be delayed forever though. Can we increase this to at least end of February or March? I'm a little bit concerned that this won't go anywhere over Christmas and New Year and end of January not being realistic. As much as I support your proposal in general, I just don't have any spare time to work on this by contacting list admins, etc. I also think we need to coordinate this. I'm sure we can get the changes applied, but it'll take more time than a month. This is a small project which needs to be coordinated: * track all affected mailing lists @kde.org address owners interact with * create a task for each mailing list on phab * get someone to contact, document when contacted * track the progress * send reminders including (hey project foo was able to change that in three days, and we are waiting now two weeks) * switch when we can be certain that it won't disrupt our work As a list moderator I also hate the spam, but not being able to be on the mailing lists with my @kde.org address on mailing lists where I am only because I consider myself as a representative for kde is worse. Cheers Martin signature.asc Description: This is a digitally signed message part.
Re: Change to Mail Infrastructure - SPF and DKIM verification will now be enforced
On 08/12/15 10:54, Ben Cooksley wrote: >> > Correction: Wayland is also affected. I didn't check a gmail mail. So given >> > that all freedesktop.org are probably affected. >> > >> > Sorry Ben, that's just a no, it will be highly disruptive to KDE to turn us >> > off from these mailing lists. > Can't recall if I stated this previously, but i'd already decided to > delay this until the end of January. > It should not be delayed forever though. I am also quite sceptical of this. The main task of a mail system is to deliver email reliably. Anti-Spam measures are only useful if they have a very very low false-positive rate. And from my personal experience, DKIM is so shaky to date that you will end up rejecting _lots_ of mail erraneously. Greetings, Sven signature.asc Description: OpenPGP digital signature
Re: Change to Mail Infrastructure - SPF and DKIM verification will now be enforced
> El 8 dic 2015, a las 07:11, Jan Kundrát escribió: > > It is irrelevant what our personal preference is. The fact of life is that > there *are* mailing lists out there which perform these modifications, and > these MLs won't change their config despite changes on our side. If we start > rejecting these e-mails, well, our addresses will be unsubscribed from these > MLs and we won't be able to participate in relevant technical discussions. If > that happens, I'm afraid that the @kde.org e-mail addresses will no longer > provide their stated value. Unfortunately, this is the mess that DMARC got us into: It is irrelevant what our personal preference about doing modifications to messages is (like the tag in the subject). The fact of life is that there *are* mail providers out there (like Yahoo) which are already enforcing DMARC and may reject messages with such DKIM-breaking modifications, and these mail providers won't change their config to accommodate us.
Re: Change to Mail Infrastructure - SPF and DKIM verification will now be enforced
On Tuesday, 8 December 2015 16:09:43 CET, Nicolás Alvarez wrote: It is irrelevant what our personal preference about doing modifications to messages is (like the tag in the subject). The fact of life is that there *are* mail providers out there (like Yahoo) which are already enforcing DMARC and may reject messages with such DKIM-breaking modifications, and these mail providers won't change their config to accommodate us. Nicely said. Yes, there are providers such as Yahoo, AOL, and nobody else :) who decided to break a long-working infrastructure. The question is whether we want to join this club. Should we start enforcing the same rules that Yahoo is enforcing? (Ben didn't say what SPF and DKIM rules he's planning to publish for @kde.org, btw.) Do we have many Yahoo/AOL users among our developers? Should we start publishing rules which effectively instruct others to discard all e-mails from @kde.org once they go through a non-DMARC mailing list? Should we discard e-mails which are intended for our developers because they went through a non-DMARC mailing list? My answer to these two questions is "no" and "no", obviously. I don't know how else to say this -- Debian is not exactly a small open source project, and their sysadmins apparently haven't cared about DKIM so far. It's a technology which requires Everybody Else™ to perform extra work and to configure new services on the servers which host various mailing lists. Are we seriously trying to push an unspecified number of third-party ML providers to deploy DKIM because Ben decided that it's worth the effort? Seriously, the Internet just doesn't work this way. Even if Debian's ifnrastructure is changed, there is still a number of mailing lists which have worked well in the past years, and now they will stop working for @kde.org accounts. Cheers, Jan -- Trojitá, a fast Qt IMAP e-mail client -- http://trojita.flaska.net/
Re: Change to Mail Infrastructure - SPF and DKIM verification will now be enforced
On Friday, 4 December 2015 10:56:42 CET, Ben Cooksley wrote: To be specific I will be enabling the following line: On-BadSignature tempfail within the configuration of OpenDKIM on our servers. Thanks, but that's not a full answer. What is the proposed content of the following DNS records? 1) TXT, kde.org (for the final SPF policy) 2) TXT, _dmarc.kde.org (for *our* DMARC policy, which is an extremely important piece of missing information) 3) TXT, default._domainkey.kde.org (and others which you intend to use) Cheers, Jan -- Trojitá, a fast Qt IMAP e-mail client -- http://trojita.flaska.net/
Re: Change to Mail Infrastructure - SPF and DKIM verification will now be enforced
On Wed, Dec 9, 2015 at 5:00 AM, Jan Kundrát wrote: > On Friday, 4 December 2015 10:56:42 CET, Ben Cooksley wrote: >> >> To be specific I will be enabling the following line: >> >> On-BadSignature tempfail >> >> within the configuration of OpenDKIM on our servers. > > > Thanks, but that's not a full answer. What is the proposed content of the > following DNS records? > > 1) TXT, kde.org (for the final SPF policy) > 2) TXT, _dmarc.kde.org (for *our* DMARC policy, which is an extremely > important piece of missing information) > 3) TXT, default._domainkey.kde.org (and others which you intend to use) This whole thread is about INBOUND email - so it is a full answer. Doesn't have anything to do with outbound, which is remaining unchanged. None of those records will be changed from their current values (which in the case of DMARC is nothing) At some point in the future we may change them, but not at this time. > > Cheers, > Jan Regards, Ben > > > -- > Trojitá, a fast Qt IMAP e-mail client -- http://trojita.flaska.net/
Re: Change to Mail Infrastructure - SPF and DKIM verification will now be enforced
On Wed, Dec 9, 2015 at 4:37 AM, Jan Kundrát wrote: > On Tuesday, 8 December 2015 16:09:43 CET, Nicolás Alvarez wrote: >> >> It is irrelevant what our personal preference about doing modifications to >> messages is (like the tag in the subject). The fact of life is that there >> *are* mail providers out there (like Yahoo) which are already enforcing >> DMARC and may reject messages with such DKIM-breaking modifications, and >> these mail providers won't change their config to accommodate us. > > > Nicely said. Yes, there are providers such as Yahoo, AOL, and nobody else :) > who decided to break a long-working infrastructure. The question is whether > we want to join this club. What you're basically saying is - you don't care about potential contributors who use these service providers. Lovely. I'm afraid that's unacceptable. A chunk of our users certainly do use them, and i'm sure we'd like them to be able to interact with our community. > > Should we start enforcing the same rules that Yahoo is enforcing? (Ben > didn't say what SPF and DKIM rules he's planning to publish for @kde.org, > btw.) Do we have many Yahoo/AOL users among our developers? The domain owner is actually the one that sets the policy under DMARC. All we'd be doing is following their wishes. I haven't said anything, because I don't plan to change our own policies at this time. > > Should we start publishing rules which effectively instruct others to > discard all e-mails from @kde.org once they go through a non-DMARC mailing > list? > > Should we discard e-mails which are intended for our developers because they > went through a non-DMARC mailing list? >From the server's point of view, these messages could also be forgeries, containing spam, phishing attacks, or otherwise attempting to misrepresent the sender. If the domain owner has stated the message should have a valid signature, we'd be right to follow what they've requested. > > My answer to these two questions is "no" and "no", obviously. I don't know > how else to say this -- Debian is not exactly a small open source project, > and their sysadmins apparently haven't cared about DKIM so far. It's a DKIM has been around for an extremely long time. It means they've not worked on improving their email deliverability. > technology which requires Everybody Else™ to perform extra work and to > configure new services on the servers which host various mailing lists. Are Configuration of new services is not required, as noted in my earlier mail. > we seriously trying to push an unspecified number of third-party ML > providers to deploy DKIM because Ben decided that it's worth the effort? Nice to demonise me here. The mailing list hosts don't have to deploy DKIM. All they have to do is not break signatures on mails bearing a DKIM signature. Which, as I noted in my email is something that only requires a few toggles within the Mailman administration interface. (And, using the withlist tool can be changed on all lists on an entire server with relative ease). This is what Debian has chosen to do. > Seriously, the Internet just doesn't work this way. Even if Debian's > ifnrastructure is changed, there is still a number of mailing lists which > have worked well in the past years, and now they will stop working for > @kde.org accounts. > > > Cheers, > Jan Regards, Ben > > -- > Trojitá, a fast Qt IMAP e-mail client -- http://trojita.flaska.net/
Re: Change to Mail Infrastructure - SPF and DKIM verification will now be enforced
Can we please lower the temperature on this discussion a bit? On Wed, Dec 9, 2015 at 1:54 AM, Ben Cooksley wrote: > On Wed, Dec 9, 2015 at 4:37 AM, Jan Kundrát wrote: >> On Tuesday, 8 December 2015 16:09:43 CET, Nicolás Alvarez wrote: >>> >>> It is irrelevant what our personal preference about doing modifications to >>> messages is (like the tag in the subject). The fact of life is that there >>> *are* mail providers out there (like Yahoo) which are already enforcing >>> DMARC and may reject messages with such DKIM-breaking modifications, and >>> these mail providers won't change their config to accommodate us. >> >> >> Nicely said. Yes, there are providers such as Yahoo, AOL, and nobody else :) >> who decided to break a long-working infrastructure. The question is whether >> we want to join this club. > > What you're basically saying is - you don't care about potential > contributors who use these service providers. > Lovely. > > I'm afraid that's unacceptable. A chunk of our users certainly do use > them, and i'm sure we'd like them to be able to interact with our > community. > >> >> Should we start enforcing the same rules that Yahoo is enforcing? (Ben >> didn't say what SPF and DKIM rules he's planning to publish for @kde.org, >> btw.) Do we have many Yahoo/AOL users among our developers? > > The domain owner is actually the one that sets the policy under DMARC. > All we'd be doing is following their wishes. > I haven't said anything, because I don't plan to change our own > policies at this time. > >> >> Should we start publishing rules which effectively instruct others to >> discard all e-mails from @kde.org once they go through a non-DMARC mailing >> list? >> >> Should we discard e-mails which are intended for our developers because they >> went through a non-DMARC mailing list? > > From the server's point of view, these messages could also be > forgeries, containing spam, phishing attacks, or otherwise attempting > to misrepresent the sender. > If the domain owner has stated the message should have a valid > signature, we'd be right to follow what they've requested. > >> >> My answer to these two questions is "no" and "no", obviously. I don't know >> how else to say this -- Debian is not exactly a small open source project, >> and their sysadmins apparently haven't cared about DKIM so far. It's a > > DKIM has been around for an extremely long time. > It means they've not worked on improving their email deliverability. > >> technology which requires Everybody Else™ to perform extra work and to >> configure new services on the servers which host various mailing lists. Are > > Configuration of new services is not required, as noted in my earlier mail. > >> we seriously trying to push an unspecified number of third-party ML >> providers to deploy DKIM because Ben decided that it's worth the effort? > > Nice to demonise me here. > > The mailing list hosts don't have to deploy DKIM. All they have to do > is not break signatures on mails bearing a DKIM signature. > Which, as I noted in my email is something that only requires a few > toggles within the Mailman administration interface. > (And, using the withlist tool can be changed on all lists on an entire > server with relative ease). This is what Debian has chosen to do. > >> Seriously, the Internet just doesn't work this way. Even if Debian's >> ifnrastructure is changed, there is still a number of mailing lists which >> have worked well in the past years, and now they will stop working for >> @kde.org accounts. >> >> >> Cheers, >> Jan > > Regards, > Ben Ben isn't the only person wrestling with these issues. We're dealing with it on a much smaller scale in Linuxchix. Anyone who runs lists is having to deal with spam, malware, forgeries, phishing, and so forth. While the tools to mitigate some of this are complicated, we will have to make changes at some time. I suggest that we calmly pull together and figure out how we can make KDE email and lists more secure, as well as all of those lists with which we use our KDE.org accounts. We *can* do this, we need to do this, and Ben doesn't have time to do all the work himself. So we need to do our part to help. Valorie
Re: Change to Mail Infrastructure - SPF and DKIM verification will now be enforced
On Wednesday, December 9, 2015 2:09:01 AM CET Valorie Zimmerman wrote: > We *can* do this, we need to do this, and Ben doesn't have time to do > all the work himself. So we need to do our part to help. That's what I suggested in one of my mails. We need to tackle this as a coordinated project where we keep track on which mailing list got contacted, which one changed and so on. I think what some see as a huge problem here is that this can be a highly disruptive change to users of @kde.org addresses and that the current time frame is too short (Christmas, New Year, etc.). We need to tackle it as a project and we need to accept that it might take half a year or year to ensure that it's not disruptive to our community. I don't like spam and forgery and what not and I'm full in support for what our sysadmins suggest, but it must not be disruptive. If I'm kicked out of one mailing list it will be disruptive to me and render my @kde.org address useless and by that harming KDE as an organization. That must not happen. Given that we need to tackle it as a coordinated project. I'm sorry I won't be able to help on that before end of January due to personal reasons restricting my spare time. Btw. this could be nice GCI tasks: contact a mailing list, keep track of it. Cheers Martin signature.asc Description: This is a digitally signed message part.
Re: Change to Mail Infrastructure - SPF and DKIM verification will now be enforced
I've taken the liberty to remove the ad-hominem which you used. I'm not happy with your approach to this discussion, but I'll try to stick with the technical points. There is active work within the DMARC WG, with first drafts being published only *two months ago* [1]. My suggestion for everybody who doesn't have time to follow this process is to sit back, relax, and watch the IETF come up with a solution, and *then* start implementing their suggestions. Asking one's user base to reach every list service administrator out there with a "fix your DKIM/DMARC" is not going to work. Deploying DMARC at this point in time, when substantial changes are still being worked on, doesn't look like a good idea, either. This is all that I'm saying. The mailing list hosts don't have to deploy DKIM. All they have to do is not break signatures on mails bearing a DKIM signature. Which, as I noted in my email is something that only requires a few toggles within the Mailman administration interface. (And, using the withlist tool can be changed on all lists on an entire server with relative ease). This is what Debian has chosen to do. You're saying that it's easy to configure a ML to stop breaking DMARC signatures. I disagree. Here's my reasoning: 1) Full compliance with DMARC requires a substantial reduction of features which distinguish mailing lists from dumb forwarders. This includes: - the Reply-To munging, - adding a [prefix] to subject headers, - automatic signatures, - in case of overly strict DKIM setup, the various List-* headers which are actually mandated by RFCs to be automatically added. 2) Some domains might specify DMARC policies which prevent *any* distribution of their e-mails over mailing lists. The only solution for this problem is rewriting the RFC5322.From header to something like: From: "Foo Bar via a KDE ML" This in turns leads to e-mails where one cannot reply to the original author anymore, etc etc etc. In case someone is still following this thread, let me quote [2] John R. Levine, one of the Internet graybeards: Mailing list apps can't "implement DMARC" other than by getting rid of every feature that makes lists more functional than simple forwarders. Given that we haven't done so for any of the previous FUSSPs that didn't contemplate mailing lists, because those features are useful to our users, it seems unlikely we'll do so now. If receivers want to implement DMARC policy, they need to make their false alarm whitelist first. This appears to be a substantial, perhaps insurmountable, hurdle. "FUSSP" is a "Final Ultimate Solution to the Spam Problem". That entire thread is worth reading, btw. Cheers, Jan [1] https://tools.ietf.org/html/draft-andersen-arc-00 [2] http://www.ietf.org/mail-archive/web/ietf/current/msg87157.html -- Trojitá, a fast Qt IMAP e-mail client -- http://trojita.flaska.net/
Re: Change to Mail Infrastructure - SPF and DKIM verification will now be enforced
Hi all, I'm going recount a personal experience here. I have my own domain (BaloneyGeek.com) and I use Google Apps for Business for my E-Mail. A couple of months ago I shifted DNS providers and took the opportunity to properly set up E-Mail verification and signing. Using Google's documentation, I enabled SPF, and then tested. Then I enabled DKIM and tested. So far, everything was fine. Then I enabled DMARC and all hell broke loose. Even though Google's configuration checker gave me a green tick on DMARC configuration, I couldn't send mail to any non Google-handled e-mail ID, without it being sent to spam. I know this because I tested with one Windows Live Mail (or whatever they call it these days) account and one Yahoo account. Both of them had a history of receiving e-mails from me. I would also get an XML file delivered to my inbox from every single e-mail server that handled my mail, with stats of how many mails they handled, how many passed auth, how many failed and how many were sent to spam. Apart from the annoyance of receiving tens of these mails per day, I noted that every single provider (other than Google) was failing auth on all my mails and sending them to spam. I dug around multiple docs (including RFC 7489, Google's docs, etc) and couldn't find any configuration errors I'd made. In they end I had to roll back DMARC (which took two days to propagate across all DNS caches, btw), while keeping SPF and DKIM enabled. Everything has been fine since then. So here's my two cents - SPF **should** always be enabled, that's the bare minimum you can do. DKIM enforces SPF using signing, so if you guys can implement that well, awesome. But be very careful when dealing with DMARC. From what I saw when I tried to set it up, no e-mail provider other than Google knows what to do with it. -- Boudhayan
Re: Change to Mail Infrastructure - SPF and DKIM verification will now be enforced
I just submitted a request to Ubuntu to upgrade their list infra. I showed Ben a sample email header and they were not compliant. I included a link to Ben's explanatory post: http://marc.info/?l=kde-core-devel&m=144956641505429&w=1 It took less than five minutes to file the ticket. All the best, Valorie On Wed, Dec 9, 2015 at 3:22 AM, Boudhayan Gupta wrote: > Hi all, > > I'm going recount a personal experience here. I have my own domain > (BaloneyGeek.com) and I use Google Apps for Business for my E-Mail. > > A couple of months ago I shifted DNS providers and took the > opportunity to properly set up E-Mail verification and signing. Using > Google's documentation, I enabled SPF, and then tested. Then I enabled > DKIM and tested. > > So far, everything was fine. > > Then I enabled DMARC and all hell broke loose. Even though Google's > configuration checker gave me a green tick on DMARC configuration, I > couldn't send mail to any non Google-handled e-mail ID, without it > being sent to spam. I know this because I tested with one Windows Live > Mail (or whatever they call it these days) account and one Yahoo > account. Both of them had a history of receiving e-mails from me. > > I would also get an XML file delivered to my inbox from every single > e-mail server that handled my mail, with stats of how many mails they > handled, how many passed auth, how many failed and how many were sent > to spam. Apart from the annoyance of receiving tens of these mails per > day, I noted that every single provider (other than Google) was > failing auth on all my mails and sending them to spam. > > I dug around multiple docs (including RFC 7489, Google's docs, etc) > and couldn't find any configuration errors I'd made. > > In they end I had to roll back DMARC (which took two days to propagate > across all DNS caches, btw), while keeping SPF and DKIM enabled. > Everything has been fine since then. > > So here's my two cents - SPF **should** always be enabled, that's the > bare minimum you can do. DKIM enforces SPF using signing, so if you > guys can implement that well, awesome. But be very careful when > dealing with DMARC. From what I saw when I tried to set it up, no > e-mail provider other than Google knows what to do with it. > > -- Boudhayan -- http://about.me/valoriez
Re: Change to Mail Infrastructure - SPF and DKIM verification will now be enforced
Hi, On Wed, 09 Dec 2015 11:51:43 +0100 Jan Kundrát wrote: [...] > There is active work within the DMARC WG, with first drafts being > published only *two months ago* [1]. My suggestion for everybody who > doesn't have time to follow this process is to sit back, relax, and > watch the IETF come up with a solution, and *then* start implementing > their suggestions. I believe this is going partially off-topic, but as I have recently (and entirely unrelatedly) run into this issue as a list admin, let me just tell you: The problem is real today. You may not be noticing it as a list-admin, yet, if you don't have a high enough rate of yahoo-users posting to the list, but if you have *any* yahoo-users posting to your list, then your mailing list is already broken for a share of your subscribers, today. For rkward-devel, as I have found out the hard way, the share of affected subscribers was slightly above 10%. The solution you are referring to, above, is not real today. But workarounds exist. [...] > You're saying that it's easy to configure a ML to stop breaking DMARC > signatures. I disagree. Here's my reasoning: > > 1) Full compliance with DMARC requires a substantial reduction of > features which distinguish mailing lists from dumb forwarders. This > includes: > > - the Reply-To munging, > - adding a [prefix] to subject headers, > - automatic signatures, > - in case of overly strict DKIM setup, the various List-* headers > which are actually mandated by RFCs to be automatically added. Yes, I am not really happy with removing these features either (not counting the fourth, which *is* a rather theoretical problem at this point of time, as far as I am aware). And I do hope to recover these features, once a better solution comes into existence. But seriously, the choice between clinging to these features and excluding 10% of current subscribers, 10+x% of future subscribers was easy to me. Regards Thomas pgpKkHUirNmYv.pgp Description: OpenPGP digital signature
