[Wikitech-l] DEPLOYED TODAY: SPF (email spoof prevention feature) test-rollout Weds 10/3
As of ~11:15AM EDT SPF is deployed for the domain wikimedia.org. Please let me know ASAP if you discover any issues with mail sent from a @wikimedia.org address. Thanks! jg Jeff Green Operations Engineer, Special Projects Wikimedia Foundation 149 New Montgomery Street, 3rd Floor San Francisco, CA 94105 415-839-6885 x6807 jgr...@wikimedia.org P.S. Ops folks, rollback is simply a matter of reverting the wikimedia.org zone file and running authdns-update. I set the TTL to 10 min just in case. -- Forwarded message -- Date: Fri, 28 Sep 2012 11:00:08 -0700 (PDT) From: Jeff Green jgr...@wikimedia.org Reply-To: Wikimedia developers wikitech-l@lists.wikimedia.org To: wmf...@lists.wikimedia.org, wikimedi...@lists.wikimedia.org, wikitech-l@lists.wikimedia.org Subject: [Wikitech-l] SPF (email spoof prevention feature) test-rollout Weds 10/5 I'm planning to deploy Sender Policy Framework (SPF) for the wikimedia.org domain on Weds October 5. SPF is a framework for validating outgoing mail, which gives the receiving side useful information for spam filtering. The main goal is to cause spoofed @wikimedia.org mail to be correctly identified as such. It should also improve our odds of getting fundraiser mailings into inboxes rather than spam folders. The change should not be noticeable, but the most likely problem would be legitimate @wikimedia.org mail being treated as spam. If you hear of this happening please let me know. Technical details are below for anyone interested . . . Thanks, jg Jeff Green Operations Engineer, Special Projects Wikimedia Foundation 149 New Montgomery Street, 3rd Floor San Francisco, CA 94105 jgr...@wikimedia.org . . . . . . . SPF overview http://en.wikipedia.org/wiki/Sender_Policy_Framework The October 8 change will be simply a matter of adding a TXT record to the wikimedia.org DNS zone: wikimedia.org IN TXT v=spf1 ip4:91.198.174.0/24 ip4:208.80.152.0/22 ip6:2620:0:860::/46 include:_spf.google.com ip4:74.121.51.111 ?all The record is a list of subnets that we identify as senders (all wmf subnets, google apps, and the fundraiser mailhouse). The ?all is a neutral policy--it doesn't state either way how mail should be handled. Eventually we'll probably bump ?all to a stricter ~all aka SoftFail, which tells the receiving side that only mail coming from the listed subnets is valid. Most ISPs will route 'other' mail to a spam folder based on SoftFail. Please bug me with any questions/comments! ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] SPF (email spoof prevention feature) test-rollout Weds 10/5
I'm planning to deploy Sender Policy Framework (SPF) for the wikimedia.org domain on Weds October 5. SPF is a framework for validating outgoing mail, which gives the receiving side useful information for spam filtering. The main goal is to cause spoofed @wikimedia.org mail to be correctly identified as such. It should also improve our odds of getting fundraiser mailings into inboxes rather than spam folders. The change should not be noticeable, but the most likely problem would be legitimate @wikimedia.org mail being treated as spam. If you hear of this happening please let me know. Technical details are below for anyone interested . . . Thanks, jg Jeff Green Operations Engineer, Special Projects Wikimedia Foundation 149 New Montgomery Street, 3rd Floor San Francisco, CA 94105 jgr...@wikimedia.org . . . . . . . SPF overview http://en.wikipedia.org/wiki/Sender_Policy_Framework The October 8 change will be simply a matter of adding a TXT record to the wikimedia.org DNS zone: wikimedia.org IN TXT v=spf1 ip4:91.198.174.0/24 ip4:208.80.152.0/22 ip6:2620:0:860::/46 include:_spf.google.com ip4:74.121.51.111 ?all The record is a list of subnets that we identify as senders (all wmf subnets, google apps, and the fundraiser mailhouse). The ?all is a neutral policy--it doesn't state either way how mail should be handled. Eventually we'll probably bump ?all to a stricter ~all aka SoftFail, which tells the receiving side that only mail coming from the listed subnets is valid. Most ISPs will route 'other' mail to a spam folder based on SoftFail. Please bug me with any questions/comments! ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] SPF (email spoof prevention feature) test-rollout Weds 10/5
On Fri, 28 Sep 2012, Daniel Friesen wrote: On Fri, 28 Sep 2012 11:00:08 -0700, Jeff Green jgr...@wikimedia.org wrote: I'm planning to deploy Sender Policy Framework (SPF) for the wikimedia.org domain on Weds October 5. SPF is a framework for validating outgoing mail, which gives the receiving side useful information for spam filtering. The main goal is to cause spoofed @wikimedia.org mail to be correctly identified as such. It should also improve our odds of getting fundraiser mailings into inboxes rather than spam folders. The change should not be noticeable, but the most likely problem would be legitimate @wikimedia.org mail being treated as spam. If you hear of this happening please let me know. Technical details are below for anyone interested . . . Thanks, jg Jeff Green Operations Engineer, Special Projects Wikimedia Foundation 149 New Montgomery Street, 3rd Floor San Francisco, CA 94105 jgr...@wikimedia.org . . . . . . . SPF overview http://en.wikipedia.org/wiki/Sender_Policy_Framework The October 8 change will be simply a matter of adding a TXT record to the wikimedia.org DNS zone: wikimedia.org IN TXT v=spf1 ip4:91.198.174.0/24 ip4:208.80.152.0/22 ip6:2620:0:860::/46 include:_spf.google.com ip4:74.121.51.111 ?all The record is a list of subnets that we identify as senders (all wmf subnets, google apps, and the fundraiser mailhouse). The ?all is a neutral policy--it doesn't state either way how mail should be handled. Eventually we'll probably bump ?all to a stricter ~all aka SoftFail, which tells the receiving side that only mail coming from the listed subnets is valid. Most ISPs will route 'other' mail to a spam folder based on SoftFail. I was under the impression that ~all softfail is not an assertion that something is not authorized and the only way to actually assert that is with -all hardfail. The distinction is essentially assert (-all) vs advise (~all). Ideally -all would result in a reject during SMTP, and ~all would be route-to-spam-folder. But I think what really happens is subjective to the receiving side. Please bug me with any questions/comments! -- ~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://daniel.friesen.name] ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] SPF (email spoof prevention feature) test-rollout Weds 10/5
Good point--thanks! jg On Fri, 28 Sep 2012, Tyler Romeo wrote: You should also add an SPF record in addition to a TXT record, as recommended by RFC 4408. The format is the same. *--* *Tyler Romeo* Stevens Institute of Technology, Class of 2015 Major in Computer Science www.whizkidztech.com | tylerro...@gmail.com On Fri, Sep 28, 2012 at 2:04 PM, Daniel Friesen dan...@nadir-seen-fire.comwrote: On Fri, 28 Sep 2012 11:00:08 -0700, Jeff Green jgr...@wikimedia.org wrote: I'm planning to deploy Sender Policy Framework (SPF) for the wikimedia.org domain on Weds October 5. SPF is a framework for validating outgoing mail, which gives the receiving side useful information for spam filtering. The main goal is to cause spoofed @wikimedia.orgmail to be correctly identified as such. It should also improve our odds of getting fundraiser mailings into inboxes rather than spam folders. The change should not be noticeable, but the most likely problem would be legitimate @wikimedia.org mail being treated as spam. If you hear of this happening please let me know. Technical details are below for anyone interested . . . Thanks, jg Jeff Green Operations Engineer, Special Projects Wikimedia Foundation 149 New Montgomery Street, 3rd Floor San Francisco, CA 94105 jgr...@wikimedia.org . . . . . . . SPF overview http://en.wikipedia.org/wiki/**Sender_Policy_Frameworkhttp://en.wikipedia.org/wiki/Sender_Policy_Framework The October 8 change will be simply a matter of adding a TXT record to the wikimedia.org DNS zone: wikimedia.org IN TXT v=spf1 ip4:91.198.174.0/24 ip4:208.80.152.0/22ip6:2620:0:860::/46 include:_ spf.google.com ip4:74.121.51.111 ?all The record is a list of subnets that we identify as senders (all wmf subnets, google apps, and the fundraiser mailhouse). The ?all is a neutral policy--it doesn't state either way how mail should be handled. Eventually we'll probably bump ?all to a stricter ~all aka SoftFail, which tells the receiving side that only mail coming from the listed subnets is valid. Most ISPs will route 'other' mail to a spam folder based on SoftFail. I was under the impression that ~all softfail is not an assertion that something is not authorized and the only way to actually assert that is with -all hardfail. Please bug me with any questions/comments! -- ~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://daniel.friesen.name] __**_ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/**mailman/listinfo/wikitech-lhttps://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] SPF (email spoof prevention feature) test-rollout Weds 10/5
On Fri, 28 Sep 2012, Daniel Friesen wrote: On Fri, 28 Sep 2012 12:19:21 -0700, Brad Jorsch b-jor...@alum.northwestern.edu wrote: On Fri, Sep 28, 2012 at 11:00:08AM -0700, Jeff Green wrote: The change should not be noticeable, but the most likely problem would be legitimate @wikimedia.org mail being treated as spam. If you hear of this happening please let me know. Anyone who sends all mail marked as from[1] their @wikimedia.org address through Gmail's SMTP server, through an SMTP server hosted by Wikimedia (is there one?), or through any other server identified in the SPF record should be fine. And anyone who isn't sending from an @wikimedia.org address should be entirely unaffected. If anyone is sending mail marked as from their @wikimedia.org address through some other SMTP server (e.g. through their home ISP), they might start to see trouble with this change and likely will when the SPF record is changed to ~all. Also, any recipient who has their mail forwarded might have trouble *receiving* messages from @wikimedia.org addresses, unless their forwarding service takes SPF into account or their destination mailbox doesn't check SPF. OTOH, these people would have the same problem with receiving mail from all the other domains that currently implement SPF. [1]: There are actually two concepts of from involved in email. The first, the envelope sender or mail from, is the address that bounce notifications should be sent to. The second is the address that actually shows up as From: in the email message. SPF is intended to target only the former, but SenderID hijacks the SPF specification to also test the latter. And to make things all fun and confusing. We shouldn't forget about the Sender: header... **mumbles about AWS-SES not supporting Sender:** Yes and SenderID is where we're running into deliverability issues for fundraiser mailings since we lack SPF, that's part of what prompted this whole initiative. Well, that and an ancient RT request from Office IT! -- ~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://daniel.friesen.name] ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] SPF (email spoof prevention feature) test-rollout Weds 10/3
Correction . . . Wednesday 10/3! (sometimes I still think it's 2011) jg On Fri, 28 Sep 2012, Jeff Green wrote: I'm planning to deploy Sender Policy Framework (SPF) for the wikimedia.org domain on Weds October 5. SPF is a framework for validating outgoing mail, which gives the receiving side useful information for spam filtering. The main goal is to cause spoofed @wikimedia.org mail to be correctly identified as such. It should also improve our odds of getting fundraiser mailings into inboxes rather than spam folders. The change should not be noticeable, but the most likely problem would be legitimate @wikimedia.org mail being treated as spam. If you hear of this happening please let me know. Technical details are below for anyone interested . . . Thanks, jg Jeff Green Operations Engineer, Special Projects Wikimedia Foundation 149 New Montgomery Street, 3rd Floor San Francisco, CA 94105 jgr...@wikimedia.org . . . . . . . SPF overview http://en.wikipedia.org/wiki/Sender_Policy_Framework The October 8 change will be simply a matter of adding a TXT record to the wikimedia.org DNS zone: wikimedia.org IN TXT v=spf1 ip4:91.198.174.0/24 ip4:208.80.152.0/22 ip6:2620:0:860::/46 include:_spf.google.com ip4:74.121.51.111 ?all The record is a list of subnets that we identify as senders (all wmf subnets, google apps, and the fundraiser mailhouse). The ?all is a neutral policy--it doesn't state either way how mail should be handled. Eventually we'll probably bump ?all to a stricter ~all aka SoftFail, which tells the receiving side that only mail coming from the listed subnets is valid. Most ISPs will route 'other' mail to a spam folder based on SoftFail. Please bug me with any questions/comments! ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] SPF (email spoof prevention feature) test-rollout Weds 10/5
On Fri, 28 Sep 2012, Daniel Friesen wrote: On Fri, 28 Sep 2012 12:47:20 -0700, Jeff Green jgr...@wikimedia.org wrote: On Fri, 28 Sep 2012, Daniel Friesen wrote: On Fri, 28 Sep 2012 12:19:21 -0700, Brad Jorsch b-jor...@alum.northwestern.edu wrote: On Fri, Sep 28, 2012 at 11:00:08AM -0700, Jeff Green wrote: The change should not be noticeable, but the most likely problem would be legitimate @wikimedia.org mail being treated as spam. If you hear of this happening please let me know. Anyone who sends all mail marked as from[1] their @wikimedia.org address through Gmail's SMTP server, through an SMTP server hosted by Wikimedia (is there one?), or through any other server identified in the SPF record should be fine. And anyone who isn't sending from an @wikimedia.org address should be entirely unaffected. If anyone is sending mail marked as from their @wikimedia.org address through some other SMTP server (e.g. through their home ISP), they might start to see trouble with this change and likely will when the SPF record is changed to ~all. Also, any recipient who has their mail forwarded might have trouble *receiving* messages from @wikimedia.org addresses, unless their forwarding service takes SPF into account or their destination mailbox doesn't check SPF. OTOH, these people would have the same problem with receiving mail from all the other domains that currently implement SPF. [1]: There are actually two concepts of from involved in email. The first, the envelope sender or mail from, is the address that bounce notifications should be sent to. The second is the address that actually shows up as From: in the email message. SPF is intended to target only the former, but SenderID hijacks the SPF specification to also test the latter. And to make things all fun and confusing. We shouldn't forget about the Sender: header... **mumbles about AWS-SES not supporting Sender:** Yes and SenderID is where we're running into deliverability issues for fundraiser mailings since we lack SPF, that's part of what prompted this whole initiative. Well, that and an ancient RT request from Office IT! T_T Not my complaint about From: @wikimedia.org spam on wikitech-l? That too! ;-) -- ~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://daniel.friesen.name] ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l -- ~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://daniel.friesen.name] ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Work offer inside
I still think it's a very good idea to deploy both SPF and domainkeys. SPF keeps coming up--twice this week from completely different quarters. Today the mailhouse we hired to help with the fundraiser tells me our deliverability with one major ISP is poor because we lack SPF. We are currently stuck at the step of mapping out how we originate mail for the whitelist. Production and Google Apps mail are easy. But people say we may have volunteers, board members, etc. who do not use our known mail routes. I think OIT is in the best position to sort that out. They're the go-to for mail client setup and can survey any outliers. I spoke to Andrew about it in June and he was up for it but felt it needs to be approved and prioritized by managers. jg On Fri, 31 Aug 2012, Tim Starling wrote: On 31/08/12 04:15, Daniel Friesen wrote: This brings up the question. Why does wikimedia.org not have a SPF record? We should be rejecting wikimedia.org emails that we know do not come from Wikimedia. In May, Jeff Green proposed deploying it with softfail, but it wasn't ever actually done. Nobody wanted to use a fail qualifier, due to the risk of legitimate mail not being delivered. So even if he had deployed it, it probably wouldn't have helped in this case. Mailman's security weaknesses are inherent to the protocol it uses, there's no way to repair it. The scam email could have been sent with a From header copied from anyone who has posted to the list recently. In the unlikely event that SPF fail was used for that sender and the receiver respected it, the scammer could have just picked again. We should use a web interface for posting to groups, web interfaces can be password protected without breaking 99% of clients. I removed bo...@wikimedia.org from the list of email addresses that are allowed to post to the list without being subscribed. -- Tim Starling ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Work offer inside
that should read survey and reconfigure any outliers On Fri, 31 Aug 2012, Jeff Green wrote: I still think it's a very good idea to deploy both SPF and domainkeys. SPF keeps coming up--twice this week from completely different quarters. Today the mailhouse we hired to help with the fundraiser tells me our deliverability with one major ISP is poor because we lack SPF. We are currently stuck at the step of mapping out how we originate mail for the whitelist. Production and Google Apps mail are easy. But people say we may have volunteers, board members, etc. who do not use our known mail routes. I think OIT is in the best position to sort that out. They're the go-to for mail client setup and can survey any outliers. I spoke to Andrew about it in June and he was up for it but felt it needs to be approved and prioritized by managers. jg On Fri, 31 Aug 2012, Tim Starling wrote: On 31/08/12 04:15, Daniel Friesen wrote: This brings up the question. Why does wikimedia.org not have a SPF record? We should be rejecting wikimedia.org emails that we know do not come from Wikimedia. In May, Jeff Green proposed deploying it with softfail, but it wasn't ever actually done. Nobody wanted to use a fail qualifier, due to the risk of legitimate mail not being delivered. So even if he had deployed it, it probably wouldn't have helped in this case. Mailman's security weaknesses are inherent to the protocol it uses, there's no way to repair it. The scam email could have been sent with a From header copied from anyone who has posted to the list recently. In the unlikely event that SPF fail was used for that sender and the receiver respected it, the scammer could have just picked again. We should use a web interface for posting to groups, web interfaces can be password protected without breaking 99% of clients. I removed bo...@wikimedia.org from the list of email addresses that are allowed to post to the list without being subscribed. -- Tim Starling ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Work offer inside
On Fri, 31 Aug 2012, Mark A. Hershberger wrote: On 08/31/2012 04:01 PM, Jeff Green wrote: We are currently stuck at the step of mapping out how we originate mail for the whitelist. Production and Google Apps mail are easy. But people say we may have volunteers, board members, etc. who do not use our known mail routes. I'm not why you couldn't give volunteers, etc. a server to send from and add that IP to your trusted senders for the domain that they use (assuming they're using one of your domains for their email address). Andrew suggested giving them Google apps accounts. I think it's a great solution--it allows people to use gmail or pretty much any mail client they want. This does seem like an OIT function instead of an Operations one, but it seems very doable. Thunderbird, at least, supports different servers per-identity. Mark. -- http://hexmode.com/ Human evil is not a problem. It is a mystery. It cannot be solved. -- When Atheism Becomes a Religion, Chris Hedges ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
