Re: IPv6 isn't SMTP
Barry Shein wrote the following on 3/27/2014 6:32 PM: On March 27, 2014 at 14:16 bl...@ispn.net (Blake Hudson) wrote: Barry Shein wrote the following on 3/27/2014 2:06 PM: I suppose the obvious question is: What's to stop a spammer from putting a totally legitimate key into their spam? It's entirely likely that a spammer would try to get a hold of a key due to its value or that someone you've done business with would share keys with a business partner . But ideally you'd authorize each sender with a unique key (or some sort of pair/combination). So that 1) you can tell who the spammer sourced the key from and 2) you can revoke the compromised key's authorization to send you subsequent email messages. There's probably some way to generate authorization such that each sender gets a unique key or a generic base is in some way salted or combined with information from the individual you're giving your authorization to such that the result is both unique and identifiable. Ok, this is a form of whitelisting with some authentication using public key technology. Sure. But is this really the problem you run into much? Someone impersonating a sender you consider whitelisted? I'm sure it happens. But at a systems level I think most of us are talking about the much more nefarious non-stop fire-hose of pure sewage. Some white list, but for many that runs too great a risk of rejecting serendipity, that great job offer from someone who was impressed by a post you made on NANOG, etc. So we get Challenge-Response etc as a workaround, which also has problems. Well, whatever, SPAM IS A BIG SUBJECT and there are a lot of perspectives. P.S. I always figured the problem you describe could be very trivially solved by just agreeing to stick some word in the header like: X-PassCode: swordfish It's not like anyone but the sender is likely to know that unless they really are in your mail stream in which case you have other problems. It would be nice if that were automated but it could be done manually. I have certain Subject: phrases I use with people, some funny, so they know it's almost certainly me. You're on the right track with what I was proposing. While spoofing can be addressed, it's not the primary goal. The idea was to verify whether an incoming email was authorized or not. Authentication is just a prereq to that. It is up to the recipient to choose what to do with unauthorized mail: Treat them the same as any other, tag them, put them in a separate folder or quarantine, reject them, or send them to the bit bucket. This may be a list, but I wouldn't consider a whitelist unless implemented as such by the user/client.
Re: IPv6 isn't SMTP
On Mar 26, 2014, at 8:12 PM, Robert Drake rdr...@direcpath.com wrote: On 3/26/2014 10:16 PM, Franck Martin wrote: and user@2001:db8::1.25 with user@192.0.2.1:25. Who had the good idea to use : for IPv6 addresses while this is the separator for the port in IPv4? A few MTA are confused by it. At the network level the IPv6 address is just a big number. No confusion there. At the plaintext level the naked IPv6 address should be wrapped in square brackets. From: http://tools.ietf.org/html/rfc3986#section-3.2.2 Two errors, actually… As an RFC-821 address, it should be user@[IP]:port in both cases (user@[192.0.2.1]:25 and user@[2001:db8::1]:25). Owen
Re: IPv6 isn't SMTP
On Mar 26, 2014, at 11:26 PM, Owen DeLong o...@delong.com wrote: On Mar 26, 2014, at 8:12 PM, Robert Drake rdr...@direcpath.com wrote: On 3/26/2014 10:16 PM, Franck Martin wrote: and user@2001:db8::1.25 with user@192.0.2.1:25. Who had the good idea to use : for IPv6 addresses while this is the separator for the port in IPv4? A few MTA are confused by it. At the network level the IPv6 address is just a big number. No confusion there. At the plaintext level the naked IPv6 address should be wrapped in square brackets. From: http://tools.ietf.org/html/rfc3986#section-3.2.2 Two errors, actually… As an RFC-821 address, it should be user@[IP]:port in both cases (user@[192.0.2.1]:25 and user@[2001:db8::1]:25). indeed, but MTAs are know to accept any kind of non RFC compliant emails and trying to make some sense out of it… :P see http://tools.ietf.org/html/rfc7103 which tries to address some of it in a more deterministic way. signature.asc Description: Message signed with OpenPGP using GPGMail
Re: IPv6 isn't SMTP
On Mar 27, 2014, at 3:24 AM, Franck Martin fmar...@linkedin.com wrote: On Mar 26, 2014, at 11:26 PM, Owen DeLong o...@delong.com wrote: On Mar 26, 2014, at 8:12 PM, Robert Drake rdr...@direcpath.com wrote: On 3/26/2014 10:16 PM, Franck Martin wrote: and user@2001:db8::1.25 with user@192.0.2.1:25. Who had the good idea to use : for IPv6 addresses while this is the separator for the port in IPv4? A few MTA are confused by it. At the network level the IPv6 address is just a big number. No confusion there. At the plaintext level the naked IPv6 address should be wrapped in square brackets. From: http://tools.ietf.org/html/rfc3986#section-3.2.2 Two errors, actually… As an RFC-821 address, it should be user@[IP]:port in both cases (user@[192.0.2.1]:25 and user@[2001:db8::1]:25). indeed, but MTAs are know to accept any kind of non RFC compliant emails and trying to make some sense out of it… :P see http://tools.ietf.org/html/rfc7103 which tries to address some of it in a more deterministic way. Sure, but that doesn’t mean we should be sending random garbage deliberately. Owen
Re: IPv6 isn't SMTP
Jimmy Hess wrote the following on 3/26/2014 7:12 PM: The problem is with SMTP and is probably best addressed in the application layer through updates to SMTP or required bolt-ons (e.g SPF or similar); it was just simpler SPF is useful, but not a complete solution. I'm curious what form you think these updates to SMTP would take? What changes to SMTP protocol itself could really have a meaningful impact, without frustrating, confusing, or terribly complicating the lives of millions of mail users? The primary issues I see with SMTP as a protocol related to the lack of authentication and authorization. Take, for instance, the fact that the SMTP protocol requires a mail from: and rcpt to: address (more or less for authentication and authorization purposes), but then in the message allows the sender to specify a completely different set of sender and recipient information that gets displayed in the mail client. SMTP is simple, it is reliable, and it works well for delivering a message, but it does not address authentication or authorization of remote users (to my knowledge). An extension to SMTP that provides some form of authentication and authorization for remote users could address the abuse concerns. For example, one could utilize a public key infrastructure through previously shared vcard or similar contact information to both authenticate and authorize a sender to be allowed to send email to a recipient. There are probably a few ways to accomplish the same goal, a PKI is just one example. This would allow one to configure his or her email as either 'default allow' to receive email from anyone at any time or 'default deny' to only allow authorized senders. There are some folks doing this today outside of SMTP, but without a mass effort these schemes will probably not take a strong foothold. Implementing something like this takes some work in the mail client to be able to generate a private key and hold the public key info of others. Realistically, the key information could be exchanged and verified simply between clients at the remote ends without any SMTP server involvement, but that seems like a waste for servers to process and store messages that are going to be junked in the end that it would make sense that the SMTP servers could also be able to make these decisions server side. On the other hand, putting spam filtering in the hands of the end users could really untie the server operators from costly or cumbersome anti-SPAM efforts. Any open source email clients want to take this task on and build an industry consensus? I'm all for having my emails tagged as trusted/authorized or untrusted/unauthorized in my email client. Once the level of untrusted, yet legitimate, messages drops to a low enough level I can stop accepting untrusted messages altogether. --Blake
Re: IPv6 isn't SMTP
John Levine jo...@iecc.com wrote: There are also some odd things in the spec. For example, according to RFC 5321 this is not a syntactically valid e-mail address: mailbox@[IPv6:2001:12:34:56::78:ab:cd] You aren't allowed to use :: to abbreviate one zero hexadectet according to RFC 5952. http://www.rfc-editor.org/errata_search.php?eid=2467 Tony. -- f.anthony.n.finch d...@dotat.at http://dotat.at/ Malin: East 5 or 6. Moderate or rough, occasionally very rough in northwest. Showers. Good, occasionally moderate.
Re: IPv6 isn't SMTP
Owen DeLong o...@delong.com wrote: Two errors, actually… As an RFC-821 address, it should be user@[IP]:port in both cases (user@[192.0.2.1]:25 and user@[2001:db8::1]:25). You have never been able to specify a port number in an email address. Tony. -- f.anthony.n.finch d...@dotat.at http://dotat.at/ Lundy, Fastnet: East or northeast 4 or 5, occasionally 6 later. Moderate becoming rough in south. Thundery showers. Good, occasionally poor.
Re: IPv6 isn't SMTP
Hi, On Thu, Mar 27, 2014 at 01:52:27PM +, Tony Finch wrote: John Levine jo...@iecc.com wrote: There are also some odd things in the spec. For example, according to RFC 5321 this is not a syntactically valid e-mail address: mailbox@[IPv6:2001:12:34:56::78:ab:cd] You aren't allowed to use :: to abbreviate one zero hexadectet according to RFC 5952. well, RFC 5952 _recommends_ against using that. Still, it's perfectly valid as of RFC 4291 and the approach can be found in quite large vendors' implementations, see http://labs.apnic.net/blabs/?p=309. RFC 5952 explicitly states: all implementations must accept and be able to handle any legitimate RFC 4291 format. best Enno http://www.rfc-editor.org/errata_search.php?eid=2467 Tony. -- f.anthony.n.finch d...@dotat.at http://dotat.at/ Malin: East 5 or 6. Moderate or rough, occasionally very rough in northwest. Showers. Good, occasionally moderate. -- Enno Rey ERNW GmbH - Carl-Bosch-Str. 4 - 69115 Heidelberg - www.ernw.de Tel. +49 6221 480390 - Fax 6221 419008 - Cell +49 173 6745902 Handelsregister Mannheim: HRB 337135 Geschaeftsfuehrer: Enno Rey === Blog: www.insinuator.net || Conference: www.troopers.de Twitter: @Enno_Insinuator ===
Re: IPv6 isn't SMTP
mailbox@[IPv6:2001:12:34:56::78:ab:cd] You aren't allowed to use :: to abbreviate one zero hexadectet according to RFC 5952. http://www.rfc-editor.org/errata_search.php?eid=2467 Oh, look at that. I wonder how many people realized that it made an incompatible change to RFC 4291 four years ago. I certainly didn't. I wonder what problem they thought they were solving. Regards, John Levine, jo...@iecc.com, Primary Perpetrator of The Internet for Dummies, Please consider the environment before reading this e-mail. http://jl.ly
Re: IPv6 isn't SMTP
On 03/26/2014 08:12 PM, Jimmy Hess wrote: As far as i'm concerned if you can force the spammer to use their own IP range, that they can setup RDNS for, then you have practically won, for all intents and purposes, as it makes blacklisting feasible, once again! Spammers can jump through these hoops --- but spammers aren't going to effectively scale up their spamming operation by using IP address ranges they can setup RDNS on. Tell that to the 100,000+ e-mails I blocked last week (and the several hundred that got through before I was able to get all the blocks entered into my ingress ACLs) from proper rDNS addresses where the addresses were hopping all over a /24, a /22, three /21's, four /20's, and six /19s in widely separated blocks. Every single address in those blocks eventually attempted to send e-mail, and every address had proper rDNS for the pseudorandom domain names, mostly in the .in TLD, but some others, too (the blocks were all over, with some registed through ARIN, some through RIPE, some through AfriNIC, and some through APNIC, with hosters in Europe, North and South America, Asia, and Africa.) Note that these passed full FCrDNS verification in postfix. They all had very similar characteristics, including an embedded image payload/ad and a couple of hundred kB of anti-Bayesian text, including the full text of Zilog's Z80 manual at one point. Of course, the other tens of thousands per day that get blocked for not having rDNS from residential bots make the case for leaving rDNS (and the FCrDNS variant) turned on, but it is not a cure-all.
Re: IPv6 isn't SMTP
On March 27, 2014 at 08:51 bl...@ispn.net (Blake Hudson) wrote: The primary issues I see with SMTP as a protocol related to the lack of authentication and authorization. Take, for instance, the fact that the SMTP protocol requires a mail from: and rcpt to: address (more or less for authentication and authorization purposes), but then in the message allows the sender to specify a completely different set of sender and recipient information that gets displayed in the mail client. This is mostly a UI issue. The user interface could show anything, custom certainly has been as you describe: Show the message From: and make it tricky (for most people) to get the envelope info. Well, it's not mandatory that an MTA transmit the envelope info into the message headers and, almost worse, different MTAs seem to use different header fields for this. For example in SpamAssassin you are encouraged to set which field it should look at for the envelope sender. But that's not REALLY a problem with SMTP per se. Only in practice, if that's a useful distinction. I won't go point by point but I will say that SMTP has been extended several times -- just throw another verb into the mix to extend it. Which is a very useful observation. SMTP also can transmit which verbs are supported. One can extend a new idea and it's immediately interoperable. I suppose the obvious question is: What's to stop a spammer from putting a totally legitimate key into their spam? You also need some sort of reputation layer. Or maybe that makes it unworkable. I remember at the 2006 MIT Spam Conference where Eric Allman and I were keynotes we got into a bit of a tussle over exactly this during his question period. It was...amusing! -- -Barry Shein The World | b...@theworld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: 800-THE-WRLD| Dial-Up: US, PR, Canada Software Tool Die| Public Access Internet | SINCE 1989 *oo*
Re: IPv6 isn't SMTP
Barry Shein wrote the following on 3/27/2014 2:06 PM: I suppose the obvious question is: What's to stop a spammer from putting a totally legitimate key into their spam? It's entirely likely that a spammer would try to get a hold of a key due to its value or that someone you've done business with would share keys with a business partner . But ideally you'd authorize each sender with a unique key (or some sort of pair/combination). So that 1) you can tell who the spammer sourced the key from and 2) you can revoke the compromised key's authorization to send you subsequent email messages. There's probably some way to generate authorization such that each sender gets a unique key or a generic base is in some way salted or combined with information from the individual you're giving your authorization to such that the result is both unique and identifiable. --Blake
Re: IPv6 isn't SMTP
Barry Shein wrote the following on 3/26/2014 11:24 PM: Some will blanche at this but the entire spam problem basically arose from the crap security in Windows systems, particularly prior to maybe XP/SP2. Not sure where all that leads us, however. Better security at those major exploitation points, in a nutshell. And if someone disagrees then please tell me how spammers as we know them (and related miscreants) can operate without these few sources of purloined resources. Preferably without a big hand-wave like oh they'll just find something else! Maybe not! You're largely right. Botnets are a big source of spam. As a mail server operator, they're the biggest source that I see. They're also easy to block through a number of means (The ISPs they're located on often block port 25, PBL (or similar), rDNS, and other behavior). It sounds like it will likely be a similar matter of blocking residential botnet participants on IPv6 due to the fact that residential ISPs will likely apply similar port 25 policy to IPv6 as they do to IPv4 and no rDNS. However, as more attention is being payed to secure these end stations, spammers are looking at alternative avenues. In recent years, they've been harvesting user credentials through various means and then exploiting these compromised accounts to send email through otherwise legitimate servers. These are the spam messages that are hard to block. And these may be the areas where reputation based services will not be able to keep up in an IPv6 landscape. At least this concentrates the sources of spam (from my server's vantage point) and reduces the attack surface so that the problem is likely addressed more quickly and by someone with a higher level of knowledge than the average (unknowing) botnet participant. Unfortunately, I can't keep Suzie teenager or Joe grandpa from giving his or her password out to a phisher. Fortunately, I can place reasonable limits on their accounts and the number of messages they're allowed to send or the rate at which they're allowed to send messages. If everyone else would just do the same we'd be a lot better off against this kind of attack. --Blake
Re: IPv6 isn't SMTP
On March 27, 2014 at 14:16 bl...@ispn.net (Blake Hudson) wrote: Barry Shein wrote the following on 3/27/2014 2:06 PM: I suppose the obvious question is: What's to stop a spammer from putting a totally legitimate key into their spam? It's entirely likely that a spammer would try to get a hold of a key due to its value or that someone you've done business with would share keys with a business partner . But ideally you'd authorize each sender with a unique key (or some sort of pair/combination). So that 1) you can tell who the spammer sourced the key from and 2) you can revoke the compromised key's authorization to send you subsequent email messages. There's probably some way to generate authorization such that each sender gets a unique key or a generic base is in some way salted or combined with information from the individual you're giving your authorization to such that the result is both unique and identifiable. Ok, this is a form of whitelisting with some authentication using public key technology. Sure. But is this really the problem you run into much? Someone impersonating a sender you consider whitelisted? I'm sure it happens. But at a systems level I think most of us are talking about the much more nefarious non-stop fire-hose of pure sewage. Some white list, but for many that runs too great a risk of rejecting serendipity, that great job offer from someone who was impressed by a post you made on NANOG, etc. So we get Challenge-Response etc as a workaround, which also has problems. Well, whatever, SPAM IS A BIG SUBJECT and there are a lot of perspectives. P.S. I always figured the problem you describe could be very trivially solved by just agreeing to stick some word in the header like: X-PassCode: swordfish It's not like anyone but the sender is likely to know that unless they really are in your mail stream in which case you have other problems. It would be nice if that were automated but it could be done manually. I have certain Subject: phrases I use with people, some funny, so they know it's almost certainly me. -- -Barry Shein The World | b...@theworld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: 800-THE-WRLD| Dial-Up: US, PR, Canada Software Tool Die| Public Access Internet | SINCE 1989 *oo*
Re: IPv6 isn't SMTP
On Mar 27, 2014, at 12:16 PM, Blake Hudson bl...@ispn.net wrote: It's entirely likely that a spammer would try to get a hold of a key due to its value or that someone you've done business with would share keys with a business partner . But ideally you'd authorize each sender with a unique key (or some sort of pair/combination). So that 1) you can tell who the spammer sourced the key from and 2) you can revoke the compromised key's authorization to send you subsequent email messages. There's probably some way to generate authorization such that each sender gets a unique key or a generic base is in some way salted or combined with information from the individual you're giving your authorization to such that the result is both unique and identifiable. (Not to single you out, but this is a good entry point.) So somewhere between this and the “every user should have their own MTA” idea, something would need to be done to close the end user usability gap. - “I just bought something from this boutique website, how do I (or my ISP) know how to let them email me my receipt?” - “My friend gave his buddy my email address to send a resume for that job opening I have. How do I permit him to send me email?” - “This .gov entity needs to email me about my (taxes|health care|car registration|…), how do I give them permission?” - “My long lost high school friend found my email address somewhere (and isn’t using gmail, hotmail, yahoo, ….), how do I keep her from getting blocked?” All of these end-user questions will have to be answered by any such technology which seeks to solve the spam problem using a manner such as you describe here. And if you’re going to say the solution is “in addition to my email address, in order to send me mail someone is going to have to know my (key|pass phrase|…)” then anything which currently collects your email address is also going to need to collect “that”. Therefore how do you control “that” from getting in the wrong hands in that list of emails someone is selling to spammers? Am I misunderstanding what’s being proposed here? To me the ubiquity of email is its own undoing — it’s so convenient because you can email anybody, anywhere, from anywhere, but it’s so spammable because you can email anybody, anywhere, from anywhere. -c
Re: IPv6 isn't SMTP
On 3/27/2014 6:51 AM, Blake Hudson wrote: The primary issues I see with SMTP as a protocol related to the lack of authentication and authorization. Take, for instance, the fact that the SMTP protocol requires a mail from: and rcpt to: address (more or less for authentication and authorization purposes), Actually, for neither. Mail from was mislabeled; it merely provides an address to send return notices to, which is why it makes sense to permit it to be different from the rfc5322.From value. And, of course, rcpt to specifies a recipient address. auth/auth functions were tacked on much, much later, which is why their utility is so constrained. (20 years?) but then in the message allows the sender to specify a completely different set of sender and recipient information that gets displayed in the mail client. Yeah. Almost like it is approximating the difference between what is on the outside of a postal envelope versus what is on the letterhead and opening of a piece of paper mail, which also permit such independence... The essential problem with seeing these as 'problems' is confusing 'common' with 'required'. Common scenarios are fine, but so are the variants. The variants often blow apart the simplifying assumption that one can incorrectly believe from the common scenarios. d/ -- Dave Crocker Brandenburg InternetWorking bbiw.net
Re: IPv6 isn't SMTP
On 03/25/2014 11:18 PM, John Levine wrote: 3. Arguing about IPv6 in the context of requirements upon SMTP connections is playing that uncomfortable game with one�s own combat boots. And not particularly productive. If you can figure out how to do effective spam filtering without looking at the IP addresses from which mail arrives, you will be in a position to make a whole lot of money. But, as always, I'm not holding my breath. R's, John PS: Note the word effective. You look at the IP, and verify forward and reverse DNS. IPv6 doesn't make this any harder a problem than IPv4, it just means that we're going to *have* to reject mail that comes in from IPv6 addresses that don't have clean DNS. -- Daniel Taylor VP OperationsVocal Laboratories, Inc. dtay...@vocalabs.com http://www.vocalabs.com/(612)235-5711
Re: IPv6 isn't SMTP
On Wed, 26 Mar 2014 07:45:06 -0500 Daniel Taylor dtay...@vocalabs.com wrote: On 03/25/2014 11:18 PM, John Levine wrote: 3. Arguing about IPv6 in the context of requirements upon SMTP connections is playing that uncomfortable game with one�s own combat boots. And not particularly productive. If you can figure out how to do effective spam filtering without looking at the IP addresses from which mail arrives, you will be in a position to make a whole lot of money. But, as always, I'm not holding my breath. R's, John PS: Note the word effective. You look at the IP, and verify forward and reverse DNS. IPv6 doesn't make this any harder a problem than IPv4, it just means that we're going to *have* to reject mail that comes in from IPv6 addresses that don't have clean DNS. -- Daniel Taylor VP OperationsVocal Laboratories, Inc. dtay...@vocalabs.com http://www.vocalabs.com/ (612)235-5711 Actually, with all the discussion about ipv6 not having rDNS, in most cases, would that not make things easier? So those that want to run email servers SHOULD be on ISP's that allow for rDNS configuration for IPv6. There should be some vetting in the process by the ISP, maybe, before allowing this. So in essence, if you are a legitimate email host, you will have rDNS configured on IPv6 for your server. Again, as others have stated, rDNS should NOT be the only deciding factor in whether or not an email is legit. No rDNS, or havinf rDNS, should have some weight assigned to it for the overall evaluation of the sender. Robert
Re: IPv6 isn't SMTP
On Wed, Mar 26, 2014 at 09:05:52AM -0400, rw...@ropeguru.com wrote: most cases, would that not make things easier? So those that want to run email servers SHOULD be on ISP's that allow for rDNS configuration for IPv6. Several years ago now the IETF DNSOP WG worked on a document about reverse mapping. The topic was so controversial that the final version of the document, which said little more than, There is a reverse tree, and you might want to take that into consideration. Or not. It's up to you, still couldn't get consensus. I think anything that involves making rules about others' reverse maps is probably an excellent way to attract ducks to nibble you to death. A -- Andrew Sullivan Dyn, Inc. asulli...@dyn.com v: +1 603 663 0448
Re: IPv6 isn't SMTP
On 03/26/2014 08:05 AM, rw...@ropeguru.com wrote: On Wed, 26 Mar 2014 07:45:06 -0500 Daniel Taylor dtay...@vocalabs.com wrote: On 03/25/2014 11:18 PM, John Levine wrote: 3. Arguing about IPv6 in the context of requirements upon SMTP connections is playing that uncomfortable game with one�s own combat boots. And not particularly productive. If you can figure out how to do effective spam filtering without looking at the IP addresses from which mail arrives, you will be in a position to make a whole lot of money. But, as always, I'm not holding my breath. R's, John PS: Note the word effective. You look at the IP, and verify forward and reverse DNS. IPv6 doesn't make this any harder a problem than IPv4, it just means that we're going to *have* to reject mail that comes in from IPv6 addresses that don't have clean DNS. -- Daniel Taylor VP OperationsVocal Laboratories, Inc. dtay...@vocalabs.com http://www.vocalabs.com/ (612)235-5711 Actually, with all the discussion about ipv6 not having rDNS, in most cases, would that not make things easier? So those that want to run email servers SHOULD be on ISP's that allow for rDNS configuration for IPv6. There should be some vetting in the process by the ISP, maybe, before allowing this. So in essence, if you are a legitimate email host, you will have rDNS configured on IPv6 for your server. Again, as others have stated, rDNS should NOT be the only deciding factor in whether or not an email is legit. No rDNS, or havinf rDNS, should have some weight assigned to it for the overall evaluation of the sender. Robert If you can't get rDNS on a mail host from your ISP, I'd say you are on the wrong ISP if you want to run your own mail server. This goes for IPv6 and IPv4 equally. -- Daniel Taylor VP OperationsVocal Laboratories, Inc. dtay...@vocalabs.com http://www.vocalabs.com/(612)235-5711
Re: IPv6 isn't SMTP
On March 25, 2014 at 23:33 larryshel...@cox.net (Larry Sheldon) wrote: Is spam fighting really about SMTP? Or is it about abuse of the transport layer by (among other things) the SMTP? That is the point, isn't it. Most see spam as its content. The real problem with spam is its volume. Without the volume, some bot operators probably send on the order of a billion messages per day, it wouldn't be much of a problem. What makes that volume possible and pervasive is IP address mobility. Otherwise we'd just block the offending IPs and be done with it, to some extent -- I have a newer view on that but it'd be distracting. What makes IP address mobility possible is mass, unauthorized if not simply illegal use of others' resources, such as with botnets or massive exploiting of holes in web hosting sites' software. Fundamentally spam is a security isse. A spammer's stock in trade is the massive, free use of IP address and bandwidth resources. That the content is unwanted is almost incidental to this fact. -- -Barry Shein The World | b...@theworld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: 800-THE-WRLD| Dial-Up: US, PR, Canada Software Tool Die| Public Access Internet | SINCE 1989 *oo*
Re: IPv6 isn't SMTP
Daniel Taylor wrote the following on 3/26/2014 7:45 AM: On 03/25/2014 11:18 PM, John Levine wrote: 3. Arguing about IPv6 in the context of requirements upon SMTP connections is playing that uncomfortable game with one�s own combat boots. And not particularly productive. If you can figure out how to do effective spam filtering without looking at the IP addresses from which mail arrives, you will be in a position to make a whole lot of money. But, as always, I'm not holding my breath. R's, John PS: Note the word effective. You look at the IP, and verify forward and reverse DNS. IPv6 doesn't make this any harder a problem than IPv4, it just means that we're going to *have* to reject mail that comes in from IPv6 addresses that don't have clean DNS. With this in mind, how hard is it for a spamming operation to setup rDNS for their IPv6 ranges? Not very hard, just like their ability to use SPF or DKIM (they will do it if it improves their deliverability). This is separate from infected bots on residential connections, which is effectively dealt with today through the PBL or some basic rDNS checks since practically everyone has rDNS in IPv4. Having rDNS doesn't automatically make you a good guy. SMTP operators will still have the need for reputation based services. Due to the design of the SMTP protocol (which provides little protection for abuse on its own), people chose to place additional checks at L3. I see this as a deficiency in SMTP that we were fortunate enough to solve in an IPv4 landscape (after a lot of initial concern about RBLs and their operators), but is going to be harder to to utilize the same tool as effectively in IPv6. The problem is with SMTP and is probably best addressed in the application layer through updates to SMTP or required bolt-ons (e.g SPF or similar); it was just simpler for most folks to address it on their own at L3 rather than trying to get everyone to agree on a new/better way to send email messages. --Blake
Re: IPv6 isn't SMTP
On 3/25/2014 10:41 PM, Jimmy Hess wrote: (1) Architectural layers are a protocol design construction, only, which assist with standardization. They are not a separation of responsibilities. Actually, they are specifically a separation of responsibilities. That the separation doesn't work adequately and all the time means that pragmatics requires wandering across layer boundaries. But pragmatics also dictate being extremely careful when choosing to do that. A SMTP server has to take care to have an implementation of all layers. There are two possible meanings to that sentence. The first prompts a 'duh' reaction, since SMTP (usually) runs over TCP/IP. So the server won't be very useful unless it 'has' an implementation of all layers. The other interpretation is that an SMTP package needs to have its own version of TCP and IP. This, of course, is silliness. SMTP packages do not typically implement TCP or IP. They might pay quite a lot of attention to those lower layers, but they don't implement them. (2) The IP protocol layer is not actually independent. Moving to IPv6 does in fact have effective new requirements for SMTP servers. Mostly, no. Not completely no, of course, but mostly. Language like #2 is leading quite a few folk to try to treat email over IPv6 as if it will be a separate service from the one currently running over v4. It won't be a separate service. Or, at least, it has better not be. The success of email has been through seamless, end-to-end integration, not through disparate silos with different service models. By way of example, please highlight which email packages require or even allow an author to dictate which version of IP to send over. For anything that someone thinks should be done for v6, pursue it instead as a change for the entire global service. It's fine for v6-related issues to /motivate/ global changes, but don't isolate the work to v6 platforms. (4) When a major change will [by necessity] be made to any layer underlying the MTA application on the protocol stack, now is also a good time to look at the overall service as a whole. Sure, but not 'also'. Rather 'only', except for relatively trivial layer convergence gluing. d/ -- Dave Crocker Brandenburg InternetWorking bbiw.net
Re: IPv6 isn't SMTP
On Wed, Mar 26, 2014 at 5:16 PM, Blake Hudson bl...@ispn.net wrote: With this in mind, how hard is it for a spamming operation to setup rDNS for their IPv6 ranges? Not very hard, just like their ability to use SPF or DKIM (they will do it if it improves their deliverability). This is separate from infected bots on residential connections, which is effectively dealt with today through the PBL or some basic rDNS checks since practically everyone has rDNS in IPv4. [snip] There are plenty of residential connections, and enterprise non-mail server connections, that PBL is no good for. As far as i'm concerned if you can force the spammer to use their own IP range, that they can setup RDNS for, then you have practically won, for all intents and purposes, as it makes blacklisting feasible, once again! Spammers can jump through these hoops --- but spammers aren't going to effectively scale up their spamming operation by using IP address ranges they can setup RDNS on. I would argue that less than 100% of spammers for sure would make the shift from compromising as many IP addresses as possible and turning them into mail servers. Following thatwhat's really left is: misconfigured mail servers allowing open relays, and mail users that fall to phishing or pick easy to guess passwords. (Spammer intruding upon and co-oping mailboxes that are legitimate in the first place.) Having rDNS doesn't automatically make you a good guy. SMTP operators will still have the need for reputation based services. Due to the design of the SMTP protocol Indeed it doesn't, but it's highly suggestive.*I would rather rDNS be augmented with a registration of intent to run a mail server that can verifiably be linked to whomever is authorized contact for the IP space. (which provides little protection for abuse on its own), people chose to place additional checks at L3. I see this as a deficiency in SMTP that we were fortunate enough to solve in an IPv4 landscape (after a lot of initial concern about RBLs and their operators), but is going to be harder to to utilize the same tool as effectively in IPv6. It will be immensly harder. The problem is with SMTP and is probably best addressed in the application layer through updates to SMTP or required bolt-ons (e.g SPF or similar); it was just simpler SPF is useful, but not a complete solution. I'm curious what form you think these updates to SMTP would take? What changes to SMTP protocol itself could really have a meaningful impact, without frustrating, confusing, or terribly complicating the lives of millions of mail users? --Blake -- -JH
Re: IPv6 isn't SMTP
On Mar 25, 2014, at 8:31 PM, Cutler James R james.cut...@consultant.com wrote: 3. Arguing about IPv6 in the context of requirements upon SMTP connections is playing that uncomfortable game with one’s own combat boots. And not particularly productive. That is one of my two big take-aways from this conversation. The other is that operators of SMTP MTAs should implement RDNS for them, which I thought we already knew. To my knowledge, there are three impacts that IPv6 implementation makes on an SMTP implementation. One is that the OS interface to get the address of the next MUA or MTA needs to use getaddrinfo() instead of gethostbyname() (and would do well to observe RFC 6555’s considerations). Another is that, whether on an incoming or an outbound connection, when the application gets its own address from the OS (binary or as a character string), it needs to allocate more storage for the data structure. The third is that it needs to be able to interpret user@2001:db8::1 as well as user@dns-name and user@192.0.2.1. All things considered, that’s a pretty narrow change set. Everyone here, no doubt, is clueful enough to implement RDNS for their MTAs. We know that there are people in the world that don’t implement it for IPv4. Yet, here we are, using SMTP/IPv4 to discuss this, and I don’t hear anyone saying that IPv4 isn’t ready for prime time as a result of the fact of some operators not implementing RDNS. ... signature.asc Description: Message signed with OpenPGP using GPGMail
Re: IPv6 isn't SMTP
On Mar 26, 2014, at 8:47 PM, Fred Baker (fred) f...@cisco.com wrote: On Mar 25, 2014, at 8:31 PM, Cutler James R james.cut...@consultant.com wrote: 3. Arguing about IPv6 in the context of requirements upon SMTP connections is playing that uncomfortable game with one’s own combat boots. And not particularly productive. That is one of my two big take-aways from this conversation. The other is that operators of SMTP MTAs should implement RDNS for them, which I thought we already knew. To my knowledge, there are three impacts that IPv6 implementation makes on an SMTP implementation. One is that the OS interface to get the address of the next MUA or MTA needs to use getaddrinfo() instead of gethostbyname() (and would do well to observe RFC 6555’s considerations). Another is that, whether on an incoming or an outbound connection, when the application gets its own address from the OS (binary or as a character string), it needs to allocate more storage for the data structure. The third is that it needs to be able to interpret user@2001:db8::1 as well as user@dns-name and user@192.0.2.1. All things considered, that’s a pretty narrow change set. Everyone here, no doubt, is clueful enough to implement RDNS for their MTAs. We know that there are people in the world that don’t implement it for IPv4. Yet, here we are, using SMTP/IPv4 to discuss this, and I don’t hear anyone saying that IPv4 isn’t ready for prime time as a result of the fact of some operators not implementing RDNS. ... Fred Baker describes the requirements in a most satisfactory manner. Thank you, Fred. James R. Cutler james.cut...@consultant.com PGP keys at http://pgp.mit.edu signature.asc Description: Message signed with OpenPGP using GPGMail
Re: IPv6 isn't SMTP
To my knowledge, there are three impacts that IPv6 implementation makes on an SMTP implementation. One is that the OS interface to get the address of the next MUA or MTA needs to use getaddrinfo() instead of gethostbyname() (and would do well to observe RFC 6555�s considerations). In practice it's considerably more complex than that due to MX handling. If you have multihomed hosts, or multiple MXes at the same priority, you need to decide in what order to try them, and what to do next if a connection attempt fails. If one MX has an A record and another has , do you always prefer the one with the ? If a host has both an A and an , you probably try the first, so if the connection fails, do you try the A or do you skip to the next host? The RFC is deliberately unhelpful here, and a fair amount of fiddling is required to come up with heuristics that work well. There are also some odd things in the spec. For example, according to RFC 5321 this is not a syntactically valid e-mail address: mailbox@[IPv6:2001:12:34:56::78:ab:cd] R's, John
Re: IPv6 isn't SMTP
On Mar 26, 2014, at 5:47 PM, Fred Baker (fred) f...@cisco.com wrote: On Mar 25, 2014, at 8:31 PM, Cutler James R james.cut...@consultant.com wrote: 3. Arguing about IPv6 in the context of requirements upon SMTP connections is playing that uncomfortable game with one’s own combat boots. And not particularly productive. That is one of my two big take-aways from this conversation. The other is that operators of SMTP MTAs should implement RDNS for them, which I thought we already knew. It is in several industry recommendations cf for instance BCP at www.m3aawg.org To my knowledge, there are three impacts that IPv6 implementation makes on an SMTP implementation. One is that the OS interface to get the address of the next MUA or MTA needs to use getaddrinfo() instead of gethostbyname() (and would do well to observe RFC 6555’s considerations). Another is that, whether on an incoming or an outbound connection, when the application gets its own address from the OS (binary or as a character string), it needs to allocate more storage for the data structure. The third is that it needs to be able to interpret user@2001:db8::1 as well as user@dns-name and user@192.0.2.1. and user@2001:db8::1.25 with user@192.0.2.1:25. Who had the good idea to use : for IPv6 addresses while this is the separator for the port in IPv4? A few MTA are confused by it. All things considered, that’s a pretty narrow change set. Everyone here, no doubt, is clueful enough to implement RDNS for their MTAs. We know that there are people in the world that don’t implement it for IPv4. Yet, here we are, using SMTP/IPv4 to discuss this, and I don’t hear anyone saying that IPv4 isn’t ready for prime time as a result of the fact of some operators not implementing RDNS. There is some confusion between MX selection and address selection, I tried to document it, and resolve the ambiguities in http://datatracker.ietf.org/doc/draft-martin-smtp-target-host-selection-ipv4-IPv6/ (comments at apps-disc...@ietf.org) Remember 70 to 90% of email is spam, blacklists can drop as much as 75% of spam at connection time (an IPv6 blacklist has problems due to size and impact on DNS). If we mess up the transition of SMTP to IPv6, less than 1 out of 10 emails in your mailbox will be remotely interesting…. signature.asc Description: Message signed with OpenPGP using GPGMail
Re: IPv6 isn't SMTP
On 3/26/2014 10:16 PM, Franck Martin wrote: and user@2001:db8::1.25 with user@192.0.2.1:25. Who had the good idea to use : for IPv6 addresses while this is the separator for the port in IPv4? A few MTA are confused by it. At the network level the IPv6 address is just a big number. No confusion there. At the plaintext level the naked IPv6 address should be wrapped in square brackets. From: http://tools.ietf.org/html/rfc3986#section-3.2.2
Re: IPv6 isn't SMTP
On 3/26/2014 11:22 AM, Barry Shein wrote: What makes IP address mobility possible is mass, unauthorized if not simply illegal use of others' resources, such as with botnets or massive exploiting of holes in web hosting sites' software. Except that compromised personal computers are 'valid' by all normal metrics. An army of such machines provides a kind of address mobility that is not detected by any normal means. Fundamentally spam is a security isse. In the same way as burglary is a security issue, yeah. Which is to say that fundamentally, spam is a social issue, like any other crime. d/ -- Dave Crocker Brandenburg InternetWorking bbiw.net
Re: IPv6 isn't SMTP
On March 26, 2014 at 20:21 d...@dcrocker.net (Dave Crocker) wrote: On 3/26/2014 11:22 AM, Barry Shein wrote: What makes IP address mobility possible is mass, unauthorized if not simply illegal use of others' resources, such as with botnets or massive exploiting of holes in web hosting sites' software. Except that compromised personal computers are 'valid' by all normal metrics. From the receiving or intermediary point of view, sure. One would like to think that the owner of the transmitting host knows s/he didn't intend to send 15,000 herbal hair regrowth ads this morning if somehow it was pointed out to them and would probably be unhappy over it. So, illegal or at best unauthorized from the POV of the transmitter, owner or manager etc of the PC. I'm simply saying that spam would barely exist without these illegal (oh let's not split that hair) resources. An army of such machines provides a kind of address mobility that is not detected by any normal means. I agree. Perhaps a more global view might work but we don't have a way to implement that, or perhaps put better, the will to implement that. For example 1,000,000 systems sending out basically the same message (BUY HERBAL HAIR RE-GROWER!) would be suspicious particularly if the sending systems were scattered hither and yon. And we do try to do this via blacklists but it's not quite enough mostly because it's after the fact, much of the damage has been done, the 1M msgs were sent and put into peoples' mailboxes already. And then the spammers change their footprint. Really not a very good method but we do what we can. Fundamentally spam is a security isse. In the same way as burglary is a security issue, yeah. Which is to say that fundamentally, spam is a social issue, like any other crime. No, I really mean that without the illegal (let's not regrow that hair) resources the spammers are sunk, kaput, out of business. It's the only way they can operate in any effective manner. The only way. There's more to this but foiling whatever it is that spammers use to build botnets and massively exploit for example web hosting software will tend to work. The list is pretty short as far as I can tell. Everything else, such as content analysis and blacklisting will tend to not work, or only so much, a never-ending battle. Some will blanche at this but the entire spam problem basically arose from the crap security in Windows systems, particularly prior to maybe XP/SP2. Not sure where all that leads us, however. Better security at those major exploitation points, in a nutshell. And if someone disagrees then please tell me how spammers as we know them (and related miscreants) can operate without these few sources of purloined resources. Preferably without a big hand-wave like oh they'll just find something else! Maybe not! -- -Barry Shein The World | b...@theworld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: 800-THE-WRLD| Dial-Up: US, PR, Canada Software Tool Die| Public Access Internet | SINCE 1989 *oo*
IPv6 isn't SMTP
Wow, what a lot of NANOG traffic about IPv6 readiness for SMTP! Please explain my misunderstanding on the following: 1. IPv6 is a Routing Layer Protocol (with some associated helpers, like RA, ND, DHCP-PD, and the like). 2. SMTP is an Application Layer Protocol, supposedly independent of Routing and lower layers of the protocol stack. Various communities have added connection initiation requirements that sometimes impinge upon layer 3 by requiring name/address correlations in DNS and none of which depend directly on technical aspects of layer 3 addressing. [ignoring obsolescent MTA implementations] 3. Arguing about IPv6 in the context of requirements upon SMTP connections is playing that uncomfortable game with one’s own combat boots. And not particularly productive. I look forward to furthering my education. James R. Cutler james.cut...@consultant.com PGP keys at http://pgp.mit.edu signature.asc Description: Message signed with OpenPGP using GPGMail
Re: IPv6 isn't SMTP
On 3/26/2014 午後 12:31, Cutler James R wrote: Wow, what a lot of NANOG traffic about IPv6 readiness for SMTP! Please explain my misunderstanding on the following: 1. IPv6 is a Routing Layer Protocol (with some associated helpers, like RA, ND, DHCP-PD, and the like). 2. SMTP is an Application Layer Protocol, supposedly independent of Routing and lower layers of the protocol stack. Various communities have added connection initiation requirements that sometimes impinge upon layer 3 by requiring name/address correlations in DNS and none of which depend directly on technical aspects of layer 3 addressing. [ignoring obsolescent MTA implementations] 3. Arguing about IPv6 in the context of requirements upon SMTP connections is playing that uncomfortable game with one’s own combat boots. And not particularly productive. I look forward to furthering my education. James R. Cutler james.cut...@consultant.com PGP keys at http://pgp.mit.edu +1, very well put.
Re: IPv6 isn't SMTP
On 3/25/2014 10:31 PM, Cutler James R wrote: Wow, what a lot of NANOG traffic about IPv6 readiness for SMTP! Please explain my misunderstanding on the following: 1. IPv6 is a Routing Layer Protocol (with some associated helpers, like RA, ND, DHCP-PD, and the like). 2. SMTP is an Application Layer Protocol, supposedly independent of Routing and lower layers of the protocol stack. Various communities have added connection initiation requirements that sometimes impinge upon layer 3 by requiring name/address correlations in DNS and none of which depend directly on technical aspects of layer 3 addressing. [ignoring obsolescent MTA implementations] 3. Arguing about IPv6 in the context of requirements upon SMTP connections is playing that uncomfortable game with one’s own combat boots. And not particularly productive. I look forward to furthering my education. [applause] -- Requiescas in pace o email Two identifying characteristics of System Administrators: Ex turpi causa non oritur actio Infallibility, and the ability to learn from their mistakes. (Adapted from Stephen Pinker)
Re: IPv6 isn't SMTP
3. Arguing about IPv6 in the context of requirements upon SMTP connections is playing that uncomfortable game with one�s own combat boots. And not particularly productive. If you can figure out how to do effective spam filtering without looking at the IP addresses from which mail arrives, you will be in a position to make a whole lot of money. But, as always, I'm not holding my breath. R's, John PS: Note the word effective.
Re: IPv6 isn't SMTP
On 3/25/2014 11:18 PM, John Levine wrote: 3. Arguing about IPv6 in the context of requirements upon SMTP connections is playing that uncomfortable game with ones own combat boots. And not particularly productive. If you can figure out how to do effective spam filtering without looking at the IP addresses from which mail arrives, you will be in a position to make a whole lot of money. But, as always, I'm not holding my breath. Is spam fighting really about SMTP? Or is it about abuse of the transport layer by (among other things) the SMTP? -- Requiescas in pace o email Two identifying characteristics of System Administrators: Ex turpi causa non oritur actio Infallibility, and the ability to learn from their mistakes. (Adapted from Stephen Pinker)
Re: IPv6 isn't SMTP
On 3/26/2014 12:33 AM, Larry Sheldon wrote: On 3/25/2014 11:18 PM, John Levine wrote: 3. Arguing about IPv6 in the context of requirements upon SMTP connections is playing that uncomfortable game with ones own combat boots. And not particularly productive. If you can figure out how to do effective spam filtering without looking at the IP addresses from which mail arrives, you will be in a position to make a whole lot of money. Is spam fighting really about SMTP? Or is it about abuse of the transport layer by (among other things) the SMTP? Well, with current spam, the transport layer is irrelevant, given the proper phished credentials :( Jeff
Re: IPv6 isn't SMTP
But, as always, I'm not holding my breath. Is spam fighting really about SMTP? Or is it about abuse of the transport layer by (among other things) the SMTP? I don't think that your typical spam recipient cares how the spam got into her inbox. Anyone who has any familiarity with large scale mail systems knows that the only way to have any hope of effective spam filtering, in any medium, is to combine all the clues you can get. For mail, the source of the message is a highly useful clue. R's, John
Re: IPv6 isn't SMTP
On Tue, Mar 25, 2014 at 11:07 PM, Larry Sheldon larryshel...@cox.netwrote: On 3/25/2014 10:31 PM, Cutler James R wrote: 2. SMTP is an Application Layer Protocol, supposedly independent of Routing and lower layers of the protocol stack. Various communities have added connection initiation requirements that sometimes impinge [snip] (1) Architectural layers are a protocol design construction, only, which assist with standardization. They are not a separation of responsibilities. A SMTP server has to take care to have an implementation of all layers. Further: A well designed SMTP server has to be built with some understanding of all layers involved, they are used to construct Received: headers, and a SMTP server cannot treat the application layer in isolation. It is a complete and utter fiction, to suppose that the layers can be treated in total isolation. (2) The IP protocol layer is not actually independent. Moving to IPv6 does in fact have effective new requirements for SMTP servers. (3) Abuse transcends all layers, and so does the administration of any service provided by an application. (4) When a major change will [by necessity] be made to any layer underlying the MTA application on the protocol stack, now is also a good time to look at the overall service as a whole. -- -JH
