In message <[email protected]>, Jo Rhett <[email protected]> wrote:
>Robert, It's Ron, actually. But you can call me rfg. Everyone else does. >I'd first like to ask that you back off on your tone. I >have treated you with respect... As I have done for you. Disagreement, even vigorous disagreement, on purely technical grounds is not equal to disrespect. I have not questioned either your motives or your ancestry, nor will I. I am completely sure that both are entirely beyond reproach. And I have not called you "ignorant" as you have done to me. If anyone has a basis for indignation, that would be me. But I prefer not to dwell on such small slights, to turn the other cheek, and to continue to engage with you on the technical points of your arguments and assertions, seeing where we may have points of common agreement. There appear to be many. >> I _am_ familiar with the IPv4 16-bit limit on port numbers, but I fail >> to see why that should affect the allocation size choice. How many >> IPv4 port numbers does it take to support 10,000 unique and distinct >> web sites? > >I have production services implemented by a small cluster of nodes which >burn through more outbound TCP ports than can be accommodated for by >less than a /26. I'll take your word for it. But I'm not sure how what you just said in any way negates the part of what I said that you quoted. You pretty obviously have some rather unique and distinctive stuff going on if your web *servers* need to make so many *outbound* connections that you need a whole /26 just to accomodate them all. Or were you seriously intending to assert that having web *servers* that make lots of lots of outbound (client) TCP connections is in some sense "typical" for the average Joe on the Internet? I'm in complete agreement if you say that one may need lots and lots and LOTS of IPv4 ports, possibly even distributed across lots and lots and LOTs of individual IPv4 addresses. Heck! Just ask Comcast! But the assertion of mine that you quoted, and that you appear to want to take issue with, intended to refer only to *typical* server-side stuff. Thus, I am inclined to wonder if you and I are even really disagreeing at all. (It would appear not, but I'm perfecvtly happty to let you be the judge of that.) >> How many IPv4 port numbers does it take to support mail service for >10,000 dmains? Answer: One. > >Only if all 10,00 domains only send mail to 6 different locations within >the TIME_WAIT period. That's a *very* interesting point, and I am very glad that you raised it. I infer from what you just said that you are assuming... quite correctly, I think... that each -outbound- (client-side) connection, e.g. from my mail server to your mail server, typically makes use of its own unique and (at least during the transaction) dedicated outbound TCP port number. Thus, if I am supporting 10,000 domains for email on my single little IPv4 address, and if -all- of them were to try, simultaneously, to send mail to 6 or more different other/remote places, then yes, I can see that this would promptly exhaust all 65536 potential outbound TCP port numbers and that thus, email havoc might then possibly ensue. (Or maybe the mail server would just wait until some of the outbound ports got freed up.) I think that you are correct that this is indeed the way that most or all existing mail servers handle their outbound flow. I mean it's just a lot simpler, programatically, to do things this way, i.e. using each local port number in a "dedicated" fashion for each outbound (client) SMTP connection/ transaction. But just because this is, or may be, both traditional and programatically much easier than the alternative, I'm not sure that it *must* be that way. I may not be the world's biggest or best hotshot sockets programmer, but I've done a bit of it in my time, and my recollection is that each TCP connection is denoted, on each end, by a unique tuple consisting of (at least) the source IP and source port number, *and* also the destination IP and destination port number, and that the only requirement is that the entire tuple, when considered as a whole, be sufficiently unique so as to allow the OS to reliably sort out which packet should go to which process on each end on the wire. If so, then it would seem to me that, in theory at least, a mail server, when acting in its role as an SMTP -client-, could carry on a mutitude of simultaneous connections / SMTP sessions, all using only a single TCP port on the local machine, provided that each of these outbound / client connections were to a unique and different remote IP address. In short, even when a given mail server hosted on a single IPv4 address was asked to make, say, a million simultaneous outbound TCP/SMTP connections to a million remote mail servers elsewhere, it could most certainly do so, I believe, by simply using each of the outbound TCP port numbers available to it for multiple simultaneous connections to mutiple other mail servers elsewhere. If someone were to code up (or if someone already _has_ coded up) a mail server which supported this rather trivial port reuse trick, then that might be reasonably expected to aleviate the kind of port number exhaustion (specifically for mail servers acting in heir capacity as mail -clients-) that you have quite reasonably noted. I'll have to ask around and find out if any mail servers/clients are already doing this, and if not why not. It may perhaps just simply be the case that nobody saw it as being worthwhile to implement, but certainly, if one were in fact trying to support, say, mail for a VERY large number of domains, using just, say, a 32-core Ryzen with a big pipe on a single IPv4 address, then it's possible that a failure to make fully efficient use of the available IPv4 outbound port numbers could create unfortunately low hard limits where using the port numbers efficiently might allow millions of domains to be supported, for both inbound and outbound email, on a single IPv4 address. As IPv4 addresses become more scarce and more expensive, it may become worthwhile (for mail server authors) to make better use of the available IPv4 port numbers on the outbound (client) side of things. In any case it is definitely worth me asking about whether or not mail servers currently can make multiple simultaneous uses of a single otbound port number or not, and I intend to do just that (and then report back). >And before you try and do the math, recall that needs to include reverse >auth, DKIM auth, SPF auth, domain callbacks, greylisting/mandatory >retrys a single message delivery can utilize as many as 24 >tcp/udp ports these days. It's an intersting assertion, although I'm not sure that I see how either DKIM or SPF enter into the equation at all. As I understand it, those are verified via DNS (udp/53) and thus should not have an effect on port usage. Callbacks mean your mail server needs to make an outbound SMTP connection for each inbound one, and thus, I must conceed that you have a point that this might impact total outbound TCP port usage. But as noted above, this likely only becomes an issue when you are already near saturation of the available 64k - 2K outbound port numbers, and even in such cases, it seems like it should be possible to use single outbound TCP ports for multiple outbound SMTP connections, which, if it's actually possible, would pretty much solve even that extreme port exhaustion problem. But I need to make some inquiries about that. Greylisting just means that your mail server, whan acting in its capacity as an SMTP client, may sometimes needs to drop the connection (thus giving back the outbound port number it was using) and then try again later, so I'm not sure that I see how that affects the calculation either, other than that fact that, at scale, it mans your mail server is likely to spend a bit more time being a client than it outwise would, i.e. in the absence of greylisting. >> How many IPv4 port numbers does it take to support DNS service for >10,000 domains? > >Only if those domains only request or forward queries for 6 outside >references a second. Here again, you are starting from the assumption that each -outbound- (client side) DNS query needs to have its own unique port number. I could be wrong about this, but I don't believe that that is actually the case, even for current gen (or even prior gen) name servers that are already widely deployed, and that have been, for many years already. In fact I seem to vaguely recall there being some setting in BIND 9 where the operator could instruct the thing to originate -all- outbound queries from a single local port number, e.g. UDP port 53. Or maybe I'm mis-remembering. On the other hand, if I am actually remembering that correctly, then I think that it is possible to operate a name server on a single IPv4 address which not only provides authoritative service (on a single UDP port, 53) for essentially unlimited numbers of local zones, but one which can also (and simultaneously) act as a general resolver, providing answers pertaining to non-local zones, and that it can serve in this (resolver) role as well while still only using only a single IPv4 address -and- only a single local port number (e.g. udp/53). But I'm sure that someone will correct me if I'm wrong about that. >And that's only on >the server side: modern clients can burn 100 query ports loading a >single web page. I have already conceeded the point that _clients_, generally speaking, and with the exceptions noted above, may need a LOT of both IP ports and a lot of IP addresses. And I cited Comcast as the quintessential example. But it is still my contention that most or all support for end-luser client stuff should move to IPv6, leaving IPv4 for use as servers. It is also my contention that when and if IPv4 addresses begin to be used only and exclusively for servers, then even a lowly /24 block will be vast overkill for the vast majority of organizations. If the addresses and ports are used with proper efficiency, then giving someone even just a whole /24, even at the present moment, is arguably analogous to giving them the entire contents of Lake Superior when really, all they needed was to fill their bathtub. Such uncareful and profligate use of IPv4 addresses, although common and routine since the beginning of the Internet, is provably unsustainable, going forward. >> Given these facts, I'm not persuaded that the portion of your comment >> relating to IPv4 port numbers either does have or should have any >> relevance to the selection of an appropriate initial IPv4 allocation size. > >You aren't making a valid argument, simply demonstrating your >own lack of knowledge of how IP works in today's services. I can only encourage you to try, as best as you are able, to stick to addressing the technical points, and leave the ad hominems for some other day and for some other and more appropriate forum. I believe that I have addressed above, point by point, all of the various (mostly obvious) ways in which organizations of virtually all stripes could make vastly more efficient use of each and every one of their IPv4 addresses. (Your organization, Mr. Rhett, may perhaps be the exception That proves the general rule.) If there is a clear and compelling case to be made that even very large (and *typical*) hosting organizations could not support even up to, say, a million customer domains, ON THE SERVER SIDE, using a single IPv4 address, then I, for one, don't believe that you have yet persuasively made that case, arm waving and personal attacks aside. >You appear to be thinking of the 1980s when a single TCP connect was the >session ;-) I would turn this around and say that you appear to be thinking in 1990s terms, where each web site needed its own dedicated IP address, where each mail server only supported a single domain, an where each name server did likewise. But we have come a long way from that, and nowadays we have better tools, better software, and much better ideas of how to support lots and lots of server-side stuff on each single IP address. Which set of notions about the current best way to maximize the efficient use of each individual IPv4 address, your's or mine, are more apropos to the current era and the current situation will, of course, be judged by the rest of the folks here. I can only restat what I believe to be my basic claim, which is that profligate and INefficent use of IPv4 addresses... by snowshoe spammers, by horders, by speculators, and even by a majority of run-of-the mill organizations... is at present far more the rule than is is th execption, and that it is appropriate that policy should be guided by that undeniable reality. (If fact, we are only even discussing these issues because, as reported by ARIN itself, the recent activities of the speculators has now, finally, gotten rather entirely out of hand.) >Further, very few services >return the entire content themselves, but instead build it from hundreds >of other sources. Now THAT is an extraordinaily sweeping assertion! Can you support it with actual facts, as opposed to just the assertion, standing by itself? I certainly know that *my* little personal web site(s), when they were up, quite certainly did not need to "build content from hundreds of other sources". I understand that you may resonably assert that me and my little web sites don't count as being "typical" in the modern era, but I would simply turn that around and ask you what makes you so sure that these complex things... whatever they are... that you allege are "built from hundreds of other sources" might not also be rather entirely atypical and not at all representative of the norm on the modern Internet, and that thus, it is your experinnce, perhaps in addition to mine, that is far out on one extreme of the bell curve r the other, and that thus, neither should be used as the last word or deciding factor in any decision intended to address the most common usage scenarios. >> Your concern about the inherent port number limitations of >> IPv4 seems to arise exclusively with respect to the use of IPv4 >> addresses as _clients_ > >I think I've already addressed this. I haven't worked >anywhere that a single connection to the public server didn't >require hundreds of related connections to other public services to >deliver the page you see. Nor have I seen many web pages which don't >contain content loaded from less than 50 places. The world is more >complex these days. Again, we seem to be talking past one another. I assert yet again that I've already conceeded that ON THE CLIENT SIDE lots and lots of IPs and also, perhaps, lots and lots of port numbers may be required, and I believe that I can count on Comacast and other such "eyeball providers" to support assertion/concession. But I also reiterate that -clients- and -eyeballs- should be migrated to IPv6, immediately if not sooner, leaving IPv4 to be used for the -efficient- (and highly shared) hosting of servers. >and ARIN issues IPs to clients, so their needs must be >considered in IP policy. I agree. But this is 2019, not 2003. If entities need to support gobs and gobs of eyeballs/clients, then ARIN should be giving them nice juicy blobs of of IPv6 that they can swim around in, all the live long day. Clients/eyeballs can live, breath, and be happy on IPv6. Unfortunately however, -servers- still need to be on IPv4 in order to be accessible to everyone. But as I have elaborated above, these days you can pack server-side support for one hell of a lot of domains... perhaps several million... onto even just a -single- IPv4 address, *if* you do it properly. So giving any new applicant even a whole IPv4 /24 is arguably ridiculous overkill (by a factor or 256) which is only even made necessary by the fact that it is hard or impossible to get anything smaller than that routed. >Yes, yes >we should be embracing IPv6, and I personally am doing everything I can >to drag organizations to that conclusion. But the overwhelming evidence >is that enterprises and many regional networks are not playing ball. Fortunately, the solution to this longstanding problem is finally at hand! We now have ARIN itself telling us all, point blank, that they have caught some folks red handed, gaming the curent IPv4 allocation system for private profit. This is a rather radical new development! To quote again from one of my favorite authors "As our situation is new, so me must think anew and act anew." The present circumstance gives the entire ARIN community exactly the kind of golden opportunity that it has never yet had, and that it may never have again. It may only be at this specific moment in history when ARIN might have, in hand, a compellling justification to begin insisting that -new- "eyeballs" (i.e. clients) be pushed onto IPv6 whether they like it or not. Everyone who has been advocating and evanglizing for IPv6 adoption since its arrival on the scene should be relishing and reveling in this moment, and should be supporting the limitation of IPv4 allocations, going forward to no more than a /24, *except* in cases where a compelling case can be be made that EVEN WITH the most modern address and port reuse techniques presently available, there is a demonstratable need for something in excess of 256 unique and independent *servers* by the requesting party. >Ronald, in summary: I get that you have only faced certain of the >limitations of IP. And I get that you want a simple world. Likewise and conversely, I get that your unique and atypical usage may compel yu to create "servers" which themselves also must act as clients to a highly unusual level, and that you thus have reasonable concerns about being able to support your unique usage patterns, going forward. I get that. I just don't think that your unique requirements are or should properly be the one and only basis for a generalized allocation policy, intended for all, going forward, >Understand >that (A) it's not as simple as we might all want it and (B) >being rude and disrespectful to people while discarding the possibility >of usage or needs outside your own experience does not help us create >good policy. I agree with both assertions and merely hope that just because you've never before been asked to consider how you migh further minimize your usage of IPv4 addresses ON THE SERVER SIDE, you will not allow that lack of experience on your part to cause you to carelessly or frivolously discard the possibility that what I have suggested in the way of optimizing usage of IPv4 addresss may in fact be do-able, even if it may also be outside of your personal past experience and/or outside of your organization's current capacities, based on the unique complexities of your operations. In short, I don't believe that it is either radical or disrespectful to state the obvious, which is that IPv4 addresses are not, by and large, used efficiently at the present time. Your organization may be, and is, for all I know, a shining exception to this general rule. But that doesn't mean that the general observation is not valid. >I spent an hour writing this message just to get you to >calm down a bit, and stop writing to us like we're all idiots. I am and have been perfectly calm. I have no idea why you would suggest otherwise. I spent more that an hour writing this reply, which attempts to address your technical points, one by one, and I don't think I've treated anyone like an idiot. But if there is consensus on that point, I will concede it. I ask for other opinions on the following simple proposition: Has rfg treated or addressed anyone as an idiot? If there is unanimity here that I have done, then I will apologize forthwith a review my manner of communication with an eye towward improvement. I believe however that I've stuck to the technical issues, and that the points I've made have technical merit. You obviously believe otherwise, and consider *me* an idiot, apparently based primarily or exclusively on the fact that I simply disagree with your view, technically. So be it. I can only encourage you to try to stick to discussion of the technical points, and leave the personally directed critiques for another time and and for some other more appropriate forum. Regards, rfg _______________________________________________ ARIN-PPML You are receiving this message because you are subscribed to the ARIN Public Policy Mailing List ([email protected]). Unsubscribe or manage your mailing list subscription at: https://lists.arin.net/mailman/listinfo/arin-ppml Please contact [email protected] if you experience any issues.
