Question... Can this be addressed in some way during or before upgrade that allows it to only affect new installs? Even a config based workaround prior to upgrade might make this a change less disruptive.
If part of the upgrade process includes a step (maybe even a script) to set the NN RPC port explicitly beforehand then it would allow existing deployments and related clients to remain whole - otherwise it will uptake the new default port. Meta note: we shouldn't be so pedantic about policy that we can't back out something that is considered a bug or even mistake. On Mon, Jan 8, 2018 at 9:17 PM, Aaron T. Myers <a...@apache.org> wrote: > Hello all, > > Over in HDFS-12990 <https://issues.apache.org/jira/browse/HDFS-12990>[1], > we're having some discussion about whether or not it's ever acceptable to > make an incompatible change in a minor or dot release. In general this is > of course undesirable and should be avoided in almost all cases. However, I > believe that each instance of someone desiring to make an incompatible > change should be evaluated on a case-by-case basis to consider the costs > and benefits of making that change. For example, I believe that we've > historically made incompatible changes in minor or dot releases which would > break older clients for security reasons. > > In this particular case linked above, I believe that given that Hadoop > 3.0.0 was just released, and thus very few folks are likely to have > deployed it, it would benefit a large number of existing deployments and > downstream applications to change the default NN RPC port number back to > what it was in all previously-released versions of Apache Hadoop. I'd like > to make this change in 3.0.1, and there is no question that doing so would > should be considered an incompatible change between 3.0.0 and 3.0.1. > However, I believe this incompatible change is warranted given the > circumstances. > > Would like to hear others' thoughts on this. > > Thanks, > Aaron > > [1] For some background, it used to be the case that many of Hadoop's > default service ports were in the ephemeral range. This could potentially > cause a service to fail to start up on a given host if some other process > had happened to have already bound to said port. As part of that effort, we > also changed the default NN RPC port from 8020 to 9820. Even though 8020 > wasn't in the ephemeral range, we moved it to 9820 to be close to the new > range of the rest of the ports. At the time this change was made, though, I > and others didn't realize the substantial downsides that doing so would > introduce, for example the Hive metastore will put full HDFS paths > including the port into its database, which can be a substantial upgrade > headache. >
