It looks like QBT tests are still being run on branch-2 ( https://builds.apache.org/view/H-L/view/Hadoop/job/hadoop-qbt-branch2-java7-linux-x86/), and they are not very helpful at this point. Can we change the QBT tests to run against branch-2.10 instead?
Jim On Mon, Dec 23, 2019 at 7:44 PM Akira Ajisaka <aajis...@apache.org> wrote: > Thank you, Ayush. > > I understand we should keep branch-2 as is, as well as master. > > -Akira > > > On Mon, Dec 23, 2019 at 9:14 PM Ayush Saxena <ayush...@gmail.com> wrote: > > > Hi Akira > > Seems there was an INFRA ticket for that. INFRA-19581, > > But the INFRA people closed as wont do and yes, the branch is protected, > > we can’t delete it directly. > > > > Ref: https://issues.apache.org/jira/browse/INFRA-19581 > > > > -Ayush > > > > On 23-Dec-2019, at 5:03 PM, Akira Ajisaka <aajis...@apache.org> wrote: > > > > Thank you for your work, Jonathan. > > > > I found branch-2 has been unintentionally pushed again. Would you remove > > it? > > I think the branch should be protected if possible. > > > > -Akira > > > > On Tue, Dec 10, 2019 at 5:17 AM Jonathan Hung <jyhung2...@gmail.com> > > wrote: > > > > It's done. The new commit chain is: trunk -> branch-3.2 -> branch-3.1 -> > > > > branch-2.10 -> branch-2.9 -> branch-2.8 (branch-2 no longer exists, > please > > > > don't try to commit to it) > > > > > > Completed procedure: > > > > > > - Verified everything in old branch-2.10 was in old branch-2 > > > > - Delete old branch-2.10 > > > > - Rename branch-2 to (new) branch-2.10 > > > > - Set version in new branch-2.10 to 2.10.1-SNAPSHOT > > > > - Renamed fix versions from 2.11.0 to 2.10.1 > > > > - Removed 2.11.0 as a version in HADOOP/YARN/HDFS/MAPREDUCE > > > > > > > > Jonathan Hung > > > > > > > > On Wed, Dec 4, 2019 at 10:55 AM Jonathan Hung <jyhung2...@gmail.com> > > > > wrote: > > > > > > FYI, starting the rename process, beginning with INFRA-19521. > > > > > > Jonathan Hung > > > > > > > > On Wed, Nov 27, 2019 at 12:15 PM Konstantin Shvachko < > > > > shv.had...@gmail.com> > > > > wrote: > > > > > > Hey guys, > > > > > > I think we diverged a bit from the initial topic of this discussion, > > > > which is removing branch-2.10, and changing the version of branch-2 from > > > > 2.11.0-SNAPSHOT to 2.10.1-SNAPSHOT. > > > > Sounds like the subject line for this thread "Making 2.10 the last minor > > > > 2.x release" confused people. > > > > It is in fact a wider matter that can be discussed when somebody > > > > actually > > > > proposes to release 2.11, which I understand nobody does at the moment. > > > > > > So if anybody objects removing branch-2.10 please make an argument. > > > > Otherwise we should go ahead and just do it next week. > > > > I see people still struggling to keep branch-2 and branch-2.10 in sync. > > > > > > Thanks, > > > > --Konstantin > > > > > > On Thu, Nov 21, 2019 at 3:49 PM Jonathan Hung <jyhung2...@gmail.com> > > > > wrote: > > > > > > Thanks for the detailed thoughts, everyone. > > > > > > Eric (Badger), my understanding is the same as yours re. minor vs patch > > > > releases. As for putting features into minor/patch releases, if we > > > > keep the > > > > convention of putting new features only into minor releases, my > > > > assumption > > > > is still that it's unlikely people will want to get them into branch-2 > > > > (based on the 2.10.0 release process). For the java 11 issue, we > > > > haven't > > > > even really removed support for java 7 in branch-2 (much less java 8), > > > > so I > > > > feel moving to java 11 would go along with a move to branch 3. And as > > > > you > > > > mentioned, if people really want to use java 11 on branch-2, we can > > > > always > > > > revive branch-2. But for now I think the convenience of not needing to > > > > port > > > > to both branch-2 and branch-2.10 (and below) outweighs the cost of > > > > potentially needing to revive branch-2. > > > > > > Jonathan Hung > > > > > > > > On Wed, Nov 20, 2019 at 10:50 AM Eric Yang <ey...@cloudera.com> wrote: > > > > > > +1 for 2.10.x as last release for 2.x version. > > > > > > Software would become more compatible when more companies stress test > > > > the same software and making improvements in trunk. Some may be extra > > > > caution on moving up the version because obligation internally to keep > > > > things running. Company obligation should not be the driving force to > > > > maintain Hadoop branches. There is no proper collaboration in the > > > > community when every name brand company maintains its own Hadoop 2.x > > > > version. I think it would be more healthy for the community to > > > > reduce the > > > > branch forking and spend energy on trunk to harden the software. > > > > This will > > > > give more confidence to move up the version than trying to fix n > > > > permutations breakage like Flash fixing the timeline. > > > > > > Apache license stated, there is no warranty of any kind for code > > > > contributions. Fewer community release process should improve > > > > software > > > > quality when eyes are on trunk, and help steering toward the same end > > > > goals. > > > > > > regards, > > > > Eric > > > > > > > > > > On Tue, Nov 19, 2019 at 3:03 PM Eric Badger > > > > <ebad...@verizonmedia.com.invalid> wrote: > > > > > > Hello all, > > > > > > Is it written anywhere what the difference is between a minor release > > > > and a > > > > point/dot/maintenance (I'll use "point" from here on out) release? I > > > > have > > > > looked around and I can't find anything other than some compatibility > > > > documentation in 2.x that has since been removed in 3.x [1] [2]. I > > > > think > > > > this would help shape my opinion on whether or not to keep branch-2 > > > > alive. > > > > My current understanding is that we can't really break compatibility > > > > in > > > > either a minor or point release. But the only mention of the > > > > difference > > > > between minor and point releases is how to deal with Stable, > > > > Evolving, > > > > and > > > > Unstable tags, and how to deal with changing default configuration > > > > values. > > > > So it seems like there really isn't a big official difference between > > > > the > > > > two. In my mind, the functional difference between the two is that > > > > the > > > > minor releases may have added features and rewrites, while the point > > > > releases only have bug fixes. This might be an incorrect > > > > understanding, but > > > > that's what I have gathered from watching the releases over the last > > > > few > > > > years. Whether or not this is a correct understanding, I think that > > > > this > > > > needs to be documented somewhere, even if it is just a convention. > > > > > > Given my assumed understanding of minor vs point releases, here are > > > > the > > > > pros/cons that I can think of for having a branch-2. Please add on or > > > > correct me for anything you feel is missing or inadequate. > > > > Pros: > > > > - Features/rewrites/higher-risk patches are less likely to be put > > > > into > > > > 2.10.x > > > > - It is less necessary to move to 3.x > > > > > > Cons: > > > > - Bug fixes are less likely to be put into 2.10.x > > > > - An extra branch to maintain > > > > - Committers have an extra branch (5 vs 4 total branches) to commit > > > > patches to if they should go all the way back to 2.10.x > > > > - It is less necessary to move to 3.x > > > > > > So on the one hand you get added stability in fewer features being > > > > committed to 2.10.x, but then on the other you get fewer bug fixes > > > > being > > > > committed. In a perfect world, we wouldn't have to make this > > > > tradeoff. > > > > But > > > > we don't live in a perfect world and committers will make mistakes > > > > either > > > > because of lack of knowledge or simply because they made a mistake. > > > > If > > > > we > > > > have a branch-2, committers will forget, not know to, or choose not > > > > to > > > > (for > > > > whatever reason) commit valid bug fixes back all the way to > > > > branch-2.10. If > > > > we don't have a branch-2, committers who want their borderline risky > > > > feature in the 2.x line will err on the side of putting it into > > > > branch-2.10 > > > > instead of proposing the creation of a branch-2. Clearly I have made > > > > quite > > > > a few assumptions here based on my own experiences, so I would like > > > > to > > > > hear > > > > if others have similar or opposing views. > > > > > > As far as 3.x goes, to me it seems like some of the reasoning for > > > > killing > > > > branch-2 is due to an effort to push the community towards 3.x. This > > > > is why > > > > I have added movement to 3.x as both a pro and a con. As a community > > > > trying > > > > to move forward, keeping as many companies on similar branches as > > > > possible > > > > is a good way to make sure the code is well-tested. However, from a > > > > stability point of view, moving to 3.x is still scary and being able > > > > to > > > > stay on 2.x until you are comfortable to move is very nice. The > > > > 2.10.0 > > > > bridge release effort has been very good at making it possible for > > > > people > > > > to move from 2.x in 3.x, but the diff between 2.x and 3.x is so large > > > > that > > > > it is reasonable for companies to want to be extra cautious with 3.x > > > > due to > > > > potential performance degradation at large scale. > > > > > > A question I'm pondering is what happens when we move to Java 11 and > > > > someone is still on 2.x? If they want to backport HADOOP-15338 > > > > <https://issues.apache.org/jira/browse/HADOOP-15338> for Java 11 > > > > support to > > > > 2.x, surely not everyone is going to want that (at least not > > > > immediately). > > > > The 2.10 documentation states, "The JVM requirements will not change > > > > across > > > > point releases within the same minor release except if the JVM > > > > version > > > > under question becomes unsupported" [1], so this would warrant a 2.11 > > > > release until Java 8 becomes unsupported (though one could argue that > > > > it is > > > > already unsupported since Oracle is no longer giving public Java 8 > > > > update). > > > > If we don't keep branch-2 around now, would a Java 11 backport be the > > > > catalyst for a branch-2 revival? > > > > > > Not sure if this really leads to any sort of answer from me on > > > > whether > > > > or > > > > not we should keep branch-2 alive, but these are the things that I am > > > > weighing in my mind. For me, the bigger problem beyond having > > > > branch-2 > > > > or > > > > not is committers not being on the same page with where they should > > > > commit > > > > their patches. > > > > > > Eric > > > > > > [1] > > > > > > > > > > > https://hadoop.apache.org/docs/r2.10.0/hadoop-project-dist/hadoop-common/Compatibility.html > > > > [2] > > > > > > > > > > > https://hadoop.apache.org/docs/r3.0.0/hadoop-project-dist/hadoop-common/Compatibility.html > > > > > > On Tue, Nov 19, 2019 at 2:49 PM epa...@apache.org <epa...@apache.org > > > > > > wrote: > > > > > > Hi Konstantin, > > > > > > Sure, I understand those concerns. On the other hand, I worry about > > > > the > > > > stability of 2.10, since we will be on it for a couple of years at > > > > least. > > > > I worry > > > > that some committers may want to put new features into a branch 2 > > > > release, > > > > and without a branch-2, they will go directly into 2.10. Since we > > > > don't > > > > always > > > > catch corner cases or performance problems for some time (usually > > > > not > > > > until > > > > the release is deployed to a busy, 4-thousand node cluster), it > > > > may > > > > be > > > > very > > > > difficult to back out those changes. > > > > > > It sounds like I'm in the minority here, so I'm not nixing the > > > > idea, > > > > but I > > > > do > > > > have these reservations. > > > > > > Thanks, > > > > -Eric > > > > > > > > > > On Tuesday, November 19, 2019, 1:04:15 AM CST, Konstantin Shvachko > > > > < > > > > shv.had...@gmail.com> wrote: > > > > Hi Eric, > > > > > > We had a long discussion on this list regarding making the 2.10 > > > > release the > > > > last of branch-2 releases. We intended 2.10 as a bridge release > > > > between > > > > Hadoop 2 and 3. We may have bug-fix releases or 2.10, but 2.11 is > > > > not in > > > > the picture right now, and many people may object this idea. > > > > > > I understand Jonathan's proposal as an attempt to > > > > 1. eliminate confusion which branches people should commit their > > > > back-ports > > > > to > > > > 2. save engineering effort committing to more branches than > > > > necessary > > > > > > "Branches are cheap" as our founder used to say. If we ever decide > > > > to > > > > release 2.11 we can resurrect the branch. > > > > Until then I am in favor of Jonathan's proposal +1. > > > > > > Thanks, > > > > --Konstantin > > > > > > > > On Mon, Nov 18, 2019 at 10:41 AM Jonathan Hung < > > > > jyhung2...@gmail.com > > > > > > wrote: > > > > > > Thanks Eric for the comments - regarding your concerns, I feel > > > > the > > > > pros > > > > outweigh the cons. To me, the chances of patch releases on 2.10.x > > > > are > > > > much > > > > higher than a new 2.11 minor release. (There didn't seem to be > > > > many > > > > people > > > > outside of our company who expressed interest in getting new > > > > features to > > > > branch-2 prior to the 2.10.0 release.) Even now, a few weeks > > > > after > > > > 2.10.0 > > > > release, there's 29 patches that have gone into branch-2 and 9 in > > > > branch-2.10, so it's already diverged quite a bit. > > > > > > In any case, we can always reverse this decision if we really > > > > need > > > > to, by > > > > recreating branch-2. But this proposal would reduce a lot of > > > > confusion > > > > IMO. > > > > > > Jonathan Hung > > > > > > > > On Fri, Nov 15, 2019 at 11:41 AM epa...@apache.org < > > > > epa...@apache.org> > > > > wrote: > > > > > > Thanks Jonathan for opening the discussion. > > > > > > I am not in favor of this proposal. 2.10 was very recently > > > > released, > > > > and > > > > moving to 2.10 will take some time for the community. It seems > > > > premature > > > > to > > > > make a decision at this point that there will never be a need > > > > for a > > > > 2.11 > > > > release. > > > > > > -Eric > > > > > > > > On Thursday, November 14, 2019, 8:51:59 PM CST, Jonathan Hung > > > > < > > > > jyhung2...@gmail.com> wrote: > > > > > > Hi folks, > > > > > > Given the release of 2.10.0, and the fact that it's intended to > > > > be a > > > > bridge > > > > release to Hadoop 3.x [1], I'm proposing we make 2.10.x the > > > > last > > > > minor > > > > release line in branch-2. Currently, the main issue is that > > > > there's > > > > many > > > > fixes going into branch-2 (the theoretical 2.11.0) that's not > > > > going > > > > into > > > > branch-2.10 (which will become 2.10.1), so the fixes in > > > > branch-2 > > > > will > > > > likely never see the light of day unless they are backported to > > > > branch-2.10. > > > > > > To do this, I propose we: > > > > > > - Delete branch-2.10 > > > > - Rename branch-2 to branch-2.10 > > > > - Set version in the new branch-2.10 to 2.10.1-SNAPSHOT > > > > > > This way we get all the current branch-2 fixes into the 2.10.x > > > > release > > > > line. Then the commit chain will look like: trunk -> branch-3.2 > > > > -> > > > > branch-3.1 -> branch-2.10 -> branch-2.9 -> branch-2.8 > > > > > > Thoughts? > > > > > > Jonathan Hung > > > > > > [1] > > > > > > > > https://www.mail-archive.com/yarn-dev@hadoop.apache.org/msg29479.html > > > > > > > > > > > > --------------------------------------------------------------------- > > > > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > > > > For additional commands, e-mail: common-dev-h...@hadoop.apache.org > > > > > > > > > > > > > > >
