Hi folks, I am still seeing some changes are being committed to branch-2. I'd like to delete the source code from branch-2 to avoid mistakes. https://issues.apache.org/jira/browse/HADOOP-16988
-Akira On Wed, Jan 1, 2020 at 2:38 AM Ayush Saxena <ayush...@gmail.com> wrote: > Hi Jim, > Thanx for catching, I have configured the build to run on branch-2.10. > > -Ayush > > On Tue, 31 Dec 2019 at 22:50, Jim Brennan <james.bren...@verizonmedia.com> > wrote: > >> It looks like QBT tests are still being run on branch-2 ( >> https://builds.apache.org/view/H-L/view/Hadoop/job/hadoop-qbt-branch2-java7-linux-x86/), >> and they are not very helpful at this point. >> Can we change the QBT tests to run against branch-2.10 instead? >> >> Jim >> >> On Mon, Dec 23, 2019 at 7:44 PM Akira Ajisaka <aajis...@apache.org> >> wrote: >> >>> Thank you, Ayush. >>> >>> I understand we should keep branch-2 as is, as well as master. >>> >>> -Akira >>> >>> >>> On Mon, Dec 23, 2019 at 9:14 PM Ayush Saxena <ayush...@gmail.com> wrote: >>> >>> > Hi Akira >>> > Seems there was an INFRA ticket for that. INFRA-19581, >>> > But the INFRA people closed as wont do and yes, the branch is >>> protected, >>> > we can’t delete it directly. >>> > >>> > Ref: https://issues.apache.org/jira/browse/INFRA-19581 >>> > >>> > -Ayush >>> > >>> > On 23-Dec-2019, at 5:03 PM, Akira Ajisaka <aajis...@apache.org> wrote: >>> > >>> > Thank you for your work, Jonathan. >>> > >>> > I found branch-2 has been unintentionally pushed again. Would you >>> remove >>> > it? >>> > I think the branch should be protected if possible. >>> > >>> > -Akira >>> > >>> > On Tue, Dec 10, 2019 at 5:17 AM Jonathan Hung <jyhung2...@gmail.com> >>> > wrote: >>> > >>> > It's done. The new commit chain is: trunk -> branch-3.2 -> branch-3.1 >>> -> >>> > >>> > branch-2.10 -> branch-2.9 -> branch-2.8 (branch-2 no longer exists, >>> please >>> > >>> > don't try to commit to it) >>> > >>> > >>> > Completed procedure: >>> > >>> > >>> > - Verified everything in old branch-2.10 was in old branch-2 >>> > >>> > - Delete old branch-2.10 >>> > >>> > - Rename branch-2 to (new) branch-2.10 >>> > >>> > - Set version in new branch-2.10 to 2.10.1-SNAPSHOT >>> > >>> > - Renamed fix versions from 2.11.0 to 2.10.1 >>> > >>> > - Removed 2.11.0 as a version in HADOOP/YARN/HDFS/MAPREDUCE >>> > >>> > >>> > >>> > Jonathan Hung >>> > >>> > >>> > >>> > On Wed, Dec 4, 2019 at 10:55 AM Jonathan Hung <jyhung2...@gmail.com> >>> > >>> > wrote: >>> > >>> > >>> > FYI, starting the rename process, beginning with INFRA-19521. >>> > >>> > >>> > Jonathan Hung >>> > >>> > >>> > >>> > On Wed, Nov 27, 2019 at 12:15 PM Konstantin Shvachko < >>> > >>> > shv.had...@gmail.com> >>> > >>> > wrote: >>> > >>> > >>> > Hey guys, >>> > >>> > >>> > I think we diverged a bit from the initial topic of this discussion, >>> > >>> > which is removing branch-2.10, and changing the version of branch-2 >>> from >>> > >>> > 2.11.0-SNAPSHOT to 2.10.1-SNAPSHOT. >>> > >>> > Sounds like the subject line for this thread "Making 2.10 the last >>> minor >>> > >>> > 2.x release" confused people. >>> > >>> > It is in fact a wider matter that can be discussed when somebody >>> > >>> > actually >>> > >>> > proposes to release 2.11, which I understand nobody does at the moment. >>> > >>> > >>> > So if anybody objects removing branch-2.10 please make an argument. >>> > >>> > Otherwise we should go ahead and just do it next week. >>> > >>> > I see people still struggling to keep branch-2 and branch-2.10 in sync. >>> > >>> > >>> > Thanks, >>> > >>> > --Konstantin >>> > >>> > >>> > On Thu, Nov 21, 2019 at 3:49 PM Jonathan Hung <jyhung2...@gmail.com> >>> > >>> > wrote: >>> > >>> > >>> > Thanks for the detailed thoughts, everyone. >>> > >>> > >>> > Eric (Badger), my understanding is the same as yours re. minor vs patch >>> > >>> > releases. As for putting features into minor/patch releases, if we >>> > >>> > keep the >>> > >>> > convention of putting new features only into minor releases, my >>> > >>> > assumption >>> > >>> > is still that it's unlikely people will want to get them into branch-2 >>> > >>> > (based on the 2.10.0 release process). For the java 11 issue, we >>> > >>> > haven't >>> > >>> > even really removed support for java 7 in branch-2 (much less java 8), >>> > >>> > so I >>> > >>> > feel moving to java 11 would go along with a move to branch 3. And as >>> > >>> > you >>> > >>> > mentioned, if people really want to use java 11 on branch-2, we can >>> > >>> > always >>> > >>> > revive branch-2. But for now I think the convenience of not needing to >>> > >>> > port >>> > >>> > to both branch-2 and branch-2.10 (and below) outweighs the cost of >>> > >>> > potentially needing to revive branch-2. >>> > >>> > >>> > Jonathan Hung >>> > >>> > >>> > >>> > On Wed, Nov 20, 2019 at 10:50 AM Eric Yang <ey...@cloudera.com> wrote: >>> > >>> > >>> > +1 for 2.10.x as last release for 2.x version. >>> > >>> > >>> > Software would become more compatible when more companies stress test >>> > >>> > the same software and making improvements in trunk. Some may be extra >>> > >>> > caution on moving up the version because obligation internally to keep >>> > >>> > things running. Company obligation should not be the driving force to >>> > >>> > maintain Hadoop branches. There is no proper collaboration in the >>> > >>> > community when every name brand company maintains its own Hadoop 2.x >>> > >>> > version. I think it would be more healthy for the community to >>> > >>> > reduce the >>> > >>> > branch forking and spend energy on trunk to harden the software. >>> > >>> > This will >>> > >>> > give more confidence to move up the version than trying to fix n >>> > >>> > permutations breakage like Flash fixing the timeline. >>> > >>> > >>> > Apache license stated, there is no warranty of any kind for code >>> > >>> > contributions. Fewer community release process should improve >>> > >>> > software >>> > >>> > quality when eyes are on trunk, and help steering toward the same end >>> > >>> > goals. >>> > >>> > >>> > regards, >>> > >>> > Eric >>> > >>> > >>> > >>> > >>> > On Tue, Nov 19, 2019 at 3:03 PM Eric Badger >>> > >>> > <ebad...@verizonmedia.com.invalid> wrote: >>> > >>> > >>> > Hello all, >>> > >>> > >>> > Is it written anywhere what the difference is between a minor release >>> > >>> > and a >>> > >>> > point/dot/maintenance (I'll use "point" from here on out) release? I >>> > >>> > have >>> > >>> > looked around and I can't find anything other than some compatibility >>> > >>> > documentation in 2.x that has since been removed in 3.x [1] [2]. I >>> > >>> > think >>> > >>> > this would help shape my opinion on whether or not to keep branch-2 >>> > >>> > alive. >>> > >>> > My current understanding is that we can't really break compatibility >>> > >>> > in >>> > >>> > either a minor or point release. But the only mention of the >>> > >>> > difference >>> > >>> > between minor and point releases is how to deal with Stable, >>> > >>> > Evolving, >>> > >>> > and >>> > >>> > Unstable tags, and how to deal with changing default configuration >>> > >>> > values. >>> > >>> > So it seems like there really isn't a big official difference between >>> > >>> > the >>> > >>> > two. In my mind, the functional difference between the two is that >>> > >>> > the >>> > >>> > minor releases may have added features and rewrites, while the point >>> > >>> > releases only have bug fixes. This might be an incorrect >>> > >>> > understanding, but >>> > >>> > that's what I have gathered from watching the releases over the last >>> > >>> > few >>> > >>> > years. Whether or not this is a correct understanding, I think that >>> > >>> > this >>> > >>> > needs to be documented somewhere, even if it is just a convention. >>> > >>> > >>> > Given my assumed understanding of minor vs point releases, here are >>> > >>> > the >>> > >>> > pros/cons that I can think of for having a branch-2. Please add on or >>> > >>> > correct me for anything you feel is missing or inadequate. >>> > >>> > Pros: >>> > >>> > - Features/rewrites/higher-risk patches are less likely to be put >>> > >>> > into >>> > >>> > 2.10.x >>> > >>> > - It is less necessary to move to 3.x >>> > >>> > >>> > Cons: >>> > >>> > - Bug fixes are less likely to be put into 2.10.x >>> > >>> > - An extra branch to maintain >>> > >>> > - Committers have an extra branch (5 vs 4 total branches) to commit >>> > >>> > patches to if they should go all the way back to 2.10.x >>> > >>> > - It is less necessary to move to 3.x >>> > >>> > >>> > So on the one hand you get added stability in fewer features being >>> > >>> > committed to 2.10.x, but then on the other you get fewer bug fixes >>> > >>> > being >>> > >>> > committed. In a perfect world, we wouldn't have to make this >>> > >>> > tradeoff. >>> > >>> > But >>> > >>> > we don't live in a perfect world and committers will make mistakes >>> > >>> > either >>> > >>> > because of lack of knowledge or simply because they made a mistake. >>> > >>> > If >>> > >>> > we >>> > >>> > have a branch-2, committers will forget, not know to, or choose not >>> > >>> > to >>> > >>> > (for >>> > >>> > whatever reason) commit valid bug fixes back all the way to >>> > >>> > branch-2.10. If >>> > >>> > we don't have a branch-2, committers who want their borderline risky >>> > >>> > feature in the 2.x line will err on the side of putting it into >>> > >>> > branch-2.10 >>> > >>> > instead of proposing the creation of a branch-2. Clearly I have made >>> > >>> > quite >>> > >>> > a few assumptions here based on my own experiences, so I would like >>> > >>> > to >>> > >>> > hear >>> > >>> > if others have similar or opposing views. >>> > >>> > >>> > As far as 3.x goes, to me it seems like some of the reasoning for >>> > >>> > killing >>> > >>> > branch-2 is due to an effort to push the community towards 3.x. This >>> > >>> > is why >>> > >>> > I have added movement to 3.x as both a pro and a con. As a community >>> > >>> > trying >>> > >>> > to move forward, keeping as many companies on similar branches as >>> > >>> > possible >>> > >>> > is a good way to make sure the code is well-tested. However, from a >>> > >>> > stability point of view, moving to 3.x is still scary and being able >>> > >>> > to >>> > >>> > stay on 2.x until you are comfortable to move is very nice. The >>> > >>> > 2.10.0 >>> > >>> > bridge release effort has been very good at making it possible for >>> > >>> > people >>> > >>> > to move from 2.x in 3.x, but the diff between 2.x and 3.x is so large >>> > >>> > that >>> > >>> > it is reasonable for companies to want to be extra cautious with 3.x >>> > >>> > due to >>> > >>> > potential performance degradation at large scale. >>> > >>> > >>> > A question I'm pondering is what happens when we move to Java 11 and >>> > >>> > someone is still on 2.x? If they want to backport HADOOP-15338 >>> > >>> > <https://issues.apache.org/jira/browse/HADOOP-15338> for Java 11 >>> > >>> > support to >>> > >>> > 2.x, surely not everyone is going to want that (at least not >>> > >>> > immediately). >>> > >>> > The 2.10 documentation states, "The JVM requirements will not change >>> > >>> > across >>> > >>> > point releases within the same minor release except if the JVM >>> > >>> > version >>> > >>> > under question becomes unsupported" [1], so this would warrant a 2.11 >>> > >>> > release until Java 8 becomes unsupported (though one could argue that >>> > >>> > it is >>> > >>> > already unsupported since Oracle is no longer giving public Java 8 >>> > >>> > update). >>> > >>> > If we don't keep branch-2 around now, would a Java 11 backport be the >>> > >>> > catalyst for a branch-2 revival? >>> > >>> > >>> > Not sure if this really leads to any sort of answer from me on >>> > >>> > whether >>> > >>> > or >>> > >>> > not we should keep branch-2 alive, but these are the things that I am >>> > >>> > weighing in my mind. For me, the bigger problem beyond having >>> > >>> > branch-2 >>> > >>> > or >>> > >>> > not is committers not being on the same page with where they should >>> > >>> > commit >>> > >>> > their patches. >>> > >>> > >>> > Eric >>> > >>> > >>> > [1] >>> > >>> > >>> > >>> > >>> > >>> https://hadoop.apache.org/docs/r2.10.0/hadoop-project-dist/hadoop-common/Compatibility.html >>> > >>> > [2] >>> > >>> > >>> > >>> > >>> > >>> https://hadoop.apache.org/docs/r3.0.0/hadoop-project-dist/hadoop-common/Compatibility.html >>> > >>> > >>> > On Tue, Nov 19, 2019 at 2:49 PM epa...@apache.org <epa...@apache.org >>> > >>> > >>> > wrote: >>> > >>> > >>> > Hi Konstantin, >>> > >>> > >>> > Sure, I understand those concerns. On the other hand, I worry about >>> > >>> > the >>> > >>> > stability of 2.10, since we will be on it for a couple of years at >>> > >>> > least. >>> > >>> > I worry >>> > >>> > that some committers may want to put new features into a branch 2 >>> > >>> > release, >>> > >>> > and without a branch-2, they will go directly into 2.10. Since we >>> > >>> > don't >>> > >>> > always >>> > >>> > catch corner cases or performance problems for some time (usually >>> > >>> > not >>> > >>> > until >>> > >>> > the release is deployed to a busy, 4-thousand node cluster), it >>> > >>> > may >>> > >>> > be >>> > >>> > very >>> > >>> > difficult to back out those changes. >>> > >>> > >>> > It sounds like I'm in the minority here, so I'm not nixing the >>> > >>> > idea, >>> > >>> > but I >>> > >>> > do >>> > >>> > have these reservations. >>> > >>> > >>> > Thanks, >>> > >>> > -Eric >>> > >>> > >>> > >>> > >>> > On Tuesday, November 19, 2019, 1:04:15 AM CST, Konstantin Shvachko >>> > >>> > < >>> > >>> > shv.had...@gmail.com> wrote: >>> > >>> > Hi Eric, >>> > >>> > >>> > We had a long discussion on this list regarding making the 2.10 >>> > >>> > release the >>> > >>> > last of branch-2 releases. We intended 2.10 as a bridge release >>> > >>> > between >>> > >>> > Hadoop 2 and 3. We may have bug-fix releases or 2.10, but 2.11 is >>> > >>> > not in >>> > >>> > the picture right now, and many people may object this idea. >>> > >>> > >>> > I understand Jonathan's proposal as an attempt to >>> > >>> > 1. eliminate confusion which branches people should commit their >>> > >>> > back-ports >>> > >>> > to >>> > >>> > 2. save engineering effort committing to more branches than >>> > >>> > necessary >>> > >>> > >>> > "Branches are cheap" as our founder used to say. If we ever decide >>> > >>> > to >>> > >>> > release 2.11 we can resurrect the branch. >>> > >>> > Until then I am in favor of Jonathan's proposal +1. >>> > >>> > >>> > Thanks, >>> > >>> > --Konstantin >>> > >>> > >>> > >>> > On Mon, Nov 18, 2019 at 10:41 AM Jonathan Hung < >>> > >>> > jyhung2...@gmail.com >>> > >>> > >>> > wrote: >>> > >>> > >>> > Thanks Eric for the comments - regarding your concerns, I feel >>> > >>> > the >>> > >>> > pros >>> > >>> > outweigh the cons. To me, the chances of patch releases on 2.10.x >>> > >>> > are >>> > >>> > much >>> > >>> > higher than a new 2.11 minor release. (There didn't seem to be >>> > >>> > many >>> > >>> > people >>> > >>> > outside of our company who expressed interest in getting new >>> > >>> > features to >>> > >>> > branch-2 prior to the 2.10.0 release.) Even now, a few weeks >>> > >>> > after >>> > >>> > 2.10.0 >>> > >>> > release, there's 29 patches that have gone into branch-2 and 9 in >>> > >>> > branch-2.10, so it's already diverged quite a bit. >>> > >>> > >>> > In any case, we can always reverse this decision if we really >>> > >>> > need >>> > >>> > to, by >>> > >>> > recreating branch-2. But this proposal would reduce a lot of >>> > >>> > confusion >>> > >>> > IMO. >>> > >>> > >>> > Jonathan Hung >>> > >>> > >>> > >>> > On Fri, Nov 15, 2019 at 11:41 AM epa...@apache.org < >>> > >>> > epa...@apache.org> >>> > >>> > wrote: >>> > >>> > >>> > Thanks Jonathan for opening the discussion. >>> > >>> > >>> > I am not in favor of this proposal. 2.10 was very recently >>> > >>> > released, >>> > >>> > and >>> > >>> > moving to 2.10 will take some time for the community. It seems >>> > >>> > premature >>> > >>> > to >>> > >>> > make a decision at this point that there will never be a need >>> > >>> > for a >>> > >>> > 2.11 >>> > >>> > release. >>> > >>> > >>> > -Eric >>> > >>> > >>> > >>> > On Thursday, November 14, 2019, 8:51:59 PM CST, Jonathan Hung >>> > >>> > < >>> > >>> > jyhung2...@gmail.com> wrote: >>> > >>> > >>> > Hi folks, >>> > >>> > >>> > Given the release of 2.10.0, and the fact that it's intended to >>> > >>> > be a >>> > >>> > bridge >>> > >>> > release to Hadoop 3.x [1], I'm proposing we make 2.10.x the >>> > >>> > last >>> > >>> > minor >>> > >>> > release line in branch-2. Currently, the main issue is that >>> > >>> > there's >>> > >>> > many >>> > >>> > fixes going into branch-2 (the theoretical 2.11.0) that's not >>> > >>> > going >>> > >>> > into >>> > >>> > branch-2.10 (which will become 2.10.1), so the fixes in >>> > >>> > branch-2 >>> > >>> > will >>> > >>> > likely never see the light of day unless they are backported to >>> > >>> > branch-2.10. >>> > >>> > >>> > To do this, I propose we: >>> > >>> > >>> > - Delete branch-2.10 >>> > >>> > - Rename branch-2 to branch-2.10 >>> > >>> > - Set version in the new branch-2.10 to 2.10.1-SNAPSHOT >>> > >>> > >>> > This way we get all the current branch-2 fixes into the 2.10.x >>> > >>> > release >>> > >>> > line. Then the commit chain will look like: trunk -> branch-3.2 >>> > >>> > -> >>> > >>> > branch-3.1 -> branch-2.10 -> branch-2.9 -> branch-2.8 >>> > >>> > >>> > Thoughts? >>> > >>> > >>> > Jonathan Hung >>> > >>> > >>> > [1] >>> > >>> > >>> > >>> > https://www.mail-archive.com/yarn-dev@hadoop.apache.org/msg29479.html >>> > >>> > >>> > >>> > >>> > >>> > --------------------------------------------------------------------- >>> > >>> > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org >>> > >>> > For additional commands, e-mail: common-dev-h...@hadoop.apache.org >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> >>
