Hi Bharath, Thank you a lot for your notice! However, I've gone through the commits of cdh branch before and found that this patch is also picked: https://github.com/cloudera/Impala/commit/f8a318d4f75e22a963b9cf4786ef07d2cd6bd93c . Is this really a compatibility breaking change?
I'm also concern that the TestDescribeTableResults it introduced is too strictly that may cause troubles. However, I found two later commits (IMPALA-7143 and IMPALA-7144) would fix this. I'm going to cherry-pick these two and IMPALA-7676 (thanks Fredy's advise too!) right after https://gerrit.cloudera.org/c/12292/ is merged. Please let me know if this will go astray. Thanks! On Wed, Jan 30, 2019 at 12:36 AM Bharath Vissapragada <[email protected]> wrote: > On Mon, Jan 28, 2019 at 11:36 PM Quanlong Huang <[email protected]> > wrote: > > > >>For the first patch, "0b540b025 IMPALA-7128 (part 1) Refactor > interfaces > > for Db, View, Table, Partition", the cherry-pick conflicts is due to the > > revert of IMPALA-6479 in 2.x. I'm testing branch-2.x with IMPALA-6479 > being > > picked back. Does anyone know why we revert it? (I also comment in the > > JIRA). > > > > > >There are test failures. I guess it's the reason. Hopefully, > > cdh-5.16.1-release already picked up this patch, which provides some > > pointers :) > > > > I fix the test failures and create a review at > > https://gerrit.cloudera.org/c/12292/ > > Waiting for Jenkins maintenance to finish and then run a GVO. Hopes > someone > > can join and have a look! > > > > On Tue, Jan 29, 2019 at 7:39 AM Quanlong Huang <[email protected]> > > wrote: > > > > > >For the first patch, "0b540b025 IMPALA-7128 (part 1) Refactor > interfaces > > > for Db, View, Table, Partition", the cherry-pick conflicts is due to > the > > > revert of IMPALA-6479 in 2.x. I'm testing branch-2.x with IMPALA-6479 > > being > > > picked back. Does anyone know why we revert it? (I also comment in the > > > JIRA). > > > > It was reverted because it is a compatibility breaking change. We typically > try not to introduce such behavioral changes in the same major version line > as that can cause upgrade issues. > > > > > > > > There are test failures. I guess it's the reason. Hopefully, > > > cdh-5.16.1-release already picked up this patch, which provides some > > > pointers :) > > > > I work at Cloudera and we've gone through this exercise before. It is > annoying to resolve the conflicts, so you can reuse our work and save some > time. > https://github.com/cloudera/Impala/tree/cdh5-2.12.0_5.16.0 > > > > > > > > On Mon, Jan 28, 2019 at 10:51 PM Quanlong Huang < > [email protected] > > > > > > wrote: > > > > > >> Yes, there are two discussion threads before that are relative to > this. > > >> One for stopping the cherrypick-2.x-and-test jenkins job: > > >> > > >> > > > https://lists.apache.org/thread.html/2b4b62d4c07661b27a5203618cb0425a429f6460f2eb505acbcd26c6@%3Cdev.impala.apache.org%3E > > >> > > >> The other for removing support for hadoop 2 in master branch: > > >> > > > https://lists.apache.org/thread.html/49f9b68ed3d6d2c0fdee16a877b259922545e4824e1233479227a657@%3Cdev.impala.apache.org%3E > > >> > > >> I'm +1 with the second thread that we only support Hadoop 2 in > > branch-2.x > > >> and support Hadoop 3 in the master branch to be more focused. I'm also > > >> agree with Paul's concern. It's such a dilemma that if we skip some > > >> commits, things will be harder and harder as we moving forward; if we > > >> cherry-pick, review, and test the commits one by one, branch-2.x will > > never > > >> catch up the master branch, which is an obstacle if someone (like me) > > wants > > >> to backport his/her new patch to branch-2.x but waits too long and > > finally > > >> fogets details of the patch. > > >> > > >> I roughly investigated how other systems deal with multiple branches. > > The > > >> efforts to backport a patch could be the same for the original patch. > > It's > > >> not a easy go, so the Hive community declares that > > >> "The decision to port a feature from master to branch-1 is at the > > >> discretion of the contributor and committer. However no features that > > break > > >> backwards compatibility will be accepted on branch-1." > > >> > > >> I think it's a chance to understand more parts of Impala by learning > and > > >> backporting the patches, since they have execellent commit messages > and > > >> were strictly reviewed. So I volunteer for the job to move forward the > > >> branch-2.x. Hopes patch authors could give some pointers when I'm > > blocked! > > >> I'll try approach (b) first and switch to (a) when (b) becomes > > impossible > > >> after too many commits are skipped. I'll confirm with the author if I > > think > > >> a patch should be skipped. > > >> > > >> For the first patch, "0b540b025 IMPALA-7128 (part 1) Refactor > interfaces > > >> for Db, View, Table, Partition", the cherry-pick conflicts is due to > the > > >> revert of IMPALA-6479 in 2.x. I'm testing branch-2.x with IMPALA-6479 > > being > > >> picked back. Does anyone know why we revert it? (I also comment in the > > >> JIRA). > > >> > > >> On Mon, Jan 28, 2019 at 12:43 PM Philip Zeyliger <[email protected] > > > > >> wrote: > > >> > > >>> As for Quanlong's question, I think the answer is however the folks > who > > >>> want to do the work prefer to do it. As you noticed in the CDH > > >>> changelists, > > >>> Cloudera's distribution has opted for something more like approach > (a), > > >>> choosing to backport individual features. For a while, we were doing > > >>> automation for cherry-picking things automatically, and it got > tedious > > >>> enough that we decided to turn it off. > > >>> > > >>> On Sun, Jan 27, 2019 at 7:37 PM Paul Rogers <[email protected]> > > >>> wrote: > > >>> > > >>> > Hi Quanlong, > > >>> > > > >>> > Thanks for the suggestion. I wonder if there is a third strategy: > > >>> > > > >>> > c) Isolate the Hadoop 2.x/3.x differences into clearly-defined > driver > > >>> > layer so that basically all of 3.x can be applied to the 2.x > branch. > > >>> Said > > >>> > another way, a single source base can work against either Hadoop > 2.x > > or > > >>> > 3.x, with the build (C++) or runtime (Java) choosing the proper > > >>> “driver” > > >>> > classes. > > >>> > > > >>> > > >>> We had such a layer for a while, where Impala master could be built > > >>> against > > >>> either Hadoop3 or Hadoop2. We decided to clean it up in commit > > >>> e4ae605b083ab536c68552e37ca3c46f6bff4c76. > > >>> > > >>> commit e4ae605b083ab536c68552e37ca3c46f6bff4c76 > > >>> Author: Fredy Wijaya <[email protected]> > > >>> Date: Thu Jul 12 17:01:13 2018 -0700 > > >>> > > >>> IMPALA-7295: Remove IMPALA_MINICLUSTER_PROFILE=2 > > >>> > > >>> This patch removes the use of IMPALA_MINICLUSTER_PROFILE. The > code > > >>> that > > >>> uses IMPALA_MINICLUSTER_PROFILE=2 is removed and it defaults to > > code > > >>> from > > >>> IMPALA_MINICLUSTER_PROFILE=3. In order to reduce having too many > > code > > >>> changes in this patch, there is no code change for the shims. The > > >>> shims > > >>> for IMPALA_MINICLUSTER_PROFILE=3 automatically become the default > > >>> implementation. > > >>> > > >>> Testing: > > >>> - Ran core and exhaustive tests > > >>> > > >>> Change-Id: Iba4a81165b3d2012dc04d4115454372c41e39f08 > > >>> Reviewed-on: http://gerrit.cloudera.org:8080/10940 > > >>> Reviewed-by: Impala Public Jenkins < > > >>> [email protected]> > > >>> Tested-by: Impala Public Jenkins < > > [email protected] > > >>> > > > >>> > > >> > > >
