As for Quanlong's question, I think the answer is however the folks who want to do the work prefer to do it. As you noticed in the CDH changelists, Cloudera's distribution has opted for something more like approach (a), choosing to backport individual features. For a while, we were doing automation for cherry-picking things automatically, and it got tedious enough that we decided to turn it off.
On Sun, Jan 27, 2019 at 7:37 PM Paul Rogers <[email protected]> wrote: > Hi Quanlong, > > Thanks for the suggestion. I wonder if there is a third strategy: > > c) Isolate the Hadoop 2.x/3.x differences into clearly-defined driver > layer so that basically all of 3.x can be applied to the 2.x branch. Said > another way, a single source base can work against either Hadoop 2.x or > 3.x, with the build (C++) or runtime (Java) choosing the proper “driver” > classes. > We had such a layer for a while, where Impala master could be built against either Hadoop3 or Hadoop2. We decided to clean it up in commit e4ae605b083ab536c68552e37ca3c46f6bff4c76. commit e4ae605b083ab536c68552e37ca3c46f6bff4c76 Author: Fredy Wijaya <[email protected]> Date: Thu Jul 12 17:01:13 2018 -0700 IMPALA-7295: Remove IMPALA_MINICLUSTER_PROFILE=2 This patch removes the use of IMPALA_MINICLUSTER_PROFILE. The code that uses IMPALA_MINICLUSTER_PROFILE=2 is removed and it defaults to code from IMPALA_MINICLUSTER_PROFILE=3. In order to reduce having too many code changes in this patch, there is no code change for the shims. The shims for IMPALA_MINICLUSTER_PROFILE=3 automatically become the default implementation. Testing: - Ran core and exhaustive tests Change-Id: Iba4a81165b3d2012dc04d4115454372c41e39f08 Reviewed-on: http://gerrit.cloudera.org:8080/10940 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]>
