Tim Armstrong has uploaded a new patch set (#7). Change subject: IMPALA-5184: build fe against both Hive 1 & 2 APIs ......................................................................
IMPALA-5184: build fe against both Hive 1 & 2 APIs This adds a compatibility shim layer with Hive 1 and Hive 2 implementations. The version-specific code lives in fe/src/compat-hive-$IMPALA_HIVE_MAJOR_VERSION and common/thrift/hive-$IMPALA_HIVE_MAJOR_VERSION-api/ The shim adds wrapper methods to handle differing method signatures and and config variables that changed slightly. For these, MetastoreCompatShim has methods that abstract away the Some thrift classes were also moved from the 'cli' to 'rpc' package. We work around these by implementing subclasses with the same name in a different package for compatibility or by implementing shim methods that operate on the classes. We also need to change the package in the TCLIService.thrift, which is done with a search-and-replace. Testing: Made sure that I could build Impala on master as normal, and also with the following config overrides in bin/impala-config-local.sh: export IMPALA_HADOOP_VERSION=3.0.0-alpha1-cdh6.x-SNAPSHOT export IMPALA_HBASE_VERSION=2.0.0-cdh6.x-SNAPSHOT export IMPALA_HIVE_MAJOR_VERSION=2 export IMPALA_HIVE_VERSION=2.1.1-cdh6.x-SNAPSHOT export IMPALA_SENTRY_VERSION=1.5.1-cdh6.x-SNAPSHOT export IMPALA_PARQUET_VERSION=1.5.0-cdh5.10.1 export KUDU_JAVA_VERSION=1.4.0-cdh6.x-SNAPSHOT I manually assembled the dependencies by copying the following files from the Hive 2 source and from a Hadoop 3 build: $CDH_COMPONENTS_HOME/hive-2.1.0-cdh6.x-SNAPSHOT/src/metastore/if/hive_metastore.thrift $CDH_COMPONENTS_HOME/hadoop-3.0.0-alpha1-cdh6.x-SNAPSHOT/lib/native/* $CDH_COMPONENTS_HOME/hadoop-3.0.0-alpha1-cdh6.x-SNAPSHOT/include/hdfs.h Change-Id: Ifbc265281c04fe3136bc3c920dbac966742ce09a --- M bin/impala-config.sh M common/thrift/.gitignore M common/thrift/CMakeLists.txt R common/thrift/hive-1-api/TCLIService.thrift M fe/pom.xml A fe/src/compat-hive-1/java/org/apache/hive/service/rpc/thrift/TGetCatalogsReq.java A fe/src/compat-hive-1/java/org/apache/hive/service/rpc/thrift/TGetColumnsReq.java A fe/src/compat-hive-1/java/org/apache/hive/service/rpc/thrift/TGetFunctionsReq.java A fe/src/compat-hive-1/java/org/apache/hive/service/rpc/thrift/TGetInfoReq.java A fe/src/compat-hive-1/java/org/apache/hive/service/rpc/thrift/TGetSchemasReq.java A fe/src/compat-hive-1/java/org/apache/hive/service/rpc/thrift/TGetTablesReq.java A fe/src/compat-hive-1/java/org/apache/impala/compat/MetaStoreCompatShim.java A fe/src/compat-hive-2/java/org/apache/impala/compat/MetaStoreCompatShim.java M fe/src/main/java/org/apache/impala/analysis/ColumnDef.java M fe/src/main/java/org/apache/impala/analysis/CreateDataSrcStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateDbStmt.java M fe/src/main/java/org/apache/impala/analysis/DropDataSrcStmt.java M fe/src/main/java/org/apache/impala/analysis/Subquery.java M fe/src/main/java/org/apache/impala/analysis/TableName.java M fe/src/main/java/org/apache/impala/analysis/TypeDef.java M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/service/Frontend.java M fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java M fe/src/test/java/org/apache/impala/analysis/AuthorizationTest.java M fe/src/test/java/org/apache/impala/analysis/ParserTest.java M fe/src/test/java/org/apache/impala/service/FrontendTest.java 27 files changed, 390 insertions(+), 71 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/38/5538/7 -- To view, visit http://gerrit.cloudera.org:8080/5538 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ifbc265281c04fe3136bc3c920dbac966742ce09a Gerrit-PatchSet: 7 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tim Armstrong <tarmstr...@cloudera.com>
