Fucun Chu has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/17793 )
Change subject: IMPALA-10870: Add Apache Hive 3.1.2 to the minicluster ...................................................................... IMPALA-10870: Add Apache Hive 3.1.2 to the minicluster This patch modifies the minicluster script to optionally use Apache Hive 3.1.2 instead of CDP Hive 3.1.3. In order to make sure that existing setups don't break this is enabled via a environment variable override to bin/impala-config.sh. When the environment variable USE_APACHE_HIVE is set to true the bootstrap_toolchain script downloads Apache Hive 3.1.2 tarballs and extracts it in the toolchain directory. These binaries are used to start the Hive services (Hiveserver2 and metastore). The default is CDP Hive 3.1.3 Since CDP Hive 3 uses some features of Apache Hive 4, this patch uses a different database name so that it is easy to switch from working from one environment which uses CDP Hive 3.1.3 metastore to another which usese Apache Hive 3.1.2 metastore. In order to start a minicluster which uses Apache Hive 3.1.2 users should follow the steps below: 1. Make sure that minicluster, if running, is stopped before you run the following commands. 2. Open a new terminal and run following commands. > export USE_APACHE_HIVE=true > source bin/impala-config.sh > bin/bootstrap_toolchain.py The above command downloads the Apache Hive 3.1.2 tarballs and extracts them in toolchain/apache_components directory. > rm $HIVE_HOME/lib/guava-*jar > cp $HADOOP_HOME/share/hadoop/hdfs/lib/guava-*.jar $HIVE_HOME/lib/ The above command is to fix HIVE-22915 > bin/create-test-configuration.sh -create_metastore The above step should provide "-create-metastore" only the first time so that a new metastore db is created and the Apache Hive 3.1.2 schema is initialized. > testdata/bin/run-all.sh Follow-up: - Add MetastoreShim to support Apache Hive 3.x in IMPALA-10871 Tests: - Made sure that the cluster comes up with Apache Hive 3.1.2 when the steps above are performed. - Made sure that existing scripts work as they do currently when argument is not provided. Change-Id: I1978909589ecacb15d32d874e97f050a85adf1f6 --- M bin/bootstrap_toolchain.py M bin/impala-config.sh M buildall.sh 3 files changed, 84 insertions(+), 23 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/93/17793/4 -- To view, visit http://gerrit.cloudera.org:8080/17793 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I1978909589ecacb15d32d874e97f050a85adf1f6 Gerrit-Change-Number: 17793 Gerrit-PatchSet: 4 Gerrit-Owner: Fucun Chu <chufu...@hotmail.com> Gerrit-Reviewer: Fucun Chu <chufu...@hotmail.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com>