Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/13005 )
Change subject: IMPALA-8369 : Impala should be able to interoperate with Hive 3.1.0 ...................................................................... Patch Set 14: (3 comments) http://gerrit.cloudera.org:8080/#/c/13005/14/bin/impala-config.sh File bin/impala-config.sh: http://gerrit.cloudera.org:8080/#/c/13005/14/bin/impala-config.sh@a747 PS14, Line 747: I think we should retain this line for now. Some build systems pass in DOWNLOAD_CDH_COMPONENTS=false, and it is good to know about it. http://gerrit.cloudera.org:8080/#/c/13005/14/bin/impala-config.sh@226 PS14, Line 226: export HIVE_HOME="$IMPALA_TOOLCHAIN/cdh_components-${CDH_BUILD_NUMBER}/hive-\ : ${IMPALA_HIVE_VERSION}" : export HIVE_METASTORE_THRIFT_DIR=$CDH_COMPONENTS_HOME/hive-\ : ${IMPALA_HIVE_VERSION}/src/metastore/if Please retain a way to build with HIVE_SRC_DIR_OVERRIDE, as some build systems rely on it to pass in a local Hive directory. It looks like it would have different meanings for Hive 2 vs Hive 3, but it might be ok to assume Hive 2 has the version we expected before and Hive 3 is pointing at the top level now. i.e. Hive 2: HIVE_METASTORE_THRIFT_DIR = HIVE_SRC_DIR/metastore/if Hive 3: HIVE_METASTORE_THRIFT_DIR = HIVE_SRC_DIR/standalone-metastore/src/main/thrift Alternatively, we could limit HIVE_SRC_DIR_OVERRIDE to Hive 2 or use separate variables for Hive 2 vs Hive 3, but both are going to be useful to build systems. We could do something like a HIVE_METASTORE_THRIFT_DIR_OVERRIDE (but we would still want to allow HIVE_SRC_DIR_OVERRIDE to allow an easy transition). http://gerrit.cloudera.org:8080/#/c/13005/14/tests/custom_cluster/test_permanent_udfs.py File tests/custom_cluster/test_permanent_udfs.py: http://gerrit.cloudera.org:8080/#/c/13005/14/tests/custom_cluster/test_permanent_udfs.py@502 PS14, Line 502: # Sample java udfs from hive-exec.jar. Function name to symbol class mapping : SAMPLE_JAVA_UDFS = [ : ('udfpi', 'org.apache.hadoop.hive.ql.udf.UDFPI'), : ('udfbin', 'org.apache.hadoop.hive.ql.udf.UDFBin'), : ('udfhex', 'org.apache.hadoop.hive.ql.udf.UDFHex'), : ('udfconv', 'org.apache.hadoop.hive.ql.udf.UDFConv'), : # TODO UDFHour was moved from UDF to GenericUDF in Hive 3 : # Add support for GenericUDFs to handle such cases : # ('udfhour', 'org.apache.hadoop.hive.ql.udf.UDFHour'), : ('udflike', 'org.apache.hadoop.hive.ql.udf.UDFLike'), : ('udfsign', 'org.apache.hadoop.hive.ql.udf.UDFSign'), : # TODO UDFYear moved to GenericUDF in Hive 3 : # Add support for GenericUDFs : # ('udfyear', 'org.apache.hadoop.hive.ql.udf.UDFYear'), : ('udfascii','org.apache.hadoop.hive.ql.udf.UDFAscii') : ] If these UDF tests work on Hive 2, then I'd like to keep them enabled. Either leave this unchanged (knowing that Hive 3 will need some modification later) or look at the environment variable and tailor this to Hive 2 vs Hive 3. -- To view, visit http://gerrit.cloudera.org:8080/13005 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I45a4dadbdfe30a02f722dbd917a49bc182fc6436 Gerrit-Change-Number: 13005 Gerrit-PatchSet: 14 Gerrit-Owner: Vihang Karajgaonkar <vih...@cloudera.com> Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Joe McDonnell <joemcdonn...@cloudera.com> Gerrit-Reviewer: Philip Zeyliger <phi...@cloudera.com> Gerrit-Reviewer: Sudhanshu Arora <sudhan...@cloudera.com> Gerrit-Reviewer: Todd Lipcon <t...@apache.org> Gerrit-Reviewer: Vihang Karajgaonkar <vih...@cloudera.com> Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com> Gerrit-Comment-Date: Mon, 29 Apr 2019 22:53:49 +0000 Gerrit-HasComments: Yes
