This is an automated email from the ASF dual-hosted git repository. srowen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push: new aed6a47580e [SPARK-43320][SQL][HIVE] Directly call Hive 2.3.9 API aed6a47580e is described below commit aed6a47580e66f92b0641d5bc08ad833be4724f4 Author: Cheng Pan <cheng...@apache.org> AuthorDate: Sat Apr 29 09:38:14 2023 -0500 [SPARK-43320][SQL][HIVE] Directly call Hive 2.3.9 API ### What changes were proposed in this pull request? Call Hive 2.3.9 API directly instead of reflection, basically reverts SPARK-37446. ### Why are the changes needed? Switch to direct calling to achieve compile time check. Spark does not officially support building against Hive other than 2.3.9, for cases listed in SPARK-37446, it's the vendor's responsibility to port HIVE-21563 into their maintained Hive 2.3.8-[vender-custom-version]. See full discussion in https://github.com/apache/spark/pull/40893. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Pass GA. Closes #40995 from pan3793/SPARK-43320. Authored-by: Cheng Pan <cheng...@apache.org> Signed-off-by: Sean Owen <sro...@gmail.com> --- .../main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala b/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala index becca8eae5e..5b0309813fc 100644 --- a/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala +++ b/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala @@ -1347,12 +1347,11 @@ private[hive] object HiveClientImpl extends Logging { new HiveConf(conf, classOf[HiveConf]) } try { - classOf[Hive].getMethod("getWithoutRegisterFns", classOf[HiveConf]) - .invoke(null, hiveConf).asInstanceOf[Hive] + Hive.getWithoutRegisterFns(hiveConf) } catch { // SPARK-37069: not all Hive versions have the above method (e.g., Hive 2.3.9 has it but // 2.3.8 don't), therefore here we fallback when encountering the exception. - case _: NoSuchMethodException => + case _: NoSuchMethodError => Hive.get(hiveConf) } } --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org