[
https://issues.apache.org/jira/browse/SPARK-26421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun closed SPARK-26421.
---------------------------------
> Spark2.4.0 integration hadoop3.1.1 causes hive sql not to use，just in idea
> local mode
> -------------------------------------------------------------------------------------
>
> Key: SPARK-26421
> URL: https://issues.apache.org/jira/browse/SPARK-26421
> Project: Spark
> Issue Type: Bug
> Components: Deploy
> Affects Versions: 2.4.0
> Environment: ).idea maven project
> ).jdk 1.8.0_191
> ).hadoop 3.1.1
> ).spark 2.4.0
> Reporter: thinktothings
> Priority: Major
>
> ).Spark2.4.0 integration hadoop3.1.1 causes hive sql not to use，just in idea
> local mode
> ).idea maven project
> ).spark.sql connect hive
> val spark = SparkSession
> .builder()
> .master("local")
> .appName("Spark Hive Example")
> .config("spark.sql.warehouse.dir", warehouseLocation)
> .enableHiveSupport()
> .getOrCreate()
> spark.sql("show databases").show()
>
> ).do this error ,local environment not cluser
> ----------------------
> Exception in thread "main" java.lang.ExceptionInInitializerError
> at org.apache.hadoop.hive.conf.HiveConf.<clinit>(HiveConf.java:105)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at org.apache.spark.util.Utils$.classForName(Utils.scala:238)
> at
> org.apache.spark.sql.SparkSession$.hiveClassesArePresent(SparkSession.scala:1117)
> at
> org.apache.spark.sql.SparkSession$Builder.enableHiveSupport(SparkSession.scala:866)
> at
> com.opensource.bigdata.spark.sql.n_10_spark_hive.n_01_show_database.Run$.main(Run.scala:19)
> at
> com.opensource.bigdata.spark.sql.n_10_spark_hive.n_01_show_database.Run.main(Run.scala)
> Caused by: java.lang.IllegalArgumentException: Unrecognized Hadoop major
> version number: 3.1.1
> at
> org.apache.hadoop.hive.shims.ShimLoader.getMajorVersion(ShimLoader.java:174)
> at org.apache.hadoop.hive.shims.ShimLoader.loadShims(ShimLoader.java:139)
> at
> org.apache.hadoop.hive.shims.ShimLoader.getHadoopShims(ShimLoader.java:100)
> at org.apache.hadoop.hive.conf.HiveConf$ConfVars.<clinit>(HiveConf.java:368)
> ... 8 more
> Process finished with exit code 1
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
