liyunzhang_intel created HIVE-15432:
---------------------------------------
Summary: java.lang.ClassCastException is thrown when setting
"hive.input.format" as "org.apache.hadoop.hive.ql.io.CombineHiveInputFormat" in
hive on spark
Key: HIVE-15432
URL: https://issues.apache.org/jira/browse/HIVE-15432
Project: Hive
Issue Type: Bug
Reporter: liyunzhang_intel
set "hive.input.format" as
"org.apache.hadoop.hive.ql.io.CombineHiveInputFormat" in
itests/qtest/target/testconf/spark/standalone/hive-site.xml and run qtest like
following cmd:
{code}
mvn test -Dtest=TestSparkCliDriver -Dtest.output.overwrite=true -Dqfile=union.q
>log.TestSparkCliDriver 2>&1
{code}
found following exception in itests/qtest-spark/target/tmp/log/hive.log
{code}
2016-12-14T23:43:17,819 INFO [stderr-redir-1] client.SparkClientImpl:
java.lang.ClassCastException: Cannot cast
org.apache.hadoop.hive.ql.io.CombineHiveInputFormat$CombineHiveInputSplit to
org.apache.hadoop.mapred.InputSplitWithLocationInfo
2016-12-14T23:43:17,819 INFO [stderr-redir-1] client.SparkClientImpl: at
java.lang.Class.cast(Class.java:3094)
2016-12-14T23:43:17,819 INFO [stderr-redir-1] client.SparkClientImpl: at
org.apache.spark.rdd.HadoopRDD.getPreferredLocations(HadoopRDD.scala:318)
2016-12-14T23:43:17,819 INFO [stderr-redir-1] client.SparkClientImpl: at
org.apache.spark.rdd.RDD$$anonfun$preferredLocations$2.apply(RDD.scala:270)
2016-12-14T23:43:17,819 INFO [stderr-redir-1] client.SparkClientImpl: at
org.apache.spark.rdd.RDD$$anonfun$preferredLocations$2.apply(RDD.scala:270)
2016-12-14T23:43:17,819 INFO [stderr-redir-1] client.SparkClientImpl: at
scala.Option.getOrElse(Option.scala:121)
2016-12-14T23:43:17,819 INFO [stderr-redir-1] client.SparkClientImpl: at
org.apache.spark.rdd.RDD.preferredLocations(RDD.scala:269)
2016-12-14T23:43:17,819 INFO [stderr-redir-1] client.SparkClientImpl: at
org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$getPreferredLocsInternal(DAGScheduler.scala:1564)
2016-12-14T23:43:17,819 INFO [stderr-redir-1] client.SparkClientImpl: at
org.apache.spark.scheduler.DAGScheduler$$anonfun$org$apache$spark$scheduler$DAGScheduler$$getPreferredLocsInternal$2$$anonfun$apply$1.apply$mcVI$sp(DAGScheduler.scala:1575)
2016-12-14T23:43:17,819 INFO [stderr-redir-1] client.SparkClientImpl: at
org.apache.spark.scheduler.DAGScheduler$$anonfun$org$apache$spark$scheduler$DAGScheduler$$getPreferredLocsInternal$2$$anonfun$apply$1.apply(DAGScheduler.scala:1574)
2016-12-14T23:43:17,819 INFO [stderr-redir-1] client.SparkClientImpl: at
org.apache.spark.scheduler.DAGScheduler$$anonfun$org$apache$spark$scheduler$DAGScheduler$$getPreferredLocsInternal$2$$anonfun$apply$1.apply(DAGScheduler.scala:1574)
2016-12-14T23:43:17,819 INFO [stderr-redir-1] client.SparkClientImpl: at
scala.collection.immutable.List.foreach(List.scala:381)
2016-12-14T23:43:17,819 INFO [stderr-redir-1] client.SparkClientImpl: at
org.apache.spark.scheduler.DAGScheduler$$anonfun$org$apache$spark$scheduler$DAGScheduler$$getPreferredLocsInternal$2.apply(DAGScheduler.scala:1574)
2016-12-14T23:43:17,820 INFO [stderr-redir-1] client.SparkClientImpl: at
org.apache.spark.scheduler.DAGScheduler$$anonfun$org$apache$spark$scheduler$DAGScheduler$$getPreferredLocsInternal$2.apply(DAGScheduler.scala:1572)
2016-12-14T23:43:17,820 INFO [stderr-redir-1] client.SparkClientImpl: at
scala.collection.immutable.List.foreach(List.scala:381)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
