liyunzhang_intel created HIVE-15432: ---------------------------------------
Summary: java.lang.ClassCastException is thrown when setting "hive.input.format" as "org.apache.hadoop.hive.ql.io.CombineHiveInputFormat" in hive on spark Key: HIVE-15432 URL: https://issues.apache.org/jira/browse/HIVE-15432 Project: Hive Issue Type: Bug Reporter: liyunzhang_intel set "hive.input.format" as "org.apache.hadoop.hive.ql.io.CombineHiveInputFormat" in itests/qtest/target/testconf/spark/standalone/hive-site.xml and run qtest like following cmd: {code} mvn test -Dtest=TestSparkCliDriver -Dtest.output.overwrite=true -Dqfile=union.q >log.TestSparkCliDriver 2>&1 {code} found following exception in itests/qtest-spark/target/tmp/log/hive.log {code} 2016-12-14T23:43:17,819 INFO [stderr-redir-1] client.SparkClientImpl: java.lang.ClassCastException: Cannot cast org.apache.hadoop.hive.ql.io.CombineHiveInputFormat$CombineHiveInputSplit to org.apache.hadoop.mapred.InputSplitWithLocationInfo 2016-12-14T23:43:17,819 INFO [stderr-redir-1] client.SparkClientImpl: at java.lang.Class.cast(Class.java:3094) 2016-12-14T23:43:17,819 INFO [stderr-redir-1] client.SparkClientImpl: at org.apache.spark.rdd.HadoopRDD.getPreferredLocations(HadoopRDD.scala:318) 2016-12-14T23:43:17,819 INFO [stderr-redir-1] client.SparkClientImpl: at org.apache.spark.rdd.RDD$$anonfun$preferredLocations$2.apply(RDD.scala:270) 2016-12-14T23:43:17,819 INFO [stderr-redir-1] client.SparkClientImpl: at org.apache.spark.rdd.RDD$$anonfun$preferredLocations$2.apply(RDD.scala:270) 2016-12-14T23:43:17,819 INFO [stderr-redir-1] client.SparkClientImpl: at scala.Option.getOrElse(Option.scala:121) 2016-12-14T23:43:17,819 INFO [stderr-redir-1] client.SparkClientImpl: at org.apache.spark.rdd.RDD.preferredLocations(RDD.scala:269) 2016-12-14T23:43:17,819 INFO [stderr-redir-1] client.SparkClientImpl: at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$getPreferredLocsInternal(DAGScheduler.scala:1564) 2016-12-14T23:43:17,819 INFO [stderr-redir-1] client.SparkClientImpl: at org.apache.spark.scheduler.DAGScheduler$$anonfun$org$apache$spark$scheduler$DAGScheduler$$getPreferredLocsInternal$2$$anonfun$apply$1.apply$mcVI$sp(DAGScheduler.scala:1575) 2016-12-14T23:43:17,819 INFO [stderr-redir-1] client.SparkClientImpl: at org.apache.spark.scheduler.DAGScheduler$$anonfun$org$apache$spark$scheduler$DAGScheduler$$getPreferredLocsInternal$2$$anonfun$apply$1.apply(DAGScheduler.scala:1574) 2016-12-14T23:43:17,819 INFO [stderr-redir-1] client.SparkClientImpl: at org.apache.spark.scheduler.DAGScheduler$$anonfun$org$apache$spark$scheduler$DAGScheduler$$getPreferredLocsInternal$2$$anonfun$apply$1.apply(DAGScheduler.scala:1574) 2016-12-14T23:43:17,819 INFO [stderr-redir-1] client.SparkClientImpl: at scala.collection.immutable.List.foreach(List.scala:381) 2016-12-14T23:43:17,819 INFO [stderr-redir-1] client.SparkClientImpl: at org.apache.spark.scheduler.DAGScheduler$$anonfun$org$apache$spark$scheduler$DAGScheduler$$getPreferredLocsInternal$2.apply(DAGScheduler.scala:1574) 2016-12-14T23:43:17,820 INFO [stderr-redir-1] client.SparkClientImpl: at org.apache.spark.scheduler.DAGScheduler$$anonfun$org$apache$spark$scheduler$DAGScheduler$$getPreferredLocsInternal$2.apply(DAGScheduler.scala:1572) 2016-12-14T23:43:17,820 INFO [stderr-redir-1] client.SparkClientImpl: at scala.collection.immutable.List.foreach(List.scala:381) {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)