[
https://issues.apache.org/jira/browse/HIVE-7387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14064204#comment-14064204
]
Xuefu Zhang commented on HIVE-7387:
-----------------------------------
Thanks, Sean. Linked.
> Guava version conflict between hadoop and spark [Spark-Branch]
> --------------------------------------------------------------
>
> Key: HIVE-7387
> URL: https://issues.apache.org/jira/browse/HIVE-7387
> Project: Hive
> Issue Type: Bug
> Components: Spark
> Reporter: Chengxiang Li
> Assignee: Chengxiang Li
>
> hadoop-hdfs and hadoop-comman have dependency on guava-11.0.2.jar, and spark
> dependent on guava-14.0.1.jar. guava-11.0.2 has API conflict with
> guava-14.0.1, as Hive CLI load both dependency into classpath currently,
> query failed on either spark engine or mr engine.
> {code}
> java.lang.NoSuchMethodError:
> com.google.common.hash.HashFunction.hashInt(I)Lcom/google/common/hash/HashCode;
> at
> org.apache.spark.util.collection.OpenHashSet.org$apache$spark$util$collection$OpenHashSet$$hashcode(OpenHashSet.scala:261)
> at
> org.apache.spark.util.collection.OpenHashSet$mcI$sp.getPos$mcI$sp(OpenHashSet.scala:165)
> at
> org.apache.spark.util.collection.OpenHashSet$mcI$sp.contains$mcI$sp(OpenHashSet.scala:102)
> at
> org.apache.spark.util.SizeEstimator$$anonfun$visitArray$2.apply$mcVI$sp(SizeEstimator.scala:214)
> at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:141)
> at
> org.apache.spark.util.SizeEstimator$.visitArray(SizeEstimator.scala:210)
> at
> org.apache.spark.util.SizeEstimator$.visitSingleObject(SizeEstimator.scala:169)
> at
> org.apache.spark.util.SizeEstimator$.org$apache$spark$util$SizeEstimator$$estimate(SizeEstimator.scala:161)
> at
> org.apache.spark.util.SizeEstimator$.estimate(SizeEstimator.scala:155)
> at org.apache.spark.storage.MemoryStore.putValues(MemoryStore.scala:75)
> at org.apache.spark.storage.MemoryStore.putValues(MemoryStore.scala:92)
> at org.apache.spark.storage.BlockManager.doPut(BlockManager.scala:661)
> at org.apache.spark.storage.BlockManager.put(BlockManager.scala:546)
> at
> org.apache.spark.storage.BlockManager.putSingle(BlockManager.scala:812)
> at
> org.apache.spark.broadcast.HttpBroadcast.<init>(HttpBroadcast.scala:52)
> at
> org.apache.spark.broadcast.HttpBroadcastFactory.newBroadcast(HttpBroadcastFactory.scala:35)
> at
> org.apache.spark.broadcast.HttpBroadcastFactory.newBroadcast(HttpBroadcastFactory.scala:29)
> at
> org.apache.spark.broadcast.BroadcastManager.newBroadcast(BroadcastManager.scala:62)
> at org.apache.spark.SparkContext.broadcast(SparkContext.scala:776)
> at org.apache.spark.rdd.HadoopRDD.<init>(HadoopRDD.scala:112)
> at org.apache.spark.SparkContext.hadoopRDD(SparkContext.scala:527)
> at
> org.apache.spark.api.java.JavaSparkContext.hadoopRDD(JavaSparkContext.scala:307)
> at
> org.apache.hadoop.hive.ql.exec.spark.SparkClient.createRDD(SparkClient.java:204)
> at
> org.apache.hadoop.hive.ql.exec.spark.SparkClient.execute(SparkClient.java:167)
> at
> org.apache.hadoop.hive.ql.exec.spark.SparkTask.execute(SparkTask.java:32)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:159)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)
> at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:72)
> {code}
> NO PRECOMMIT TESTS. This is for spark branch only.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
