[ https://issues.apache.org/jira/browse/SPARK-2998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen resolved SPARK-2998. ------------------------------ Resolution: Duplicate > scala.collection.mutable.HashSet cannot be cast to > scala.collection.mutable.BitSet > ---------------------------------------------------------------------------------- > > Key: SPARK-2998 > URL: https://issues.apache.org/jira/browse/SPARK-2998 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 1.0.2 > Reporter: pengyanhong > Priority: Blocker > > run a HiveQL via yarn-cluster, got error as below: > {quote} > 14/08/13 11:10:01 INFO > org.apache.spark.Logging$class.logInfo(Logging.scala:58): Serialized task > 8.0:2 as 20849 bytes in 0 ms > 14/08/13 11:10:01 INFO > org.apache.spark.Logging$class.logInfo(Logging.scala:58): Finished TID 812 in > 24 ms on A01-R06-I149-32.jd.local (progress: 2/200) > 14/08/13 11:10:01 INFO > org.apache.spark.Logging$class.logInfo(Logging.scala:58): Completed > ResultTask(8, 1) > 14/08/13 11:10:01 INFO > org.apache.spark.Logging$class.logInfo(Logging.scala:58): Failed to run > reduce at joins.scala:336 > 14/08/13 11:10:01 INFO > org.apache.spark.Logging$class.logInfo(Logging.scala:58): > finishApplicationMaster with FAILED > Exception in thread "Thread-2" java.lang.reflect.InvocationTargetException > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:199) > Caused by: org.apache.spark.SparkDriverExecutionException: Execution error > at > org.apache.spark.scheduler.DAGScheduler.handleTaskCompletion(DAGScheduler.scala:849) > at > org.apache.spark.scheduler.DAGSchedulerEventProcessActor$$anonfun$receive$2.applyOrElse(DAGScheduler.scala:1231) > at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498) > at akka.actor.ActorCell.invoke(ActorCell.scala:456) > at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237) > at akka.dispatch.Mailbox.run(Mailbox.scala:219) > at > akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386) > at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) > at > scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) > at > scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) > at > scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) > Caused by: java.lang.ClassCastException: scala.collection.mutable.HashSet > cannot be cast to scala.collection.mutable.BitSet > at > org.apache.spark.sql.execution.BroadcastNestedLoopJoin$$anonfun$7.apply(joins.scala:336) > at org.apache.spark.rdd.RDD$$anonfun$19.apply(RDD.scala:813) > at org.apache.spark.rdd.RDD$$anonfun$19.apply(RDD.scala:810) > at > org.apache.spark.scheduler.JobWaiter.taskSucceeded(JobWaiter.scala:56) > at > org.apache.spark.scheduler.DAGScheduler.handleTaskCompletion(DAGScheduler.scala:845) > ... 10 more > 14/08/13 11:10:01 INFO > org.apache.spark.Logging$class.logInfo(Logging.scala:58): Invoking sc stop > from shutdown hook > 14/08/13 11:10:01 INFO > org.apache.spark.Logging$class.logInfo(Logging.scala:58): AppMaster received > a signal. > 14/08/13 11:10:01 INFO > org.apache.spark.Logging$class.logInfo(Logging.scala:58): Starting task 8.0:3 > as TID 814 on executor 1: A01-R06-I149-32.jd.local (PROCESS_LOCAL) > 14/08/13 11:10:01 INFO > org.apache.spark.Logging$class.logInfo(Logging.scala:58): Serialized task > 8.0:3 as 20849 bytes in 0 ms > 14/08/13 11:10:01 INFO > org.apache.spark.Logging$class.logInfo(Logging.scala:58): Finished TID 813 in > 25 ms on A01-R06-I149-32.jd.local (progress: 3/200) > 14/08/13 11:10:01 INFO > org.apache.spark.Logging$class.logInfo(Logging.scala:58): Completed > ResultTask(8, 2) > ...... > {quote} > It runs successfully if removing the configuration about Kryo -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org