Hi, I have spark 1.1.0 yarn installation. I am using spark-submit to run a
simple application.
>From the console output, I have 769 partitions and after task 768 in stage 0
>(count) finished,
it hangs. I used jstack to dump the stacktop and it shows it is waiting ...
Any suggestion what might go wrong and how to debug this kind of hanging?
Thanks.
Tian
"main" prio=10 tid=0x00007f6058009000 nid=0x7ecd in Object.wait()
[0x00007f605e4d9000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000fdd30500> (a
org.apache.spark.scheduler.JobWaiter)
at java.lang.Object.wait(Object.java:503)
at org.apache.spark.scheduler.JobWaiter.awaitResult(JobWaiter.scala:73)
- locked <0x00000000fdd30500> (a org.apache.spark.scheduler.JobWaiter)
at
org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:511)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1088)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1107)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1121)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1135)
at org.apache.spark.rdd.RDD.count(RDD.scala:904)
at
com.oncue.rna.realtime.streaming.spark.TopShowsToKafkaJob.getTopShows(TopShowsToKafkaJob.scala:29)
at
com.oncue.rna.realtime.streaming.spark.TopShowsToKafkaJob.getTopShows(TopShowsToKafkaJob.scala:45)
at
com.oncue.rna.realtime.streaming.spark.TopShowsToKafkaJob$$anonfun$5.apply(TopShowsToKafkaJob.scala:79)
at
com.oncue.rna.realtime.streaming.spark.TopShowsToKafkaJob$$anonfun$5.apply(TopShowsToKafkaJob.scala:76)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at scala.collection.immutable.Range.foreach(Range.scala:141)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
at scala.collection.AbstractTraversable.map(Traversable.scala:105)
at
com.oncue.rna.realtime.streaming.spark.TopShowsToKafkaJob.processRecentWindow(TopShowsToKafkaJob.scala:76)
at
com.oncue.rna.realtime.streaming.spark.TopShowsToKafkaJob$.processRecentWindow(TopShowsToKafkaJob.scala:98)
at
com.oncue.rna.realtime.streaming.spark.TopShowsToKafkaJob$.main(TopShowsToKafkaJob.scala:112)
at
com.oncue.rna.realtime.streaming.spark.TopShowsToKafkaJob.main(TopShowsToKafkaJob.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:328)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Locked ownable synchronizers:
- None
