Re: Spark 2.0.2 : Hang at "org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:623)"

2017-03-24 Thread Ravindra
Also noticed that there are 8 - "dispatcher-event-loop-0 7" and 8 - "map-output-dispatcher-0 7" all waiting at the same location in the code that is - *sun.misc.Unsafe.park(Native Method)* *java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)*

Spark 2.0.2 : Hang at "org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:623)"

2017-03-24 Thread Ravindra
Hi All, My Spark job hangs here... Looking into the thread dump I noticed that it hangs here (stack trace given below) on the count action on dataframe (given below). Data is very small. Its actually not more than even 10 rows. I noticed some JIRAs about this issue but all are resolved-closed in