Hi Andrew,

The behavior that I see now is that under the hood it tries to reconnect
endlessly. While this lasts, the thread that tries to fire a new task is
blocked at JobWaiter.awaitResult() and never gets released.
The full stacktrace for spark-1.0.2 is:

"jmsContainer-7" prio=10 tid=0x00007f18fc05b800 nid=0x6250 in Object.wait()
[0x00007f18bdb4b000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.$$YJP$$wait(Native Method)
at java.lang.Object.wait(Object.java)
at java.lang.Object.wait(Object.java:503)
at org.apache.spark.scheduler.JobWaiter.awaitResult(JobWaiter.scala:73) -
locked <0x00000007e735d3b0> (a org.apache.spark.scheduler.JobWaiter)
at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:452)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1051)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1069)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1083)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1097)
at org.apache.spark.rdd.RDD.foreachPartition(RDD.scala:724)
...

I observed the same behavior on both spark-1.0.2 and spark-1.1.0.



--
View this message in context: 
http://apache-spark-user-list.1001560.n3.nabble.com/Designed-behavior-when-master-is-unreachable-tp16677p16819.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to