Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/21356#discussion_r189702478
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/AsyncEventQueue.scala ---
@@ -130,7 +129,11 @@ private class AsyncEventQueue(val name: String, conf:
SparkConf, metrics: LiveLi
eventCount.incrementAndGet()
eventQueue.put(POISON_PILL)
}
- dispatchThread.join()
+ // this thread might be trying to stop itself as part of error
handling -- we can't join
+ // in that case.
+ if (Thread.currentThread() != dispatchThread) {
--- End diff --
It does still happen, we need this. We see the interrupt in postToAll,
which is in the queue thread. If it fails, we call `removeListenerOnError`.
If that results in the queue being empty, we stop the queue.
```
"spark-listener-group-eventLog" #20 daemon prio=5 os_prio=31
tid=0x00007f831379e800 nid=0x6303 in Object.wait() [0x0000000129226000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x000000078047ae28> (a
org.apache.spark.scheduler.AsyncEventQueue$$anon$1)
at java.lang.Thread.join(Thread.java:1245)
- locked <0x000000078047ae28> (a
org.apache.spark.scheduler.AsyncEventQueue$$anon$1)
at java.lang.Thread.join(Thread.java:1319)
at
org.apache.spark.scheduler.AsyncEventQueue.stop(AsyncEventQueue.scala:135)
at
org.apache.spark.scheduler.LiveListenerBus$$anonfun$removeListener$2.apply(LiveListenerBus.scala:123)
at
org.apache.spark.scheduler.LiveListenerBus$$anonfun$removeListener$2.apply(LiveListenerBus.scala:121)
at
scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at
scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at
org.apache.spark.scheduler.LiveListenerBus.removeListener(LiveListenerBus.scala:121)
- locked <0x0000000780475fe8> (a
org.apache.spark.scheduler.LiveListenerBus)
at
org.apache.spark.scheduler.AsyncEventQueue.removeListenerOnError(AsyncEventQueue.scala:196)
at
org.apache.spark.scheduler.AsyncEventQueue.removeListenerOnError(AsyncEventQueue.scala:37)
at
org.apache.spark.util.ListenerBus$class.postToAll(ListenerBus.scala:101)
```
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
