[
https://issues.apache.org/jira/browse/SPARK-21197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nan Zhu updated SPARK-21197:
----------------------------
Summary: Tricky use case makes dead application struggle for a long
duration (was: Tricky use cases makes dead application struggle for a long
duration)
> Tricky use case makes dead application struggle for a long duration
> -------------------------------------------------------------------
>
> Key: SPARK-21197
> URL: https://issues.apache.org/jira/browse/SPARK-21197
> Project: Spark
> Issue Type: Bug
> Components: DStreams, Spark Core
> Affects Versions: 2.0.2, 2.1.1
> Reporter: Nan Zhu
>
> The use case is in Spark Streaming while the root cause is in DAGScheduler,
> so I said the component as both of DStreams and Core
> Use case:
> the user has a thread periodically triggering Spark jobs, and in the same
> application, they retrieve data through Spark Streaming from somewhere....in
> the Streaming logic, an exception is thrown so that the whole application is
> supposed to be shutdown and let YARN restart it...
> The user observed that after the exception is propagated to Spark core and
> SparkContext.stop() is called, after 18 hours, the application is still
> running...
> The root cause is that when we call DAGScheduler.stop(), we will wait for
> eventLoop's thread to finish
> (https://github.com/apache/spark/blob/03eb6117affcca21798be25706a39e0d5a2f7288/core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala#L1704
> and
> https://github.com/apache/spark/blob/03eb6117affcca21798be25706a39e0d5a2f7288/core/src/main/scala/org/apache/spark/util/EventLoop.scala#L40)
> Since there is a thread periodically push events to DAGScheduler's event
> queue, it will never finish
> a potential solution is that in EventLoop, we should allow interrupt the
> thread directly for some cases, e.g. this one, and simultaneously allow
> graceful shutdown for other cases, e.g. ListenerBus one,
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
