[
https://issues.apache.org/jira/browse/SPARK-21029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16779765#comment-16779765
]
Arun Mahadevan commented on SPARK-21029:
----------------------------------------
I encountered this issue and propose an alternative here
https://github.com/apache/spark/pull/23912
To keep it simple, this just stops the streaming queries when the spark session
is stopped. It does not handle the spark context being directly stopped, since
that requires more changes to add a callback hook that the spark context can
invoke and the session can leverage. Using the shutdown hook like proposed
previously does not look clean either.
> All StreamingQuery should be stopped when the SparkSession is stopped
> ---------------------------------------------------------------------
>
> Key: SPARK-21029
> URL: https://issues.apache.org/jira/browse/SPARK-21029
> Project: Spark
> Issue Type: Improvement
> Components: Structured Streaming
> Affects Versions: 2.2.0, 2.3.0
> Reporter: Felix Cheung
> Priority: Major
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
