[ https://issues.apache.org/jira/browse/SPARK-20935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16038549#comment-16038549 ]
Sean Owen commented on SPARK-20935: ----------------------------------- Sounds OK to me [~hyukjin.kwon] if you're up for it > A daemon thread, "BatchedWriteAheadLog Writer", left behind after terminating > StreamingContext. > ----------------------------------------------------------------------------------------------- > > Key: SPARK-20935 > URL: https://issues.apache.org/jira/browse/SPARK-20935 > Project: Spark > Issue Type: Bug > Components: DStreams > Affects Versions: 1.6.3, 2.1.1 > Reporter: Terence Yim > > With batched write ahead log on by default in driver (SPARK-11731), if there > is no receiver based {{InputDStream}}, the "BatchedWriteAheadLog Writer" > thread created by {{BatchedWriteAheadLog}} never get shutdown. > The root cause is due to > https://github.com/apache/spark/blob/master/streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala#L168 > that it never call {{ReceivedBlockTracker.stop()}} (which in turn call > {{BatchedWriteAheadLog.close()}}) if there is no receiver based input. -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org