[ https://issues.apache.org/jira/browse/SPARK-1331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tathagata Das updated SPARK-1331: --------------------------------- Priority: Blocker (was: Critical) > Graceful shutdown of Spark Streaming computation > ------------------------------------------------ > > Key: SPARK-1331 > URL: https://issues.apache.org/jira/browse/SPARK-1331 > Project: Spark > Issue Type: Improvement > Components: Streaming > Affects Versions: 0.9.0 > Reporter: Tathagata Das > Assignee: Tathagata Das > Priority: Blocker > > Current version of StreamingContext.stop() directly kills all the data > receivers (NetworkReceiver) without waiting for the data already received to > be persisted and processed. Fixing this requires the following. > 1. Each receiver, when it gets a stop signal from the driver, should stop > receiving, and then wait for all the received data to have been persisted and > reported to the driver. > 2. The driver, after stopping all the receivers, should wait for all the > received to be processed. -- This message was sent by Atlassian JIRA (v6.2#6252)