[
https://issues.apache.org/jira/browse/SPARK-1730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13990124#comment-13990124
]
Tathagata Das commented on SPARK-1730:
--------------------------------------
[~hshreedharan] This could also be something you can work on after 1729. The
way this can be done is to add a set of new methods to the Receiver name
storeReliably, which will return true or false based on whether the data was
successfully saved or not. And for now the system will decide true or false
after the receiver has reported the name of the block to the driver. This
ignores driver failures and assumes reporting to the driver is sufficient for
ensuring reliable receiving. Once we have a better idea of driver recovery, we
can figure out how to extend this to make it more reliable.
> Make receiver store data reliably to avoid data-loss on executor failures
> -------------------------------------------------------------------------
>
> Key: SPARK-1730
> URL: https://issues.apache.org/jira/browse/SPARK-1730
> Project: Spark
> Issue Type: Sub-task
> Components: Streaming
> Affects Versions: 1.0.0
> Reporter: Tathagata Das
>
--
This message was sent by Atlassian JIRA
(v6.2#6252)
