[ https://issues.apache.org/jira/browse/SPARK-3129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14140049#comment-14140049 ]
Hari Shreedharan commented on SPARK-3129: ----------------------------------------- Do these numbers look ok enough to you guys, [~tdas], [~matei], [~pwendell]? If you want to experiment, it is you can use the app and play around with it. I don't think this number is too bad - though I don't know how the current block replication code performs - but I'd estimate it to be in a few 10's of millis. > Prevent data loss in Spark Streaming > ------------------------------------ > > Key: SPARK-3129 > URL: https://issues.apache.org/jira/browse/SPARK-3129 > Project: Spark > Issue Type: New Feature > Components: Streaming > Reporter: Hari Shreedharan > Assignee: Hari Shreedharan > Attachments: SecurityFix.diff, StreamingPreventDataLoss.pdf > > > Spark Streaming can small amounts of data when the driver goes down - and the > sending system cannot re-send the data (or the data has already expired on > the sender side). The document attached has more details. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org