[jira] [Commented] (SPARK-19645) structured streaming job restart bug

2017-02-17 Thread Hongyao Zhao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871529#comment-15871529 ] Hongyao Zhao commented on SPARK-19645: -- It's seems that this is related to filesystem type. All the

[jira] [Commented] (SPARK-16746) Spark streaming lost data when ReceiverTracker writes Blockinfo to hdfs timeout

2016-07-28 Thread Hongyao Zhao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15398615#comment-15398615 ] Hongyao Zhao commented on SPARK-16746: -- I did some test yesterday, It seems that spark 1.6 direct

[jira] [Comment Edited] (SPARK-16746) Spark streaming lost data when ReceiverTracker writes Blockinfo to hdfs timeout

2016-07-27 Thread Hongyao Zhao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395354#comment-15395354 ] Hongyao Zhao edited comment on SPARK-16746 at 7/27/16 10:06 AM: I think

[jira] [Commented] (SPARK-16746) Spark streaming lost data when ReceiverTracker writes Blockinfo to hdfs timeout

2016-07-27 Thread Hongyao Zhao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395354#comment-15395354 ] Hongyao Zhao commented on SPARK-16746: -- I think if spark.streaming.receiver.writeAheadLog.enable is

[jira] [Updated] (SPARK-16746) Spark streaming lost data when ReceiverTracker writes Blockinfo to hdfs timeout

2016-07-26 Thread Hongyao Zhao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hongyao Zhao updated SPARK-16746: - Description: I wrote a spark streaming program which consume 1000 messages from one topic of

[jira] [Updated] (SPARK-16746) Spark streaming lost data when ReceiverTracker writes Blockinfo to hdfs timeout

2016-07-26 Thread Hongyao Zhao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hongyao Zhao updated SPARK-16746: - Description: I wrote a spark streaming program which consume 1000 messages from one topic of

[jira] [Updated] (SPARK-16746) Spark streaming lost data when ReceiverTracker writes Blockinfo to hdfs timeout

2016-07-26 Thread Hongyao Zhao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hongyao Zhao updated SPARK-16746: - Description: I wrote a spark streaming program which consume 1000 messages from one topic of

[jira] [Updated] (SPARK-16746) Spark streaming lost data when ReceiverTracker writes Blockinfo to hdfs timeout

2016-07-26 Thread Hongyao Zhao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hongyao Zhao updated SPARK-16746: - Description: I wrote a spark streaming program which consume 1000 messages from one topic of

[jira] [Created] (SPARK-16746) Spark streaming lost data when ReceiverTracker writes Blockinfo to hdfs timeout

2016-07-26 Thread Hongyao Zhao (JIRA)
Hongyao Zhao created SPARK-16746: Summary: Spark streaming lost data when ReceiverTracker writes Blockinfo to hdfs timeout Key: SPARK-16746 URL: https://issues.apache.org/jira/browse/SPARK-16746