[jira] [Commented] (SPARK-19645) structured streaming job restart bug

2017-02-17 Thread guifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872921#comment-15872921 ] guifeng commented on SPARK-19645: - [~zsxwing] ok, I think the solution is that determining whether the

[jira] [Commented] (SPARK-19645) structured streaming job restart bug

2017-02-17 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872319#comment-15872319 ] Shixiong Zhu commented on SPARK-19645: -- [~guifengl...@gmail.com] Thanks for reporting. Could you

[jira] [Commented] (SPARK-19645) structured streaming job restart bug

2017-02-17 Thread guifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871716#comment-15871716 ] guifeng commented on SPARK-19645: - So, I think current workaround is to delete(if delta file exist) and

[jira] [Commented] (SPARK-19645) structured streaming job restart bug

2017-02-17 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871680#comment-15871680 ] Sean Owen commented on SPARK-19645: --- I'm not suggesting it works with Hadoop 2.6; I'm responding to the

[jira] [Commented] (SPARK-19645) structured streaming job restart bug

2017-02-17 Thread guifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871627#comment-15871627 ] guifeng commented on SPARK-19645: - aBut, spark mater don't set rename options that support overwrite, so

[jira] [Commented] (SPARK-19645) structured streaming job restart bug

2017-02-17 Thread guifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871552#comment-15871552 ] guifeng commented on SPARK-19645: - But, spark mater don't set rename options that support overwrite. >

[jira] [Commented] (SPARK-19645) structured streaming job restart bug

2017-02-17 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871542#comment-15871542 ] Sean Owen commented on SPARK-19645: --- Note that master requires Hadoop 2.6+ now. > structured streaming

[jira] [Commented] (SPARK-19645) structured streaming job restart bug

2017-02-17 Thread guifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871536#comment-15871536 ] guifeng commented on SPARK-19645: - spark's default hadoop version is hadoop 2.2 that rename method don't

[jira] [Commented] (SPARK-19645) structured streaming job restart bug

2017-02-17 Thread Hongyao Zhao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871529#comment-15871529 ] Hongyao Zhao commented on SPARK-19645: -- It's seems that this is related to filesystem type. All the