[jira] [Commented] (SPARK-18970) FileSource failure during file list refresh doesn't cause an application to fail, but stops further processing

2017-01-17 Thread Genmao Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15825672#comment-15825672 ] Genmao Yu commented on SPARK-18970: --- [~lev.numerify] Any update about this issue with 2

[jira] [Commented] (SPARK-18970) FileSource failure during file list refresh doesn't cause an application to fail, but stops further processing

2016-12-22 Thread Lev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15771583#comment-15771583 ] Lev commented on SPARK-18970: - Actually this is exactly the behavior I want. My problem is th

[jira] [Commented] (SPARK-18970) FileSource failure during file list refresh doesn't cause an application to fail, but stops further processing

2016-12-22 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15771304#comment-15771304 ] Shixiong Zhu commented on SPARK-18970: -- I see. FileStreamSource ignores FileNotFound

[jira] [Commented] (SPARK-18970) FileSource failure during file list refresh doesn't cause an application to fail, but stops further processing

2016-12-22 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15771221#comment-15771221 ] Shixiong Zhu commented on SPARK-18970: -- Where did you find the exception? If it's in

[jira] [Commented] (SPARK-18970) FileSource failure during file list refresh doesn't cause an application to fail, but stops further processing

2016-12-22 Thread Lev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15771199#comment-15771199 ] Lev commented on SPARK-18970: - I am not sure whether task was retried or not, but Spark appli

[jira] [Commented] (SPARK-18970) FileSource failure during file list refresh doesn't cause an application to fail, but stops further processing

2016-12-22 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15771166#comment-15771166 ] Shixiong Zhu commented on SPARK-18970: -- Did the Spark task fail or not? Looks like t