[jira] [Commented] (SPARK-24552) Task attempt numbers are reused when stages are retried

[ https://issues.apache.org/jira/browse/SPARK-24552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16520752#comment-16520752 ] Apache Spark commented on SPARK-24552: -- User 'vanzin' has created a pull request for this issue:

[jira] [Commented] (SPARK-24552) Task attempt numbers are reused when stages are retried

[ https://issues.apache.org/jira/browse/SPARK-24552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16520742#comment-16520742 ] Apache Spark commented on SPARK-24552: -- User 'vanzin' has created a pull request for this issue:

[jira] [Commented] (SPARK-24552) Task attempt numbers are reused when stages are retried

[ https://issues.apache.org/jira/browse/SPARK-24552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519723#comment-16519723 ] Apache Spark commented on SPARK-24552: -- User 'vanzin' has created a pull request for this issue:

[jira] [Commented] (SPARK-24552) Task attempt numbers are reused when stages are retried

[ https://issues.apache.org/jira/browse/SPARK-24552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519445#comment-16519445 ] Thomas Graves commented on SPARK-24552: --- more details on hadoop committer side: So I think the

[jira] [Commented] (SPARK-24552) Task attempt numbers are reused when stages are retried

[ https://issues.apache.org/jira/browse/SPARK-24552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519430#comment-16519430 ] Thomas Graves commented on SPARK-24552: --- this is actually a problem with hadoop committers, v1 and

[jira] [Commented] (SPARK-24552) Task attempt numbers are reused when stages are retried

[ https://issues.apache.org/jira/browse/SPARK-24552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16516522#comment-16516522 ] Marcelo Vanzin commented on SPARK-24552: I forked the output commiter issue into SPARK-24589 so

[jira] [Commented] (SPARK-24552) Task attempt numbers are reused when stages are retried

[ https://issues.apache.org/jira/browse/SPARK-24552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514526#comment-16514526 ] Apache Spark commented on SPARK-24552: -- User 'vanzin' has created a pull request for this issue:

[jira] [Commented] (SPARK-24552) Task attempt numbers are reused when stages are retried

[ https://issues.apache.org/jira/browse/SPARK-24552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16512519#comment-16512519 ] Thomas Graves commented on SPARK-24552: --- sorry just realized the v2 api is still marked experiment

[jira] [Commented] (SPARK-24552) Task attempt numbers are reused when stages are retried

[ https://issues.apache.org/jira/browse/SPARK-24552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16512504#comment-16512504 ] Thomas Graves commented on SPARK-24552: --- Note if this is a correctness bug and can cause data

[jira] [Commented] (SPARK-24552) Task attempt numbers are reused when stages are retried

[ https://issues.apache.org/jira/browse/SPARK-24552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16512500#comment-16512500 ] Thomas Graves commented on SPARK-24552: --- I agree, I don't think changing the attempt number at

[jira] [Commented] (SPARK-24552) Task attempt numbers are reused when stages are retried

[ https://issues.apache.org/jira/browse/SPARK-24552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16511695#comment-16511695 ] Jiang Xingbo commented on SPARK-24552: -- IIUC stageAttemptId + taskAttemptId shall probably define a

[jira] [Commented] (SPARK-24552) Task attempt numbers are reused when stages are retried

[ https://issues.apache.org/jira/browse/SPARK-24552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16511674#comment-16511674 ] Imran Rashid commented on SPARK-24552: -- I wouldn't call this a bug in the scheduler, though I agree

[jira] [Commented] (SPARK-24552) Task attempt numbers are reused when stages are retried

[ https://issues.apache.org/jira/browse/SPARK-24552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16511593#comment-16511593 ] Apache Spark commented on SPARK-24552: -- User 'rdblue' has created a pull request for this issue:

[jira] [Commented] (SPARK-24552) Task attempt numbers are reused when stages are retried

[ https://issues.apache.org/jira/browse/SPARK-24552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16511590#comment-16511590 ] Ryan Blue commented on SPARK-24552: --- cc [~vanzin], [~henryr], [~cloud_fan] > Task attempt numbers are