[ https://issues.apache.org/jira/browse/SPARK-16709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413065#comment-15413065 ]
Apache Spark commented on SPARK-16709: -------------------------------------- User 'shenh062326' has created a pull request for this issue: https://github.com/apache/spark/pull/14557 > Task with commit failed will retry infinite when speculation set to true > ------------------------------------------------------------------------ > > Key: SPARK-16709 > URL: https://issues.apache.org/jira/browse/SPARK-16709 > Project: Spark > Issue Type: Bug > Affects Versions: 1.6.0 > Reporter: Hong Shen > Attachments: commit failed.png > > > In our cluster, we set spark.speculation=true, but when a task throw > exception at SparkHadoopMapRedUtil.performCommit(), this task can retry > infinite. > https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/mapred/SparkHadoopMapRedUtil.scala#L83 -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org