[ https://issues.apache.org/jira/browse/SPARK-19565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15930910#comment-15930910 ]
Kay Ousterhout commented on SPARK-19565: ---------------------------------------- [~jinxing6...@126.com] I closed this because it looks like a duplicate with the word you did for SPARK-19263. Feel free to re-open if I've misunderstood. > After fetching failed, success of old attempt of stage should be taken as > valid. > -------------------------------------------------------------------------------- > > Key: SPARK-19565 > URL: https://issues.apache.org/jira/browse/SPARK-19565 > Project: Spark > Issue Type: Test > Components: Scheduler > Affects Versions: 2.1.0 > Reporter: jin xing > > This is related to SPARK-19263. > When fetch failed, stage will be resubmitted. There can be running tasks from > both old and new stage attempts. Success of tasks from old stage attempt > should be taken as valid and partitionId should be removed from stage's > pendingPartitions accordingly. When pending partitions is empty, downstream > stage can be scheduled, even though there's still running tasks in the > active(new) stage attempt. -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org