GitHub user squito opened a pull request:
https://github.com/apache/spark/pull/21131
[SPARK-23433][CORE] Late zombie task completions update all tasksets
Fetch failure lead to multiple tasksets which are active for a given
stage. A late completion from an earlier attempt of the stage
should update the most recent attempt for the stage, so it does
not try to submit another task for the same partition, and so that
it knows when it is completed and when it should be marked as
a "zombie".
Added a regression test.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/squito/spark SPARK-23433
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21131.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21131
----
commit 0720a7cd6826614e516c3d3a51bd4519259cbe3b
Author: Imran Rashid <irashid@...>
Date: 2018-02-21T20:21:14Z
[SPARK-23433][CORE] Late zombie task completions update all tasksets
After a fetch failure and stage retry, we may have multiple tasksets
which are active for a given stage. A late completion from an earlier
attempt of the stage should update the most recent attempt for the
stage, so it does not try to submit another task for the same partition,
and so that it knows when it is completed.
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
