Imran Rashid created SPARK-10370:
------------------------------------
Summary: After a stages map outputs are registered, all running
attempts should be marked as zombies
Key: SPARK-10370
URL: https://issues.apache.org/jira/browse/SPARK-10370
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 1.5.0
Reporter: Imran Rashid
Assignee: Imran Rashid
Follow up to SPARK-5259. During stage retry, its possible for a stage to
"complete" by registering all its map output and starting the downstream
stages, before the latest task set has completed. This will result in the
earlier task set continuing to submit tasks, that are both unnecessary and
increase the chance of hitting SPARK-8029.
Spark should mark all tasks sets for a stage as zombie as soon as its map
output is registered. Note that this involves coordination between the various
scheduler components ({{DAGScheduler}} and {{TaskSetManager}} at least) which
isn't easily testable with the current setup.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
