Eric Vandenberg created SPARK-21219: ---------------------------------------
Summary: Task retry occurs on same executor due to race condition with blacklisting Key: SPARK-21219 URL: https://issues.apache.org/jira/browse/SPARK-21219 Project: Spark Issue Type: Bug Components: Scheduler Affects Versions: 2.1.1 Reporter: Eric Vandenberg Priority: Minor When a task fails it is added into the pending task list and corresponding black list policy is enforced (ie, specifying if it can/can't run on a particular node/executor/etc.) Unfortunately the ordering is such that retrying the task could assign the task to the same executor, which, incidentally could be shutting down and immediately fail the retry. Instead the black list state should be updated and then the task assigned, ensuring that the black list policy is properly enforced. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org