[jira] [Updated] (SPARK-11163) Remove unnecessary addPendingTask calls in TaskSetManager.executorLost
[ https://issues.apache.org/jira/browse/SPARK-11163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-11163: -- Target Version/s: 1.5.2, 1.6.0 (was: 1.6.0) Fix Version/s: (was: 1.5.2) (was: 1.5.1) > Remove unnecessary addPendingTask calls in TaskSetManager.executorLost > -- > > Key: SPARK-11163 > URL: https://issues.apache.org/jira/browse/SPARK-11163 > Project: Spark > Issue Type: Improvement > Components: Scheduler >Reporter: Kay Ousterhout >Assignee: Kay Ousterhout >Priority: Minor > > The proposed commit removes unnecessary calls to addPendingTask in > TaskSetManager.executorLost. These calls are unnecessary: for > tasks that are still pending and haven't been launched, they're > still in all of the correct pending lists, so calling addPendingTask > has no effect. For tasks that are currently running (which may still be > in the pending lists, depending on how they were scheduled), we call > addPendingTask in handleFailedTask, so the calls at the beginning > of executorLost are redundant. > I think these calls are left over from when we re-computed the locality > levels in addPendingTask; now that we call recomputeLocality separately, > I don't think these are necessary. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-11163) Remove unnecessary addPendingTask calls in TaskSetManager.executorLost
[ https://issues.apache.org/jira/browse/SPARK-11163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout updated SPARK-11163: --- Summary: Remove unnecessary addPendingTask calls in TaskSetManager.executorLost (was: Remove unnecessary addPendingTask calls in TaskSetManager) > Remove unnecessary addPendingTask calls in TaskSetManager.executorLost > -- > > Key: SPARK-11163 > URL: https://issues.apache.org/jira/browse/SPARK-11163 > Project: Spark > Issue Type: Improvement > Components: Scheduler >Reporter: Kay Ousterhout >Assignee: Kay Ousterhout >Priority: Minor > Fix For: 1.5.1, 1.5.2 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-11163) Remove unnecessary addPendingTask calls in TaskSetManager.executorLost
[ https://issues.apache.org/jira/browse/SPARK-11163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout updated SPARK-11163: --- Description: The proposed commit removes unnecessary calls to addPendingTask in TaskSetManager.executorLost. These calls are unnecessary: for tasks that are still pending and haven't been launched, they're still in all of the correct pending lists, so calling addPendingTask has no effect. For tasks that are currently running (which may still be in the pending lists, depending on how they were scheduled), we call addPendingTask in handleFailedTask, so the calls at the beginning of executorLost are redundant. I think these calls are left over from when we re-computed the locality levels in addPendingTask; now that we call recomputeLocality separately, I don't think these are necessary. > Remove unnecessary addPendingTask calls in TaskSetManager.executorLost > -- > > Key: SPARK-11163 > URL: https://issues.apache.org/jira/browse/SPARK-11163 > Project: Spark > Issue Type: Improvement > Components: Scheduler >Reporter: Kay Ousterhout >Assignee: Kay Ousterhout >Priority: Minor > Fix For: 1.5.1, 1.5.2 > > > The proposed commit removes unnecessary calls to addPendingTask in > TaskSetManager.executorLost. These calls are unnecessary: for > tasks that are still pending and haven't been launched, they're > still in all of the correct pending lists, so calling addPendingTask > has no effect. For tasks that are currently running (which may still be > in the pending lists, depending on how they were scheduled), we call > addPendingTask in handleFailedTask, so the calls at the beginning > of executorLost are redundant. > I think these calls are left over from when we re-computed the locality > levels in addPendingTask; now that we call recomputeLocality separately, > I don't think these are necessary. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
