GitHub user squito opened a pull request: https://github.com/apache/spark/pull/13826
[SPARK-16106][CORE] TaskSchedulerImpl should properly track executors added to existing hosts ## What changes were proposed in this pull request? TaskSchedulerImpl used to only set `newExecAvailable` when a new *host* was added, not when a new executor was added to an existing host. It also didn't update some internal state tracking live executors until a task was scheduled on the executor. This patch changes it to properly update as soon as it knows about a new executor. ## How was this patch tested? added a unit test, ran everything via jenkins. You can merge this pull request into a Git repository by running: $ git pull https://github.com/squito/spark SPARK-16106_executorByHosts Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/13826.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #13826 ---- commit 39d18eddee6f2f38663fa53770ab172cb68a6a5a Author: Imran Rashid <iras...@cloudera.com> Date: 2016-06-21T22:19:04Z refactor and fix tests commit 195408be6749b18fc28bd8fdb1512c6da760b90b Author: Imran Rashid <iras...@cloudera.com> Date: 2016-06-21T22:19:28Z make sure executorsByHost and localityLevels updated on new executors for same host commit 40b8ee7db3a8bd54110a4b834ec14e5b0b8f150d Author: Imran Rashid <iras...@cloudera.com> Date: 2016-06-21T22:35:03Z slightly expand test ---- --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org