GitHub user squito opened a pull request:

    https://github.com/apache/spark/pull/13826

    [SPARK-16106][CORE] TaskSchedulerImpl should properly track executors added 
to existing hosts

    ## What changes were proposed in this pull request?
    
    TaskSchedulerImpl used to only set `newExecAvailable` when a new *host* was 
added, not when a new executor was added to an existing host.  It also didn't 
update some internal state tracking live executors until a task was scheduled 
on the executor.  This patch changes it to properly update as soon as it knows 
about a new executor.
    
    ## How was this patch tested?
    
    added a unit test, ran everything via jenkins.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/squito/spark SPARK-16106_executorByHosts

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/13826.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #13826
    
----
commit 39d18eddee6f2f38663fa53770ab172cb68a6a5a
Author: Imran Rashid <iras...@cloudera.com>
Date:   2016-06-21T22:19:04Z

    refactor and fix tests

commit 195408be6749b18fc28bd8fdb1512c6da760b90b
Author: Imran Rashid <iras...@cloudera.com>
Date:   2016-06-21T22:19:28Z

    make sure executorsByHost and localityLevels updated on new executors for 
same host

commit 40b8ee7db3a8bd54110a4b834ec14e5b0b8f150d
Author: Imran Rashid <iras...@cloudera.com>
Date:   2016-06-21T22:35:03Z

    slightly expand test

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

Reply via email to