Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/8093#issuecomment-130016534
The race part is pretty tricky. The only solution I can think of is to
delay the posting of the `SparkListenerExecutorRemoved` event until an explicit
`RemoveExecutor` message arrives from the AM. That means all state about the
executor is cleaned up in the scheduler, but the UI / event log will be lagging
behind a little bit.
Thinking out loud, this could be done by putting the code that posts the
event in a new protected method, and overriding that method in
`YarnSchedulerBackend` so that it only posts the event when the message from
the AM is received.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
