Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/8093#issuecomment-130103072
> there are cases where the RemoveExecutor event may never come, right
That can happen, yes. In a normal RPC system, that message not arriving
would probably mean that the AM would also end up disconnecting from the
driver, and you could use that disconnect event to drain the queue of removed
executors. But we're talking akka here, and it may have different semantics, so
it probably means adding some sort of timeout (ugh)...
> How would you feel about digging into the race condition in a follow-up
JIRA
I guess we could do that. My concern is that the race is probably always
going to be won by the executor disconnect message (instead of the explicit
`RemoveExecutor` message), which means that most of the time these messages
will still not show up in the driver UI...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
