Github user markgrover commented on the pull request:
https://github.com/apache/spark/pull/8093#issuecomment-130159464
Thanks for the review, @squito. I did consider that as well. The UI listens
to whatever is sent as `SparkListenerTaskEnd` message by the
DAGScheduler::handleTaskCompletion(). This is in turn called by a long set of
hoops from TaskSetManager::handleFailedTask(). The interesting thing is that
the DAGScheduler only sends one `SparkListenerTaskEnd` down the bus, based on
who won the race. And, I didn't really feel like this issue was worthy enough
for me to warrant in the DAGScheduler semantics of sending down multiple
`SparkListenerTaskEnd` events down the bus.
Your comments have been super helpful though. How about another possibility
- creating a new completely event that will only get triggered on the
RemoveExecutor message (not onDisconnected), we can post it on the bus and UI
can update itself if it gets it. I did something similar but later reverted it
((here)[https://github.com/markgrover/spark/commit/79bb8dd487adbb2a28c12e909eba56db743ae12d]'s
the revert) because I didn't think it was necessary. This thought is not well
formed, and I'll need to poke more to see if it's feasible at all without
changing the DAGScheduler too much or at all. Let me know what you think.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
