Github user tgravescs commented on the pull request:
https://github.com/apache/spark/pull/3771#issuecomment-68367226
@SaintBacchus so I'm still a bit unclear of the exact scenario. I just want
to make sure we are handling everything properly so want to make sure I
understand fully.
So this is when the RM goes down and is being brought back up or fails over
to a standby. At that point it restarts the applications to start a new
attempt. The shutdown hook is run and the code you mention above runs and
unregisters. I understand client mode can't set it because spark context is not
in the same process. The thing that is unclear to me is how is cluster mode
setting the finalStatus to something other then succeeded? Is sparkContext
being signalled and then throwing exception so that startUserClass catches it
and marks it as failed?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
