[
https://issues.apache.org/jira/browse/YARN-1029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13848168#comment-13848168
]
Todd Lipcon commented on YARN-1029:
-----------------------------------
I agree with Karthik here -- the main reasons to pursue a separate ZKFC in HDFS
were:
- avoid failover in the case of GC (since ZKFC has a very low heap requirement)
but still failover fast in machine failure.
- avoid adding any dependency on ZK within the NN
- allow the option to use other resource managers
-- in practice no one has done this and I think the extra complexity all of our
pluggability introduces is not worth it
In the case of RM HA, as I understand it (apologies if I got anything wrong -
only tangentially followed this discussion):
- RM HA uses ZK itself for shared storage, so it already has a dependency on ZK.
- Given that the shared state is in ZK, we don't need fencing if the same ZK
client does election. The reason is that, if an RM loses its ZK lease, it will
simultaneously trigger the failover _and_ be unable to make further changes in
ZK. This exactly the semantics that we want.
Having a separate ZKFC actually complicates things, because we may have to
reintroduce some kind of fencing. What does it mean if the ZKFC loses its ZK
lease, but the RM itself continues to have access to ZK? It multiplies the
'state diagram' in two, and doesn't seem to offer any particular advantages.
As for embedding ZKFC (and refactoring it so it can (a) not do health checks,
(b) not control the RM via RPC, but directly, (c) re-use the same ZK session)
seems more complicated than it's worth. Given we'd be throwing away all of the
ZKFC features beyond the elector, why not just use the elector?
I'm also not sure why we want to preserve the "external ZKFC" option - per
above it's a more complicated deployment scenario and seems to offer little
tangible benefit.
> Allow embedding leader election into the RM
> -------------------------------------------
>
> Key: YARN-1029
> URL: https://issues.apache.org/jira/browse/YARN-1029
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Bikas Saha
> Assignee: Karthik Kambatla
> Attachments: embedded-zkfc-approach.patch, yarn-1029-0.patch,
> yarn-1029-0.patch, yarn-1029-approach.patch
>
>
> It should be possible to embed common ActiveStandyElector into the RM such
> that ZooKeeper based leader election and notification is in-built. In
> conjunction with a ZK state store, this configuration will be a simple
> deployment option.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
