[
https://issues.apache.org/jira/browse/HDFS-7858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14341849#comment-14341849
]
Bikas Saha commented on HDFS-7858:
----------------------------------
What are long lived client examples? How many such clients would be there in a
large busy cluster? Will they be setting watches on ZK?
bq. Adding a cached entry to user's home dir to pick last active NN. If entry
is not present, the client picks the Standby from the configuration.
This seems like a reasonable improvement to the current scheme which will allow
a client to connect to the current active directly (even though it may be
listed later in the NN names list).
Please do keep in mind that ZK is just a notifier in the leader election
scheme. The real control lies in the FailoverController which is pluggable. A
different FailoverController may not use ZK. The status of the master flag may
not be valid/be-empty while the FailoverController is fencing the old master
and bringing up the new master.
Getting configuration from ZK is related but probably orthogonal. The entire
config for HDFS could be downloaded from ZK based on a well known HDFS service
name.
> Improve HA Namenode Failover detection on the client using Zookeeper
> --------------------------------------------------------------------
>
> Key: HDFS-7858
> URL: https://issues.apache.org/jira/browse/HDFS-7858
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Arun Suresh
> Assignee: Arun Suresh
>
> In an HA deployment, Clients are configured with the hostnames of both the
> Active and Standby Namenodes.Clients will first try one of the NNs
> (non-deterministically) and if its a standby NN, then it will respond to the
> client to retry the request on the other Namenode.
> If the client happens to talks to the Standby first, and the standby is
> undergoing some GC / is busy, then those clients might not get a response
> soon enough to try the other NN.
> Proposed Approach to solve this :
> 1) Since Zookeeper is already used as the failover controller, the clients
> could talk to ZK and find out which is the active namenode before contacting
> it.
> 2) Long-lived DFSClients would have a ZK watch configured which fires when
> there is a failover so they do not have to query ZK everytime to find out the
> active NN
> 2) Clients can also cache the last active NN in the user's home directory
> (~/.lastNN) so that short-lived clients can try that Namenode first before
> querying ZK
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
