[
https://issues.apache.org/jira/browse/HDFS-13183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17108132#comment-17108132
]
Xiaoqiao He commented on HDFS-13183:
------------------------------------
Thanks [~weichiu] for your comments.
{quote}does it work in federated cluster? IIRC you have a large federated
cluster so I am assuming the answer is yes, but does work out of box or does it
require extra configuration ? (Sorry, don't have much experience with HDFS
federation){quote}
In our practice, we deploy multi-balancers for each namespace in order to
monitor smoothly. And the current balancer solution also support federation
arch after check the logic IMO. Also this PR does not change this core logic.
{quote}failover. if a failover happens, the balancer can't adapt and will then
send the requests to ANN. That is fine as it shouldn't fail the balancer, but
it increases the new ANN overhead.{quote}
v006 try to create new {{NameNodeConnector}} for each iterator and keep to
request SBN even failover.
{quote}Also, just want to say that you don't actually need to UNCHECKED
FSNamesystem#getBlocks(). If dfs.ha.allow.stale.reads is true, Standby NN
accepts the request as well. That is an extra configuration so probably not
ideal.{quote}
Yes, it is true. v006 does not involve extra configuration just rely on
'dfs.ha.allow.stale.reads'.
Please give another review if have time. Thanks.
BTW, I am not sure why configuration key 'dfs.ha.allow.stale.reads' is not
defined at DFSConfigKeys, I would like file another JIRA to unify it.
> Standby NameNode process getBlocks request to reduce Active load
> ----------------------------------------------------------------
>
> Key: HDFS-13183
> URL: https://issues.apache.org/jira/browse/HDFS-13183
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: balancer & mover, namenode
> Reporter: Xiaoqiao He
> Assignee: Xiaoqiao He
> Priority: Major
> Attachments: HDFS-13183-trunk.001.patch, HDFS-13183-trunk.002.patch,
> HDFS-13183-trunk.003.patch, HDFS-13183.004.patch, HDFS-13183.005.patch,
> HDFS-13183.006.patch
>
>
> The performance of Active NameNode could be impact when {{Balancer}} requests
> #getBlocks, since query blocks of overly full DNs performance is extremely
> inefficient currently. The main reason is {{NameNodeRpcServer#getBlocks}}
> hold read lock for long time. In extreme case, all handlers of Active
> NameNode RPC server are occupied by one reader
> {{NameNodeRpcServer#getBlocks}} and other write operation calls, thus Active
> NameNode enter a state of false death for number of seconds even for minutes.
> The similar performance concerns of Balancer have reported by HDFS-9412,
> HDFS-7967, etc.
> If Standby NameNode can shoulder #getBlocks heavy burden, it could speed up
> the progress of balancing and reduce performance impact to Active NameNode.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
