[
https://issues.apache.org/jira/browse/HDFS-13183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16503271#comment-16503271
]
He Xiaoqiao commented on HDFS-13183:
------------------------------------
Thanks [~elgoiri] and [~shv] for your comments.
{quote}
If we have that mechanism, we could make this generic and the
ActiveDenyOfServiceException would only need to implement this failover
exception.
{quote}
ActiveDenyOfServiceException can trigger client failover now actually. but even
that it also has some problems, for instance, Balancer would not work well if
SBN shutdown. Of course, these cases can be resolved, if someone interested,
please go on work, I am sorry that I have no time to fix it recently.
{quote}
I see that you are coming to the same problems as we do, but in a more general
case. getBlocks() was actually one of our initial use cases for the feature.
{quote}
HDFS-12943 is a very compelling feature indeed. However, there are many users
who used branch-2.7 or earlier are eager for resolving ANN overhead of
#getBlocks from Balancer as far as I know. This patch just puts forward one
solution to this problem for reference.
> Standby NameNode process getBlocks request to reduce Active load
> ----------------------------------------------------------------
>
> Key: HDFS-13183
> URL: https://issues.apache.org/jira/browse/HDFS-13183
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: balancer & mover, namenode
> Affects Versions: 2.7.5, 3.1.0, 2.9.1, 2.8.4, 3.0.2
> Reporter: He Xiaoqiao
> Assignee: He Xiaoqiao
> Priority: Major
> Attachments: HDFS-13183-trunk.001.patch, HDFS-13183-trunk.002.patch,
> HDFS-13183-trunk.003.patch
>
>
> The performance of Active NameNode could be impact when {{Balancer}} requests
> #getBlocks, since query blocks of overly full DNs performance is extremely
> inefficient currently. The main reason is {{NameNodeRpcServer#getBlocks}}
> hold read lock for long time. In extreme case, all handlers of Active
> NameNode RPC server are occupied by one reader
> {{NameNodeRpcServer#getBlocks}} and other write operation calls, thus Active
> NameNode enter a state of false death for number of seconds even for minutes.
> The similar performance concerns of Balancer have reported by HDFS-9412,
> HDFS-7967, etc.
> If Standby NameNode can shoulder #getBlocks heavy burden, it could speed up
> the progress of balancing and reduce performance impact to Active NameNode.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
