[
https://issues.apache.org/jira/browse/HDFS-16368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846512#comment-17846512
]
ASF GitHub Bot commented on HDFS-16368:
---------------------------------------
hfutatzhanghb commented on PR #3743:
URL: https://github.com/apache/hadoop/pull/3743#issuecomment-2111693845
> Not sure why this PR has been pick up again. IIRC we have discussed add
tools to support refresh topology online many times. I would like to share my
concern again. This is one high-risky operation, especially for one large
cluster. Considering the case that change the cluster topology randomly then
namenode failover or restart, there will be block transfer flood where the
replications of one block should be going to have enough racks. Before all
blocks have enough racks (which is cost long times) the cluster could enter
high-load phase and couldn't process any requests from client. In one word, be
careful to refresh topology online if there is no stability guarantee measures.
@Hexiaoqiao Sir, thanks for your reminding. I got your concern here. I
think this PR just provide a tool which can refresh topology without restarting
namenode. As for how to use this tool safely to prevent causing transfer flood
belongs to end-user. They can modify a small part of topologay_mappings.data
every time, then refreshTopology and wait for transfering completed. Do this
procedure until all parts of topologay_mappings.data updated finally.
> DFSAdmin supports refresh topology info without restarting namenode
> --------------------------------------------------------------------
>
> Key: HDFS-16368
> URL: https://issues.apache.org/jira/browse/HDFS-16368
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: dfsadmin, namanode
> Affects Versions: 2.7.7, 3.3.1
> Reporter: farmmamba
> Assignee: farmmamba
> Priority: Major
> Labels: features, pull-request-available
> Attachments: 0001.patch
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Currently in HDFS, if we update the rack info for rack-awareness, we may need
> to rolling restart namenodes to let it be effective. If cluster is large, the
> cost time of rolling restart namenodes is very log. So, we develope a method
> to refresh topology info without rolling restart namenodes.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
