[
https://issues.apache.org/jira/browse/HDFS-14563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16880562#comment-16880562
]
Íñigo Goiri commented on HDFS-14563:
------------------------------------
Thanks [~hexiaoqiao] for the design doc.
A few comments:
* I don't quite get the need for NFSNodeStateStore having FileNodeStateStore.
One could just point to a NFS path from FileNodeStateStore.
* I would avoid reusing {{dfs.hosts}} and {{dfs.hosts.exclude}} for the ZK
case. I would just add a new property or leverage the {{ha.zookeeper.quorum}}.
* I would try to use the new {{setNodeState()}} interface also when we have the
file based. We will have to be a little careful on who is the writer but I
think we should go ahead as we can keep it backwards compatible even in this
way.
* For the ZK management I already did some refactoring in the past to provide
{{ZKCuratorManager}}, we could refactor further if needed.
> Enhance interface about recommissioning/decommissioning
> -------------------------------------------------------
>
> Key: HDFS-14563
> URL: https://issues.apache.org/jira/browse/HDFS-14563
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs-client, namenode
> Reporter: He Xiaoqiao
> Assignee: He Xiaoqiao
> Priority: Major
> Attachments: HDFS-14563.001.patch, HDFS-14563.002.patch, mt_mode-2.txt
>
>
> In current implementation, if we need to decommissioning or recommissioning
> one datanode, the only way is add the datanode to include or exclude file
> under namenode configuration path then execute command `bin/hadoop dfsadmin
> -refreshNodes` and trigger namenode to reload include/exclude and start to
> recommissioning or decommissioning datanode.
> The shortcomings of this approach is that:
> a. namenode reload include/exclude configuration file from devices, if I/O
> load is high, handler may be blocked.
> b. namenode has to process every datnodes in include and exclude
> configurations, if there are many datanodes (very common for large cluster)
> pending to process, namenode will be hung for hundred seconds to wait
> recommision/decommision finish at the worst since holding write lock.
> I think we should expose one lightweight interface to support recommissioning
> or decommissioning single datanode, thus we can operate datanode using
> dfsadmin more smooth.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
