[
https://issues.apache.org/jira/browse/HDFS-1547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978182#action_12978182
]
Suresh Srinivas commented on HDFS-1547:
---------------------------------------
> Does the dfsadmin -refreshNodes command upload the excludes and includes and
> the decom file (all three?) into the namenode's in memory state?
It is good to persist this at namenode. That way post namenode restart, the
datanodes that were intended to be out of service will not come back into
service.
> when a node is being-decommissioned state, why do you propose that it reduces
> the frequency of block reports and heartbeats? Is this really needed?...
Sanjay's comment addresses this.
> I really like the fact that you are proposing the decommissioned nodes are
> not auto shutdown.
This was my original proposal. After thinking a bit, I see following issues:
* Not shutting down datanodes changes the intent of HADOOP-442; shutting down
datanode ensures problematic datanodes cannot be used any more.
* Currently the shutdown ensures datanodes are not used by the namenode. I am
concerned not shutting down datanode could result in namenode using using
decommissioned nodes in unintended ways.
* My concern earlier was, there is no way to figure out if datanode is dead
because decommission is complete or for other reasons. However namenode has
the state that datanode is decommissioned. We could improve current dead node
list to show two lists, decommissioned and dead list in namenode WebUI.
* The storage free capacity from the decommissioned datanodes should not be
counted towards available storage capacity of the cluster. Only used capacity
should count towards cluster used capacity.
Current behavior is:
# A currently registered datanode is decommissioned and then disallowed to
communicate with NN.
# A datanode that had registered previously with NN (after NN is restarted) and
currently not registered is decommissioned, if it registers with NN.
# A datanode that had not registered (after NN restart) is disallowed from
registering and is never decommissioned.
By changing the behavior (3), most of what I had proposed for decom file can be
achieved. This also avoids two config files for exclude and decom with very
little and subtle sematic difference.
> Improve decommission mechanism
> ------------------------------
>
> Key: HDFS-1547
> URL: https://issues.apache.org/jira/browse/HDFS-1547
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: name-node
> Affects Versions: 0.23.0
> Reporter: Suresh Srinivas
> Assignee: Suresh Srinivas
> Fix For: 0.23.0
>
>
> Current decommission mechanism driven using exclude file has several issues.
> This bug proposes some changes in the mechanism for better manageability. See
> the proposal in the next comment for more details.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
