[jira] [Commented] (HDFS-11034) Provide a command line tool to clear decommissioned DataNode information from the NameNode without restarting.
[
https://issues.apache.org/jira/browse/HDFS-11034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16173117#comment-16173117
]
Brahma Reddy Battula commented on HDFS-11034:
-
can we add one argument to {{hdfs dfsadmin -refreshNodes}} itself instead of
new admin command..?
May be something like {{clearDeadNodes}}..?
> Provide a command line tool to clear decommissioned DataNode information from
> the NameNode without restarting.
> --
>
> Key: HDFS-11034
> URL: https://issues.apache.org/jira/browse/HDFS-11034
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
>Reporter: Chris Nauroth
>Assignee: Gergely Novák
>
> Information about decommissioned DataNodes remains tracked in the NameNode
> for the entire NameNode process lifetime. Currently, the only way to clear
> this information is to restart the NameNode. This issue proposes to add a
> way to clear this information online, without requiring a process restart.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11034) Provide a command line tool to clear decommissioned DataNode information from the NameNode without restarting.
[
https://issues.apache.org/jira/browse/HDFS-11034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15610068#comment-15610068
]
Chris Nauroth commented on HDFS-11034:
--
Hello [~GergelyNovak].
If the decommissioned host is removed from the {{dfs.hosts.exclude}} file,
followed by running {{hdfs dfsadmin -refreshNodes}}, then the host is no longer
considered to be excluded. If the DataNode process is still running, or if
it's restarted accidentally, then that DataNode will re-register with the
NameNode, come back into service and become a candidate for writing new blocks.
I was imagining a new workflow, where the host remains decommissioned, but the
administrator has a way to clear out the in-memory tracked state about that
node. It's interesting that you brought up the exclude file. Since that's
already the existing mechanism for inclusion/exclusion of hosts, I wonder if
there is a way to enhance it to cover this use case, so that administrators
wouldn't need to learn a new command. I'll think about it more (and comments
are welcome from others who have ideas too).
> Provide a command line tool to clear decommissioned DataNode information from
> the NameNode without restarting.
> --
>
> Key: HDFS-11034
> URL: https://issues.apache.org/jira/browse/HDFS-11034
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
>Reporter: Chris Nauroth
>Assignee: Gergely Novák
>
> Information about decommissioned DataNodes remains tracked in the NameNode
> for the entire NameNode process lifetime. Currently, the only way to clear
> this information is to restart the NameNode. This issue proposes to add a
> way to clear this information online, without requiring a process restart.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11034) Provide a command line tool to clear decommissioned DataNode information from the NameNode without restarting.
[
https://issues.apache.org/jira/browse/HDFS-11034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15604985#comment-15604985
]
Gergely Novák commented on HDFS-11034:
--
I'm new here. [~cnauroth] could you please explain to me the difference between
this expected new dfsadmin command and removing the decommissioned datanode(s)
from the {{dfs.hosts}} file and executing {{hdfs dfsadmin -refreshNodes}}?
Thank you.
> Provide a command line tool to clear decommissioned DataNode information from
> the NameNode without restarting.
> --
>
> Key: HDFS-11034
> URL: https://issues.apache.org/jira/browse/HDFS-11034
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
>Reporter: Chris Nauroth
>Assignee: Gergely Novák
>
> Information about decommissioned DataNodes remains tracked in the NameNode
> for the entire NameNode process lifetime. Currently, the only way to clear
> this information is to restart the NameNode. This issue proposes to add a
> way to clear this information online, without requiring a process restart.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11034) Provide a command line tool to clear decommissioned DataNode information from the NameNode without restarting.
[ https://issues.apache.org/jira/browse/HDFS-11034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15589341#comment-15589341 ] Chris Nauroth commented on HDFS-11034: -- We can add a new dfsadmin command to clear this state. It's important to note that for some operations workflows, it's valuable to retain the decommissioned node information. If the operator is working on a series of decommission/recommission steps, then this information is valuable to see which nodes are still remaining in decommissioned state. That likely means that the command line needs to accept an argument for a specific host instead of just blindly clearing all decommissioned node information. Remember to clear from both NameNodes in an HA pair. > Provide a command line tool to clear decommissioned DataNode information from > the NameNode without restarting. > -- > > Key: HDFS-11034 > URL: https://issues.apache.org/jira/browse/HDFS-11034 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Reporter: Chris Nauroth > > Information about decommissioned DataNodes remains tracked in the NameNode > for the entire NameNode process lifetime. Currently, the only way to clear > this information is to restart the NameNode. This issue proposes to add a > way to clear this information online, without requiring a process restart. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
