[jira] [Commented] (HDFS-6729) Support maintenance mode for DN

2015-03-08 Thread Ming Ma (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-6729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14352477#comment-14352477
 ] 

Ming Ma commented on HDFS-6729:
---

Eddy, thanks for the work. We didn't know about this at all until Allen pointed 
it out HDFS-7877. Sounds like we should combine the effort.

Maybe we can step back and discuss the design. There are couple key things we 
want to take care of. It will be great if you can check out the design there.

1. Admin interface. Based on our admins input, it seems "dfsadmin 
-refreshNodes" might be easier to use.
2. DN state machine. We define two new states for maintenance states, 
ENTERING_MAINTENANCE and IN_MAINTENANCE. It takes care of the case where there 
are no replicas on other datanodes. It also takes care of different state 
transition,  decomm states to maintenance states.
3. Block management. We alos enforce the read and write operations when 
machines are in maintenance states.

Look forward to the collaboration.


> Support maintenance mode for DN
> ---
>
> Key: HDFS-6729
> URL: https://issues.apache.org/jira/browse/HDFS-6729
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: datanode
>Affects Versions: 2.5.0
>Reporter: Lei (Eddy) Xu
>Assignee: Lei (Eddy) Xu
> Attachments: HDFS-6729.000.patch, HDFS-6729.001.patch, 
> HDFS-6729.002.patch, HDFS-6729.003.patch, HDFS-6729.004.patch, 
> HDFS-6729.005.patch
>
>
> Some maintenance works (e.g., upgrading RAM or add disks) on DataNode only 
> takes a short amount of time (e.g., 10 minutes). In these cases, the users do 
> not want to report missing blocks on this DN because the DN will be online 
> shortly without data lose. Thus, we need a maintenance mode for a DN so that 
> maintenance work can be carried out on the DN without having to decommission 
> it or the DN being marked as dead.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HDFS-6729) Support maintenance mode for DN

2015-01-27 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-6729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294539#comment-14294539
 ] 

Hadoop QA commented on HDFS-6729:
-

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12694856/HDFS-6729.005.patch
  against trunk revision 8bf6f0b.

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 3 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 core tests{color}.  The patch failed these unit tests in 
hadoop-hdfs-project/hadoop-hdfs:

  org.apache.hadoop.hdfs.TestLeaseRecovery2

Test results: 
https://builds.apache.org/job/PreCommit-HDFS-Build/9350//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9350//console

This message is automatically generated.

> Support maintenance mode for DN
> ---
>
> Key: HDFS-6729
> URL: https://issues.apache.org/jira/browse/HDFS-6729
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: datanode
>Affects Versions: 2.5.0
>Reporter: Lei (Eddy) Xu
>Assignee: Lei (Eddy) Xu
> Attachments: HDFS-6729.000.patch, HDFS-6729.001.patch, 
> HDFS-6729.002.patch, HDFS-6729.003.patch, HDFS-6729.004.patch, 
> HDFS-6729.005.patch
>
>
> Some maintenance works (e.g., upgrading RAM or add disks) on DataNode only 
> takes a short amount of time (e.g., 10 minutes). In these cases, the users do 
> not want to report missing blocks on this DN because the DN will be online 
> shortly without data lose. Thus, we need a maintenance mode for a DN so that 
> maintenance work can be carried out on the DN without having to decommission 
> it or the DN being marked as dead.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HDFS-6729) Support maintenance mode for DN

2015-01-23 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-6729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14290402#comment-14290402
 ] 

Hadoop QA commented on HDFS-6729:
-

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12694290/HDFS-6729.004.patch
  against trunk revision 8f26d5a.

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 2 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 core tests{color}.  The patch failed these unit tests in 
hadoop-hdfs-project/hadoop-hdfs:

  org.apache.hadoop.cli.TestHDFSCLI

Test results: 
https://builds.apache.org/job/PreCommit-HDFS-Build/9321//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9321//console

This message is automatically generated.

> Support maintenance mode for DN
> ---
>
> Key: HDFS-6729
> URL: https://issues.apache.org/jira/browse/HDFS-6729
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: datanode
>Affects Versions: 2.5.0
>Reporter: Lei (Eddy) Xu
>Assignee: Lei (Eddy) Xu
> Attachments: HDFS-6729.000.patch, HDFS-6729.001.patch, 
> HDFS-6729.002.patch, HDFS-6729.003.patch, HDFS-6729.004.patch
>
>
> Some maintenance works (e.g., upgrading RAM or add disks) on DataNode only 
> takes a short amount of time (e.g., 10 minutes). In these cases, the users do 
> not want to report missing blocks on this DN because the DN will be online 
> shortly without data lose. Thus, we need a maintenance mode for a DN so that 
> maintenance work can be carried out on the DN without having to decommission 
> it or the DN being marked as dead.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HDFS-6729) Support maintenance mode for DN

2015-01-23 Thread Andrew Wang (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-6729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14290240#comment-14290240
 ] 

Andrew Wang commented on HDFS-6729:
---

Hey Eddy, quick question, this looks like soft state that isn't persisted 
across NN restarts / failovers. Is that suitable for the target usecases?

> Support maintenance mode for DN
> ---
>
> Key: HDFS-6729
> URL: https://issues.apache.org/jira/browse/HDFS-6729
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: datanode
>Affects Versions: 2.5.0
>Reporter: Lei (Eddy) Xu
>Assignee: Lei (Eddy) Xu
> Attachments: HDFS-6729.000.patch, HDFS-6729.001.patch, 
> HDFS-6729.002.patch, HDFS-6729.003.patch, HDFS-6729.004.patch
>
>
> Some maintenance works (e.g., upgrading RAM or add disks) on DataNode only 
> takes a short amount of time (e.g., 10 minutes). In these cases, the users do 
> not want to report missing blocks on this DN because the DN will be online 
> shortly without data lose. Thus, we need a maintenance mode for a DN so that 
> maintenance work can be carried out on the DN without having to decommission 
> it or the DN being marked as dead.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HDFS-6729) Support maintenance mode for DN

2014-09-19 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-6729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14141320#comment-14141320
 ] 

Hadoop QA commented on HDFS-6729:
-

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12670046/HDFS-6729.003.patch
  against trunk revision 444acf8.

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 1 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 core tests{color}.  The patch failed these unit tests in 
hadoop-hdfs-project/hadoop-hdfs:

  org.apache.hadoop.hdfs.web.TestWebHdfsFileSystemContract
  org.apache.hadoop.hdfs.server.balancer.TestBalancer
  org.apache.hadoop.hdfs.server.mover.TestStorageMover
  
org.apache.hadoop.hdfs.server.namenode.ha.TestPipelinesFailover
  org.apache.hadoop.hdfs.TestEncryptionZonesWithKMS

{color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-HDFS-Build/8113//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8113//console

This message is automatically generated.

> Support maintenance mode for DN
> ---
>
> Key: HDFS-6729
> URL: https://issues.apache.org/jira/browse/HDFS-6729
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: datanode
>Affects Versions: 2.5.0
>Reporter: Lei (Eddy) Xu
>Assignee: Lei (Eddy) Xu
> Attachments: HDFS-6729.000.patch, HDFS-6729.001.patch, 
> HDFS-6729.002.patch, HDFS-6729.003.patch
>
>
> Some maintenance works (e.g., upgrading RAM or add disks) on DataNode only 
> takes a short amount of time (e.g., 10 minutes). In these cases, the users do 
> not want to report missing blocks on this DN because the DN will be online 
> shortly without data lose. Thus, we need a maintenance mode for a DN so that 
> maintenance work can be carried out on the DN without having to decommission 
> it or the DN being marked as dead.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HDFS-6729) Support maintenance mode for DN

2014-08-25 Thread Lei (Eddy) Xu (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-6729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14109396#comment-14109396
 ] 

Lei (Eddy) Xu commented on HDFS-6729:
-

In this patch, the user should use CLI tool {{hdfs dfsadmin -maintainDatanode}} 
to enable and disable maintenance mode for DataNode, instead of using a 
"maintenance file" as the "exclude file" used in decommission mode, since the 
_maintenance mode_ should be temporary. This way could avoid the case that 
system administrator runs {{dfsadmin -refreshNodes}} to accidentally extends 
the expiration time for DataNode.

> Support maintenance mode for DN
> ---
>
> Key: HDFS-6729
> URL: https://issues.apache.org/jira/browse/HDFS-6729
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: datanode
>Affects Versions: 2.5.0
>Reporter: Lei (Eddy) Xu
>Assignee: Lei (Eddy) Xu
> Attachments: HDFS-6729.000.patch, HDFS-6729.001.patch, 
> HDFS-6729.002.patch
>
>
> Some maintenance works (e.g., upgrading RAM or add disks) on DataNode only 
> takes a short amount of time (e.g., 10 minutes). In these cases, the users do 
> not want to report missing blocks on this DN because the DN will be online 
> shortly without data lose. Thus, we need a maintenance mode for a DN so that 
> maintenance work can be carried out on the DN without having to decommission 
> it or the DN being marked as dead.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HDFS-6729) Support maintenance mode for DN

2014-08-23 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-6729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14107969#comment-14107969
 ] 

Hadoop QA commented on HDFS-6729:
-

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12663844/HDFS-6729.002.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 1 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 core tests{color}.  The patch failed these unit tests in 
hadoop-hdfs-project/hadoop-hdfs:

  org.apache.hadoop.hdfs.web.TestWebHdfsFileSystemContract
  
org.apache.hadoop.hdfs.server.namenode.ha.TestPipelinesFailover

{color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-HDFS-Build/7741//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7741//console

This message is automatically generated.

> Support maintenance mode for DN
> ---
>
> Key: HDFS-6729
> URL: https://issues.apache.org/jira/browse/HDFS-6729
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: datanode
>Affects Versions: 2.5.0
>Reporter: Lei (Eddy) Xu
>Assignee: Lei (Eddy) Xu
> Attachments: HDFS-6729.000.patch, HDFS-6729.001.patch, 
> HDFS-6729.002.patch
>
>
> Some maintenance works (e.g., upgrading RAM or add disks) on DataNode only 
> takes a short amount of time (e.g., 10 minutes). In these cases, the users do 
> not want to report missing blocks on this DN because the DN will be online 
> shortly without data lose. Thus, we need a maintenance mode for a DN so that 
> maintenance work can be carried out on the DN without having to decommission 
> it or the DN being marked as dead.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HDFS-6729) Support maintenance mode for DN

2014-08-22 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-6729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14107756#comment-14107756
 ] 

Hadoop QA commented on HDFS-6729:
-

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12663686/HDFS-6729.001.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 1 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 core tests{color}.  The patch failed these unit tests in 
hadoop-hdfs-project/hadoop-hdfs:

  org.apache.hadoop.hdfs.TestEncryptionZones
  org.apache.hadoop.hdfs.server.datanode.TestBPOfferService
  org.apache.hadoop.security.TestRefreshUserMappings
  
org.apache.hadoop.hdfs.server.namenode.ha.TestPipelinesFailover
  org.apache.hadoop.hdfs.server.namenode.TestCacheDirectives
  
org.apache.hadoop.hdfs.server.namenode.TestNamenodeCapacityReport

{color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-HDFS-Build/7725//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7725//console

This message is automatically generated.

> Support maintenance mode for DN
> ---
>
> Key: HDFS-6729
> URL: https://issues.apache.org/jira/browse/HDFS-6729
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: datanode
>Affects Versions: 2.5.0
>Reporter: Lei (Eddy) Xu
>Assignee: Lei (Eddy) Xu
> Attachments: HDFS-6729.000.patch, HDFS-6729.001.patch
>
>
> Some maintenance works (e.g., upgrading RAM or add disks) on DataNode only 
> takes a short amount of time (e.g., 10 minutes). In these cases, the users do 
> not want to report missing blocks on this DN because the DN will be online 
> shortly without data lose. Thus, we need a maintenance mode for a DN so that 
> maintenance work can be carried out on the DN without having to decommission 
> it or the DN being marked as dead.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HDFS-6729) Support maintenance mode for DN

2014-08-21 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-6729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14106354#comment-14106354
 ] 

Hadoop QA commented on HDFS-6729:
-

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12663545/HDFS-6729.000.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 1 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 2.0.3) warnings.

{color:red}-1 release audit{color}.  The applied patch generated 3 
release audit warnings.

{color:red}-1 core tests{color}.  The test build failed in 
hadoop-hdfs-project/hadoop-hdfs 

{color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-HDFS-Build/7714//testReport/
Release audit warnings: 
https://builds.apache.org/job/PreCommit-HDFS-Build/7714//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7714//console

This message is automatically generated.

> Support maintenance mode for DN
> ---
>
> Key: HDFS-6729
> URL: https://issues.apache.org/jira/browse/HDFS-6729
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: datanode
>Affects Versions: 2.5.0
>Reporter: Lei (Eddy) Xu
>Assignee: Lei (Eddy) Xu
> Attachments: HDFS-6729.000.patch
>
>
> Some maintenance works (e.g., upgrading RAM or add disks) on DataNode only 
> takes a short amount of time (e.g., 10 minutes). In these cases, the users do 
> not want to report missing blocks on this DN because the DN will be online 
> shortly without data lose. Thus, we need a maintenance mode for a DN so that 
> maintenance work can be carried out on the DN without having to decommission 
> it or the DN being marked as dead.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HDFS-6729) Support maintenance mode for DN

2014-07-25 Thread Lei (Eddy) Xu (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-6729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14075005#comment-14075005
 ] 

Lei (Eddy) Xu commented on HDFS-6729:
-

[~aw] and [~cmccabe] Thanks for looking into this issue! 

We have customers encountering significant lag time between each decommissioned 
node (e.g., pulling data away from each other node), as described by 
[~cmccabe]. This significant lag time "_blew users maintenance window_."

So, I am wondering whether it is possible to allow users to set a maintenance 
mode for DN for a given time (e.g., the user specifies the maintenance time as 
1 hour), after that if the DN does not come back, NN starts the normal 
re-replicate process?

> Support maintenance mode for DN
> ---
>
> Key: HDFS-6729
> URL: https://issues.apache.org/jira/browse/HDFS-6729
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: datanode
>Affects Versions: 2.4.0
>Reporter: Lei (Eddy) Xu
>
> Some maintenance works (e.g., upgrading RAM or add disks) on DataNode only 
> takes a short amount of time (e.g., 10 minutes). In these cases, the users do 
> not want to report missing blocks on this DN because the DN will be online 
> shortly without data lose. Thus, we need a maintenance mode for a DN so that 
> maintenance work can be carried out on the DN without having to decommission 
> it or the DN being marked as dead.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HDFS-6729) Support maintenance mode for DN

2014-07-25 Thread Colin Patrick McCabe (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-6729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14074651#comment-14074651
 ] 

Colin Patrick McCabe commented on HDFS-6729:


By default it takes 10 and a half minutes until the NameNode starts 
re-replicating anything.  With the stale DN feature turned on, applications 
trying to read from the stale node will be re-directed, so the cluster won't 
experience lag (or at least, not because of applications trying to contact the 
node under maintenance).

So I guess the question is, is it worth adding another state in case the 
maintenance on the datanode can't be finished in 10 minutes?  On the upside, I 
suppose it probably wouldn't be a lot of code.  It would be very similar to the 
stale datanode stuff we already implemented.

> Support maintenance mode for DN
> ---
>
> Key: HDFS-6729
> URL: https://issues.apache.org/jira/browse/HDFS-6729
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: datanode
>Affects Versions: 2.4.0
>Reporter: Lei (Eddy) Xu
>
> Some maintenance works (e.g., upgrading RAM or add disks) on DataNode only 
> takes a short amount of time (e.g., 10 minutes). In these cases, the users do 
> not want to report missing blocks on this DN because the DN will be online 
> shortly without data lose. Thus, we need a maintenance mode for a DN so that 
> maintenance work can be carried out on the DN without having to decommission 
> it or the DN being marked as dead.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HDFS-6729) Support maintenance mode for DN

2014-07-22 Thread Allen Wittenauer (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-6729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14070921#comment-14070921
 ] 

Allen Wittenauer commented on HDFS-6729:


Is this just to avoid the replication? If it is a 10 min downtime, the 
replication will have just started and stop just as fast.  If it is longer than 
10 mins, then you really are better off letting the system do its thing in case 
the node doesn't come back.  There is also the problem of what to do about the 
NM, since it has the same problem...

> Support maintenance mode for DN
> ---
>
> Key: HDFS-6729
> URL: https://issues.apache.org/jira/browse/HDFS-6729
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: datanode
>Affects Versions: 2.4.0
>Reporter: Lei (Eddy) Xu
>
> Some maintenance works (e.g., upgrading RAM or add disks) on DataNode only 
> takes a short amount of time (e.g., 10 minutes). In these cases, the users do 
> not want to report missing blocks on this DN because the DN will be online 
> shortly without data lose. Thus, we need a maintenance mode for a DN so that 
> maintenance work can be carried out on the DN without having to decommission 
> it or the DN being marked as dead.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HDFS-6729) Support maintenance mode for DN

2014-07-22 Thread Lei (Eddy) Xu (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-6729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14070869#comment-14070869
 ] 

Lei (Eddy) Xu commented on HDFS-6729:
-

Hi, [~aw]. We have cases users to upgrade RAM and/or add disks etc., which only 
takes a short amount of time (e.g., 10 minutes downtime). It might not need to 
report missing data on this DN if the users are certain that the DN will come 
back shortly.

> Support maintenance mode for DN
> ---
>
> Key: HDFS-6729
> URL: https://issues.apache.org/jira/browse/HDFS-6729
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: datanode
>Affects Versions: 2.4.0
>Reporter: Lei (Eddy) Xu
>
> We need a maintenance mode for a DN so that maintenance work can be carried 
> out on the DN without having to decommission it or the DN being marked as 
> dead.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HDFS-6729) Support maintenance mode for DN

2014-07-22 Thread Allen Wittenauer (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-6729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14070852#comment-14070852
 ] 

Allen Wittenauer commented on HDFS-6729:


What types of operations are expected to be done while the DN is in maint mode?

> Support maintenance mode for DN
> ---
>
> Key: HDFS-6729
> URL: https://issues.apache.org/jira/browse/HDFS-6729
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: datanode
>Affects Versions: 2.4.0
>Reporter: Lei (Eddy) Xu
>
> We need a maintenance mode for a DN so that maintenance work can be carried 
> out on the DN without having to decommission it or the DN being marked as 
> dead.



--
This message was sent by Atlassian JIRA
(v6.2#6252)