[jira] [Comment Edited] (HDFS-9391) Update webUI/JMX to display maintenance state info

2017-01-10 Thread Ming Ma (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-9391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817119#comment-15817119
 ] 

Ming Ma edited comment on HDFS-9391 at 1/11/17 4:23 AM:


Thanks [~manojg] for the contribution. Thanks [~dilaver] and [~eddyxu] for the 
review. I have committed the patch to trunk and branch-2.


was (Author: mingma):
Thanks [~manojg] for the contribution. Thanks [~eddyxu] for the review. I have 
committed the patch to trunk and branch-2.

> Update webUI/JMX to display maintenance state info
> --
>
> Key: HDFS-9391
> URL: https://issues.apache.org/jira/browse/HDFS-9391
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>Affects Versions: 3.0.0-alpha1
>Reporter: Ming Ma
>Assignee: Manoj Govindassamy
> Fix For: 2.9.0, 3.0.0-alpha2
>
> Attachments: HDFS-9391-MaintenanceMode-WebUI.pdf, 
> HDFS-9391-branch-2-MaintenanceMode-WebUI.pdf, HDFS-9391-branch-2.01.patch, 
> HDFS-9391-branch-2.02.patch, HDFS-9391.01.patch, HDFS-9391.02.patch, 
> HDFS-9391.03.patch, HDFS-9391.04.patch, Maintenance webUI.png
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Comment Edited] (HDFS-9391) Update webUI/JMX to display maintenance state info

2017-01-09 Thread Manoj Govindassamy (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-9391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15812904#comment-15812904
 ] 

Manoj Govindassamy edited comment on HDFS-9391 at 1/9/17 9:30 PM:
--

Sure. 

Its about whether we should include all blocks of *Maintenance + Decommission* 
states under "Block with No Live Replicas" for each of DN in the "Entering  
Maintenance" and "Decommissioning" page. Previously i was trying to have them 
include only one of these states (as per the initial discussion in this jira). 
But, thinking more about it and after your discussing i feel including both 
these states makes sense. Will upload the new patch soon. Thanks a lot for the 
review.


was (Author: manojg):
Sure. 

Its about whether we should include all blocks of *Maintenance + Decommission* 
states under "Block with No Live Replicas" for each of DN in the "Entering  
Maintenance" and "Decommissioning" page. Previously i was trying to have them 
include one of these states. But, thinking more about it and after your 
discussing i feel including both these states makes sense. Will upload the new 
patch soon. Thanks a lot for the review.

> Update webUI/JMX to display maintenance state info
> --
>
> Key: HDFS-9391
> URL: https://issues.apache.org/jira/browse/HDFS-9391
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>Affects Versions: 3.0.0-alpha1
>Reporter: Ming Ma
>Assignee: Manoj Govindassamy
> Attachments: HDFS-9391-MaintenanceMode-WebUI.pdf, HDFS-9391.01.patch, 
> HDFS-9391.02.patch, HDFS-9391.03.patch, Maintenance webUI.png
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Comment Edited] (HDFS-9391) Update webUI/JMX to display maintenance state info

2017-01-05 Thread Ming Ma (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-9391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15803538#comment-15803538
 ] 

Ming Ma edited comment on HDFS-9391 at 1/6/17 4:41 AM:
---

A given replica is only in one admin state, normal, decommission or 
maintenance. But {{NumberReplicas}} represents the state of all replicas. Thus 
for the case "One replica is decommissioning and two replicas of the same block 
are entering maintenance", {{NumberReplicas#decommissionedAndDecommissioning == 
1}}, {{NumberReplicas#maintenanceReplicas() == 2}}. No?


was (Author: mingma):
A given replica is only in one state, either decommission or maintenance. But 
{{NumberReplicas}} represents the state of all replicas. Thus for the case "One 
replica is decommissioning and two replicas of the same block are entering 
maintenance", {{NumberReplicas#decommissionedAndDecommissioning == 1}}, 
{{NumberReplicas#maintenanceReplicas() == 2}}. No?

> Update webUI/JMX to display maintenance state info
> --
>
> Key: HDFS-9391
> URL: https://issues.apache.org/jira/browse/HDFS-9391
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>Affects Versions: 3.0.0-alpha1
>Reporter: Ming Ma
>Assignee: Manoj Govindassamy
> Attachments: HDFS-9391-MaintenanceMode-WebUI.pdf, HDFS-9391.01.patch, 
> HDFS-9391.02.patch, Maintenance webUI.png
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Comment Edited] (HDFS-9391) Update webUI/JMX to display maintenance state info

2017-01-03 Thread Manoj Govindassamy (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-9391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15796527#comment-15796527
 ] 

Manoj Govindassamy edited comment on HDFS-9391 at 1/4/17 12:03 AM:
---

sure, will do 1 & 2.

3. 
>> For the EnteringMaintenanceNodes page, it uses maintenanceOnlyReplicas to 
>> describe Blocks with no live replicas. Should we use 
>> OutOfServiceOnlyReplicas?

Thanks for bringing up this [~mingma]. There are some inconsistencies even with 
"Decommissioning" page and would like to get clarified on that number as well.

* HDFS-9390 updated the {{Decommissioning}} page to use 
{{getOutOfServiceOnlyReplicas()}} instead of {{getDecommissionOnlyReplicas()}}

* But as part of HDFS-9390, getOutOfServiceOnlyReplicas() which got introduced, 
included all Maintenance and Decommission replicas. Effectively, the page has 
been showing all "out of service" replicas, even though the page name is 
"Decommissioning"

Excerpts from Patch v02:
{code}
if ((liveReplicas == 0) &&
(num.decommissionedAndDecommissioning() > 0)) {
  decommissionOnlyReplicas++;
}
if ((liveReplicas == 0) && (num.maintenanceReplicas() > 0)) {
  maintenanceOnlyReplicas++;
}
if ((liveReplicas == 0) && (num.outOfServiceReplicas() > 0)) {
  outOfServiceOnlyReplicas++;
}
{code}

* So, what should "Decommissioning" page actually show ? In the patch v02 
uploaded here, I made this page to include decommission related replicas only. 
And, not all out of service replicas.

* Now coming to "Entering Maintenance" page, what exact replicas should be 
included here ? If we show up "OutOfServiceOnlyReplicas" then it will include 
all decommissioning related replicas as well. So, I am using 
"maintenanceOnlyReplicas" for this page. Do, you still believe showing all 
"OutOfServiceOnlyReplicas" would be better here ? Please let me know.



was (Author: manojg):

>> For the EnteringMaintenanceNodes page, it uses maintenanceOnlyReplicas to 
>> describe Blocks with no live replicas. Should we use 
>> OutOfServiceOnlyReplicas?

Thanks for bringing up this [~mingma]. There are some inconsistencies even with 
"Decommissioning" page and would like to get clarified on that number as well.

* HDFS-9390 updated the {{Decommissioning}} page to use 
{{getOutOfServiceOnlyReplicas()}} instead of {{getDecommissionOnlyReplicas()}}

* But as part of HDFS-9390, getOutOfServiceOnlyReplicas() which got introduced 
included all Maintenance and Decommission replicas. Effectively, the page has 
been showing all "out of service" replicas, even though the page name is 
"Decommissioning"

Excerpts from Patch v02:
{code}
if ((liveReplicas == 0) &&
(num.decommissionedAndDecommissioning() > 0)) {
  decommissionOnlyReplicas++;
}
if ((liveReplicas == 0) && (num.maintenanceReplicas() > 0)) {
  maintenanceOnlyReplicas++;
}
if ((liveReplicas == 0) && (num.outOfServiceReplicas() > 0)) {
  outOfServiceOnlyReplicas++;
}
{code}

* So, what should "Decommissioning" page actually show ? In the patch v02 
uploaded here, I made this page to include decommission related replicas only. 
And, not all out of service replicas.

* Now coming to "Entering Maintenance" page, what exact replicas should be 
included here ? If we show up "OutOfServiceOnlyReplicas" then it will include 
all decommissioning related replicas as well. So, I am using 
"maintenanceOnlyReplicas" for this page. Do, you still believe showing all 
"OutOfServiceOnlyReplicas" would be better here ? 






> Update webUI/JMX to display maintenance state info
> --
>
> Key: HDFS-9391
> URL: https://issues.apache.org/jira/browse/HDFS-9391
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>Affects Versions: 3.0.0-alpha1
>Reporter: Ming Ma
>Assignee: Manoj Govindassamy
> Attachments: HDFS-9391-MaintenanceMode-WebUI.pdf, HDFS-9391.01.patch, 
> HDFS-9391.02.patch, Maintenance webUI.png
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Comment Edited] (HDFS-9391) Update webUI/JMX to display maintenance state info

2016-12-19 Thread Manoj Govindassamy (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-9391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15762583#comment-15762583
 ] 

Manoj Govindassamy edited comment on HDFS-9391 at 12/19/16 11:12 PM:
-

Thanks for the quick review. Much appreciated.

bq. {{.put("inMaintenance", node.isInMaintenance())}} might not be necessary 
given adminState
True. Now that {{getDeadNodes}} JSON also has adminState, both 
{{decommissioned}} and {{inMaintenance}} are not needed. Have changed the usage 
in {{dfshealth.js}} to check for nodes' {{adminState}} instead of 
{{decommissioned}} and {{inMaintenance}} attributes.

bq. Should {{liveDecommissioningReplicas}} be {{OnlyDecommissioningReplicas}} 
which is the old behavior ?
Sounds good.

* Previous to HDFS-9390 fix when MM was not there, in 
{{FSNamesystem#getDecomNodes}}, for _decommissionOnlyReplicas_, we used to 
return {{getDecommissionOnlyReplicas()}} which is both _decommissioning_ and 
_decommissioned_ replicas.
* After HDFS-9390 fix, we started returning 
{{getLeavingServiceStatus().getOutOfServiceOnlyReplicas()}} which is both 
decommission and maintenance replicas. Based on our previous comments 
discussions, I assume we can go back to the old behavior of returning 
{{getDecommissionOnlyReplicas()}} here. I incorporated this change in patch v02.
* Now, in {{FSNameSystem#getEnteringMaintenanceNodes()}}, for 
_maintenanceOnlyReplicas_, will follow the same model and return maintenance 
replicas only -- {{getLeavingServiceStatus().getMaintenanceOnlyReplicas()}}. 





was (Author: manojg):
Thanks for the quick review. Much appreciated.

1. bq. {{.put("inMaintenance", node.isInMaintenance())}} might not be necessary 
given adminState
True. Now that {{getDeadNodes}} JSON also has adminState, both 
{{decommissioned}} and {{inMaintenance}} are not needed. Have changed the usage 
in {{dfshealth.js}} to check for nodes' {{adminState}} instead of 
{{decommissioned}} and {{inMaintenance}} attributes.

2. bq. Should {{liveDecommissioningReplicas}} be 
{{OnlyDecommissioningReplicas}} which is the old behavior ?
Sounds good.

* Previous to HDFS-9390 fix when MM was not there, in 
{{FSNamesystem#getDecomNodes}}, for __decommissionOnlyReplicas__, we used to 
return {{getDecommissionOnlyReplicas()}} which is both __decommissioning__ and 
__decommissioned__ replicas.
* After HDFS-9390 fix, we started returning 
{{getLeavingServiceStatus().getOutOfServiceOnlyReplicas()}} which is both 
decommission and maintenance replicas. Based on our previous comments 
discussions, I assume we can go back to the old behavior of returning 
{{getDecommissionOnlyReplicas()}} here. I incorporated this change in patch v02.
* Now, in {{FSNameSystem#getEnteringMaintenanceNodes()}}, for 
__maintenanceOnlyReplicas__, will follow the same model and return maintenance 
replicas only -- {{getLeavingServiceStatus().getMaintenanceOnlyReplicas()}}. 




> Update webUI/JMX to display maintenance state info
> --
>
> Key: HDFS-9391
> URL: https://issues.apache.org/jira/browse/HDFS-9391
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>Affects Versions: 3.0.0-alpha1
>Reporter: Ming Ma
>Assignee: Manoj Govindassamy
> Attachments: HDFS-9391-MaintenanceMode-WebUI.pdf, HDFS-9391.01.patch, 
> Maintenance webUI.png
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Comment Edited] (HDFS-9391) Update webUI/JMX to display maintenance state info

2016-12-16 Thread Manoj Govindassamy (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-9391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15756191#comment-15756191
 ] 

Manoj Govindassamy edited comment on HDFS-9391 at 12/17/16 3:19 AM:


Attaching v01 patch to address the following:
1. Introduced NameNodeMXBean#getEnteringMaintenanceNodes and the same is 
implemented in FSNameSystem
2. Updated LeavingServiceStatus to include details on Decommissioning, 
Maintenance and OutOfService replicas
3. DecommissionManager to update LeavingServiceStatus
4. Updated dfshealth.html to have details on Maintenance nodes in Summary and 
DataNode Information pages. (Entering Maintenance, Live Maintenance, Dead 
Maintenance)
5. Unit test for the Maintenance mode JMX.

[~eddyxu], [~mingma], Can you please take a look at the patch and comment on 
what can be improved ?


was (Author: manojg):
Attaching v01 patch to address the following:
1. Introduced NameNodeMXBean#getEnteringMaintenanceNodes and the same is 
implemented in FSNameSystem
2. Updated LeavingServiceStatus to include details on Decommissioning, 
Maintenance and OutOfService replicas
3. DecommissionManager to update LeavingServiceStatus
4. Updated dfshealth.html to have details on Maintenance nodes in Summary and 
DataNode Information pages. (Entering Maintenance, Live Maintenance, Dead 
Maintenance)
5. Unit test for the Maintenance mode JMX.

> Update webUI/JMX to display maintenance state info
> --
>
> Key: HDFS-9391
> URL: https://issues.apache.org/jira/browse/HDFS-9391
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>Affects Versions: 3.0.0-alpha1
>Reporter: Ming Ma
>Assignee: Manoj Govindassamy
> Attachments: HDFS-9391-MaintenanceMode-WebUI.pdf, HDFS-9391.01.patch, 
> Maintenance webUI.png
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org