[jira] [Updated] (HDFS-12115) Ozone: SCM: Add queryNode RPC Call

2017-07-25 Thread Anu Engineer (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HDFS-12115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anu Engineer updated HDFS-12115:

  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

Thank you all for the reviews. I have committed this to the feature branch.

> Ozone: SCM: Add queryNode RPC Call
> --
>
> Key: HDFS-12115
> URL: https://issues.apache.org/jira/browse/HDFS-12115
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Anu Engineer
>Assignee: Anu Engineer
> Fix For: HDFS-7240
>
> Attachments: HDFS-12115-HDFS-7240.001.patch, 
> HDFS-12115-HDFS-7240.002.patch, HDFS-12115-HDFS-7240.003.patch, 
> HDFS-12115-HDFS-7240.004.patch, HDFS-12115-HDFS-7240.005.patch, 
> HDFS-12115-HDFS-7240.006.patch, HDFS-12115-HDFS-7240.007.patch, 
> HDFS-12115-HDFS-7240.008.patch, HDFS-12115-HDFS-7240.009.patch
>
>
> Add queryNode RPC to Storage container location protocol. This allows 
> applications like SCM CLI to get the list of nodes in various states, like 
> Healthy, live or Dead.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-12115) Ozone: SCM: Add queryNode RPC Call

2017-07-24 Thread Anu Engineer (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HDFS-12115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anu Engineer updated HDFS-12115:

Attachment: HDFS-12115-HDFS-7240.009.patch

rebase and updated.

> Ozone: SCM: Add queryNode RPC Call
> --
>
> Key: HDFS-12115
> URL: https://issues.apache.org/jira/browse/HDFS-12115
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Anu Engineer
>Assignee: Anu Engineer
> Fix For: HDFS-7240
>
> Attachments: HDFS-12115-HDFS-7240.001.patch, 
> HDFS-12115-HDFS-7240.002.patch, HDFS-12115-HDFS-7240.003.patch, 
> HDFS-12115-HDFS-7240.004.patch, HDFS-12115-HDFS-7240.005.patch, 
> HDFS-12115-HDFS-7240.006.patch, HDFS-12115-HDFS-7240.007.patch, 
> HDFS-12115-HDFS-7240.008.patch, HDFS-12115-HDFS-7240.009.patch
>
>
> Add queryNode RPC to Storage container location protocol. This allows 
> applications like SCM CLI to get the list of nodes in various states, like 
> Healthy, live or Dead.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-12115) Ozone: SCM: Add queryNode RPC Call

2017-07-21 Thread Anu Engineer (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HDFS-12115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anu Engineer updated HDFS-12115:

Attachment: HDFS-12115-HDFS-7240.008.patch

Thank you for catching that, fixed the test issue.

> Ozone: SCM: Add queryNode RPC Call
> --
>
> Key: HDFS-12115
> URL: https://issues.apache.org/jira/browse/HDFS-12115
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Anu Engineer
>Assignee: Anu Engineer
> Fix For: HDFS-7240
>
> Attachments: HDFS-12115-HDFS-7240.001.patch, 
> HDFS-12115-HDFS-7240.002.patch, HDFS-12115-HDFS-7240.003.patch, 
> HDFS-12115-HDFS-7240.004.patch, HDFS-12115-HDFS-7240.005.patch, 
> HDFS-12115-HDFS-7240.006.patch, HDFS-12115-HDFS-7240.007.patch, 
> HDFS-12115-HDFS-7240.008.patch
>
>
> Add queryNode RPC to Storage container location protocol. This allows 
> applications like SCM CLI to get the list of nodes in various states, like 
> Healthy, live or Dead.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-12115) Ozone: SCM: Add queryNode RPC Call

2017-07-20 Thread Anu Engineer (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HDFS-12115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anu Engineer updated HDFS-12115:

Attachment: HDFS-12115-HDFS-7240.007.patch

fixed the check style issues. One of the UT failures is related to this patch. 
Other failures seem not be.

> Ozone: SCM: Add queryNode RPC Call
> --
>
> Key: HDFS-12115
> URL: https://issues.apache.org/jira/browse/HDFS-12115
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Anu Engineer
>Assignee: Anu Engineer
> Fix For: HDFS-7240
>
> Attachments: HDFS-12115-HDFS-7240.001.patch, 
> HDFS-12115-HDFS-7240.002.patch, HDFS-12115-HDFS-7240.003.patch, 
> HDFS-12115-HDFS-7240.004.patch, HDFS-12115-HDFS-7240.005.patch, 
> HDFS-12115-HDFS-7240.006.patch, HDFS-12115-HDFS-7240.007.patch
>
>
> Add queryNode RPC to Storage container location protocol. This allows 
> applications like SCM CLI to get the list of nodes in various states, like 
> Healthy, live or Dead.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-12115) Ozone: SCM: Add queryNode RPC Call

2017-07-19 Thread Anu Engineer (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HDFS-12115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anu Engineer updated HDFS-12115:

Attachment: HDFS-12115-HDFS-7240.006.patch

bq. InProgressPool.java:  NIT, line 203, extra space between NodeState and 
getNodeState
Fixed.
bq. MockNodeManager.java: line 161-173, it seems this can be replaced by 
getNodes(nodestate).size() 
Fixed. 

bq. Ozone.proto : Add a placeholder for DECOMMISSIONING state?
Fixed.

bq. SCMNodeManager.java : line 413-435, like you mentioned earlier, a node may 
have more than 1 state, e.g both HEALTHY 

The reason why queryNode needs to be able to read multiple states is because
it is an RPC and a network call. But getNodeState is a local call and currently 
only use is to get the liveness of the nodes. If and when we get to automatic 
membership management, we can make this modification. Since no one is calling 
it now, I have left it as it is for now.

bq. line 491: instead of creating a new list, this can be done in java8 style
return currentSet.stream().collect(Collectors.toList());

I presume you are referring to  StorageContainerManager.java.
The reason why I am not doing it is Line: 479. Since this is an "And",
if we find any null sets for any attribute, we return an empty set. 
With an empty ResultSet, the return is consistent, the same set object which is 
always non-null.


> Ozone: SCM: Add queryNode RPC Call
> --
>
> Key: HDFS-12115
> URL: https://issues.apache.org/jira/browse/HDFS-12115
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Anu Engineer
>Assignee: Anu Engineer
> Fix For: HDFS-7240
>
> Attachments: HDFS-12115-HDFS-7240.001.patch, 
> HDFS-12115-HDFS-7240.002.patch, HDFS-12115-HDFS-7240.003.patch, 
> HDFS-12115-HDFS-7240.004.patch, HDFS-12115-HDFS-7240.005.patch, 
> HDFS-12115-HDFS-7240.006.patch
>
>
> Add queryNode RPC to Storage container location protocol. This allows 
> applications like SCM CLI to get the list of nodes in various states, like 
> Healthy, live or Dead.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-12115) Ozone: SCM: Add queryNode RPC Call

2017-07-16 Thread Anu Engineer (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HDFS-12115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anu Engineer updated HDFS-12115:

Attachment: HDFS-12115-HDFS-7240.005.patch

updating the patch since HDFS-12148 fixed the same issue this patch was fixing 
too. Removed Ozone-defaults.xml changes from this new patch

> Ozone: SCM: Add queryNode RPC Call
> --
>
> Key: HDFS-12115
> URL: https://issues.apache.org/jira/browse/HDFS-12115
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Anu Engineer
>Assignee: Anu Engineer
> Fix For: HDFS-7240
>
> Attachments: HDFS-12115-HDFS-7240.001.patch, 
> HDFS-12115-HDFS-7240.002.patch, HDFS-12115-HDFS-7240.003.patch, 
> HDFS-12115-HDFS-7240.004.patch, HDFS-12115-HDFS-7240.005.patch
>
>
> Add queryNode RPC to Storage container location protocol. This allows 
> applications like SCM CLI to get the list of nodes in various states, like 
> Healthy, live or Dead.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-12115) Ozone: SCM: Add queryNode RPC Call

2017-07-14 Thread Anu Engineer (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HDFS-12115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anu Engineer updated HDFS-12115:

Attachment: HDFS-12115-HDFS-7240.004.patch

Addressing a test failure

> Ozone: SCM: Add queryNode RPC Call
> --
>
> Key: HDFS-12115
> URL: https://issues.apache.org/jira/browse/HDFS-12115
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Anu Engineer
>Assignee: Anu Engineer
> Fix For: HDFS-7240
>
> Attachments: HDFS-12115-HDFS-7240.001.patch, 
> HDFS-12115-HDFS-7240.002.patch, HDFS-12115-HDFS-7240.003.patch, 
> HDFS-12115-HDFS-7240.004.patch
>
>
> Add queryNode RPC to Storage container location protocol. This allows 
> applications like SCM CLI to get the list of nodes in various states, like 
> Healthy, live or Dead.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-12115) Ozone: SCM: Add queryNode RPC Call

2017-07-14 Thread Anu Engineer (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HDFS-12115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anu Engineer updated HDFS-12115:

Attachment: HDFS-12115-HDFS-7240.003.patch

> Ozone: SCM: Add queryNode RPC Call
> --
>
> Key: HDFS-12115
> URL: https://issues.apache.org/jira/browse/HDFS-12115
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Anu Engineer
>Assignee: Anu Engineer
> Fix For: HDFS-7240
>
> Attachments: HDFS-12115-HDFS-7240.001.patch, 
> HDFS-12115-HDFS-7240.002.patch, HDFS-12115-HDFS-7240.003.patch
>
>
> Add queryNode RPC to Storage container location protocol. This allows 
> applications like SCM CLI to get the list of nodes in various states, like 
> Healthy, live or Dead.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-12115) Ozone: SCM: Add queryNode RPC Call

2017-07-13 Thread Anu Engineer (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HDFS-12115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anu Engineer updated HDFS-12115:

Attachment: HDFS-12115-HDFS-7240.002.patch

[~xyao] Thank you for the review comments. Patch v2 addresses all review 
comments.

> Ozone: SCM: Add queryNode RPC Call
> --
>
> Key: HDFS-12115
> URL: https://issues.apache.org/jira/browse/HDFS-12115
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Anu Engineer
>Assignee: Anu Engineer
> Fix For: HDFS-7240
>
> Attachments: HDFS-12115-HDFS-7240.001.patch, 
> HDFS-12115-HDFS-7240.002.patch
>
>
> Add queryNode RPC to Storage container location protocol. This allows 
> applications like SCM CLI to get the list of nodes in various states, like 
> Healthy, live or Dead.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-12115) Ozone: SCM: Add queryNode RPC Call

2017-07-11 Thread Anu Engineer (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HDFS-12115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anu Engineer updated HDFS-12115:

Attachment: HDFS-12115-HDFS-7240.001.patch

This patch will create a test failure in TestKeys, that is because the Datanode 
restart was not restarting the DatanodeStateMachine. With this patch, It does 
restart the DatanodeStateMachine, I will follow up with another JIRA to fix 
that issue.


> Ozone: SCM: Add queryNode RPC Call
> --
>
> Key: HDFS-12115
> URL: https://issues.apache.org/jira/browse/HDFS-12115
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Anu Engineer
>Assignee: Anu Engineer
> Fix For: HDFS-7240
>
> Attachments: HDFS-12115-HDFS-7240.001.patch
>
>
> Add queryNode RPC to Storage container location protocol. This allows 
> applications like SCM CLI to get the list of nodes in various states, like 
> Healthy, live or Dead.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-12115) Ozone: SCM: Add queryNode RPC Call

2017-07-11 Thread Anu Engineer (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HDFS-12115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anu Engineer updated HDFS-12115:

Status: Patch Available  (was: Open)

> Ozone: SCM: Add queryNode RPC Call
> --
>
> Key: HDFS-12115
> URL: https://issues.apache.org/jira/browse/HDFS-12115
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Anu Engineer
>Assignee: Anu Engineer
> Fix For: HDFS-7240
>
> Attachments: HDFS-12115-HDFS-7240.001.patch
>
>
> Add queryNode RPC to Storage container location protocol. This allows 
> applications like SCM CLI to get the list of nodes in various states, like 
> Healthy, live or Dead.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org