[GitHub] [ozone] wycccccc opened a new pull request #1648: HDDS-4449. Rename config ozone.datanode.pipeline.limit

2020-12-01 Thread GitBox
wycc opened a new pull request #1648: URL: https://github.com/apache/ozone/pull/1648 ## What changes were proposed in this pull request? Renamed `ozone.datanode.pipeline.limit` a suitable name. ## What is the link to the Apache JIRA https://issues.apache.org/jira/bro

[jira] [Commented] (HDDS-4538) SCM HA needs take care of the generation of clusterID and scmUuid

2020-12-01 Thread Glen Geng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17242127#comment-17242127 ] Glen Geng commented on HDDS-4538: - This is a short term walk around that facilitates dev o

[jira] [Updated] (HDDS-4538) SCM HA needs take care of the generation of clusterID and scmUuid

2020-12-01 Thread Glen Geng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Glen Geng updated HDDS-4538: Description: During the initialization of SCM, it will generate a random clusterID and scmUuid, which will

[jira] [Updated] (HDDS-4538) SCM HA needs take care of the generation of clusterID and scmUuid

2020-12-01 Thread Glen Geng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Glen Geng updated HDDS-4538: Description: During the initialization of SCM, it will generate a random clusterID and scmUuid.   On Data

[jira] [Created] (HDDS-4538) SCM HA needs take care of the generation of clusterID and scmUuid

2020-12-01 Thread Glen Geng (Jira)
Glen Geng created HDDS-4538: --- Summary: SCM HA needs take care of the generation of clusterID and scmUuid Key: HDDS-4538 URL: https://issues.apache.org/jira/browse/HDDS-4538 Project: Hadoop Distributed Data

[jira] [Assigned] (HDDS-4449) Rename config ozone.datanode.pipeline.limit

2020-12-01 Thread YI-CHEN WANG (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YI-CHEN WANG reassigned HDDS-4449: -- Assignee: YI-CHEN WANG > Rename config ozone.datanode.pipeline.limit >

[jira] [Updated] (HDDS-1001) Switch Hadoop version to 3.2.0

2020-12-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-1001: - Labels: pull-request-available (was: ) > Switch Hadoop version to 3.2.0 > ---

[jira] [Updated] (HDDS-4530) Add authorization check for SCM

2020-12-01 Thread YI-CHEN WANG (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YI-CHEN WANG updated HDDS-4530: --- Description: Currently a number of the SCM Client RPCs don't perform authorization check. In my opini

[jira] [Updated] (HDDS-4530) Add authorization check for SCM

2020-12-01 Thread YI-CHEN WANG (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YI-CHEN WANG updated HDDS-4530: --- Description: * Currently a number of the SCM Client RPCs don't perform authorization check. In my o

[jira] [Updated] (HDDS-4265) Refactor OzoneQuota to make it easy to support more quota type

2020-12-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-4265: - Labels: pull-request-available (was: ) > Refactor OzoneQuota to make it easy to support more quot

[GitHub] [ozone] wycccccc opened a new pull request #1647: HDDS-4265. Refactor OzoneQuota to make it easy to support more quota …

2020-12-01 Thread GitBox
wycc opened a new pull request #1647: URL: https://github.com/apache/ozone/pull/1647 ## What changes were proposed in this pull request? This task is to make it easy to support more quota type. I replaced the repeated judgment statements, and now just configure the new type 's

[GitHub] [ozone] GlenGeng commented on a change in pull request #1314: HDDS-3988: DN can distinguish SCMCommand from stale leader SCM

2020-12-01 Thread GitBox
GlenGeng commented on a change in pull request #1314: URL: https://github.com/apache/ozone/pull/1314#discussion_r533924763 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/SCMNodeManager.java ## @@ -130,6 +135,14 @@ public SCMNodeManager(Ozone

[GitHub] [ozone] GlenGeng commented on a change in pull request #1314: HDDS-3988: DN can distinguish SCMCommand from stale leader SCM

2020-12-01 Thread GitBox
GlenGeng commented on a change in pull request #1314: URL: https://github.com/apache/ozone/pull/1314#discussion_r533923194 ## File path: hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/statemachine/StateContext.java ## @@ -470,6 +484,65 @@

[GitHub] [ozone] GlenGeng commented on a change in pull request #1314: HDDS-3988: DN can distinguish SCMCommand from stale leader SCM

2020-12-01 Thread GitBox
GlenGeng commented on a change in pull request #1314: URL: https://github.com/apache/ozone/pull/1314#discussion_r533922364 ## File path: hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/statemachine/StateContext.java ## @@ -470,6 +484,65 @@

[GitHub] [ozone] GlenGeng commented on a change in pull request #1314: HDDS-3988: DN can distinguish SCMCommand from stale leader SCM

2020-12-01 Thread GitBox
GlenGeng commented on a change in pull request #1314: URL: https://github.com/apache/ozone/pull/1314#discussion_r533922178 ## File path: hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/statemachine/StateContext.java ## @@ -470,6 +484,65 @@

[jira] [Comment Edited] (HDDS-3816) Erasure Coding in Apache Hadoop Ozone

2020-12-01 Thread Uma Maheswara Rao G (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17242065#comment-17242065 ] Uma Maheswara Rao G edited comment on HDDS-3816 at 12/2/20, 6:05 AM: ---

[jira] [Commented] (HDDS-3816) Erasure Coding in Apache Hadoop Ozone

2020-12-01 Thread Uma Maheswara Rao G (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17242065#comment-17242065 ] Uma Maheswara Rao G commented on HDDS-3816: --- Thanks a lot [~elek] for taking a l

[GitHub] [ozone] prashantpogde commented on pull request #1611: HDDS-4179. Implement post-finalize SCM logic.

2020-12-01 Thread GitBox
prashantpogde commented on pull request #1611: URL: https://github.com/apache/ozone/pull/1611#issuecomment-736997665 TestOmBlockVersioning failure is unrelated. Its passing on my local setup. This is an automated message from

[jira] [Comment Edited] (HDDS-4533) Avoid rewriting pipeline information during PipelineStateManagerV2Impl initialization

2020-12-01 Thread Glen Geng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17242031#comment-17242031 ] Glen Geng edited comment on HDDS-4533 at 12/2/20, 4:42 AM: --- The

[jira] [Commented] (HDDS-4533) Avoid rewriting pipeline information during PipelineStateManagerV2Impl initialization

2020-12-01 Thread Glen Geng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17242031#comment-17242031 ] Glen Geng commented on HDDS-4533: - The rocksDB could be considered as a snapshot of SCM, a

[GitHub] [ozone] avijayanhwx commented on pull request #1645: HDDS-4463. Verify that no node can start when Software Layout Version < Metadata Layout Version.

2020-12-01 Thread GitBox
avijayanhwx commented on pull request #1645: URL: https://github.com/apache/ozone/pull/1645#issuecomment-736979522 @errose28 The CI failure looks related. Please take a look. This is an automated message from the Apache Git S

[GitHub] [ozone] avijayanhwx commented on a change in pull request #1613: HDDS-4480. Implement OM Prepare Request/Response.

2020-12-01 Thread GitBox
avijayanhwx commented on a change in pull request #1613: URL: https://github.com/apache/ozone/pull/1613#discussion_r533841025 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/upgrade/OMPrepareRequest.java ## @@ -0,0 +1,204 @@ +/* + * Li

[GitHub] [ozone] avijayanhwx commented on a change in pull request #1613: HDDS-4480. Implement OM Prepare Request/Response.

2020-12-01 Thread GitBox
avijayanhwx commented on a change in pull request #1613: URL: https://github.com/apache/ozone/pull/1613#discussion_r533840601 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/upgrade/OMPrepareRequest.java ## @@ -0,0 +1,204 @@ +/* + * Li

[GitHub] [ozone] swagle commented on a change in pull request #1613: HDDS-4480. Implement OM Prepare Request/Response.

2020-12-01 Thread GitBox
swagle commented on a change in pull request #1613: URL: https://github.com/apache/ozone/pull/1613#discussion_r533823145 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/upgrade/OMPrepareRequest.java ## @@ -0,0 +1,196 @@ +/* + * License

[jira] [Created] (HDDS-4537) Implement listKeys that takes a param to not to refreshPipeline

2020-12-01 Thread Bharat Viswanadham (Jira)
Bharat Viswanadham created HDDS-4537: Summary: Implement listKeys that takes a param to not to refreshPipeline Key: HDDS-4537 URL: https://issues.apache.org/jira/browse/HDDS-4537 Project: Hadoop D

[jira] [Updated] (HDDS-4537) Implement listKeys that takes a param to not to refreshPipeline

2020-12-01 Thread Bharat Viswanadham (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharat Viswanadham updated HDDS-4537: - Issue Type: Improvement (was: Bug) > Implement listKeys that takes a param to not to refr

[GitHub] [ozone] bharatviswa504 commented on a change in pull request #1613: HDDS-4480. Implement OM Prepare Request/Response.

2020-12-01 Thread GitBox
bharatviswa504 commented on a change in pull request #1613: URL: https://github.com/apache/ozone/pull/1613#discussion_r533813624 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/upgrade/OMPrepareRequest.java ## @@ -0,0 +1,204 @@ +/* + *

[GitHub] [ozone] bharatviswa504 commented on a change in pull request #1613: HDDS-4480. Implement OM Prepare Request/Response.

2020-12-01 Thread GitBox
bharatviswa504 commented on a change in pull request #1613: URL: https://github.com/apache/ozone/pull/1613#discussion_r533813624 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/upgrade/OMPrepareRequest.java ## @@ -0,0 +1,204 @@ +/* + *

[GitHub] [ozone] bharatviswa504 commented on a change in pull request #1613: HDDS-4480. Implement OM Prepare Request/Response.

2020-12-01 Thread GitBox
bharatviswa504 commented on a change in pull request #1613: URL: https://github.com/apache/ozone/pull/1613#discussion_r533779978 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/upgrade/OMPrepareRequest.java ## @@ -0,0 +1,204 @@ +/* + *

[jira] [Updated] (HDDS-4463) Verify that no node can start when Software Layout Version < Metadata Layout Version

2020-12-01 Thread Ethan Rose (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Rose updated HDDS-4463: - Description: Add tests for OzoneManager and StorageContainerManager classes that will verify an Exception

[GitHub] [ozone] bharatviswa504 commented on a change in pull request #1613: HDDS-4480. Implement OM Prepare Request/Response.

2020-12-01 Thread GitBox
bharatviswa504 commented on a change in pull request #1613: URL: https://github.com/apache/ozone/pull/1613#discussion_r533772936 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/upgrade/OMPrepareRequest.java ## @@ -0,0 +1,204 @@ +/* + *

[jira] [Updated] (HDDS-4529) Upgrade snakeyaml to 1.26+

2020-12-01 Thread Xiaoyu Yao (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HDDS-4529: - Fix Version/s: 1.1.0 Resolution: Fixed Status: Resolved (was: Patch Available) Thanks [

[GitHub] [ozone] xiaoyuyao merged pull request #1640: HDDS-4529. Upgrade snakeyaml to 1.26

2020-12-01 Thread GitBox
xiaoyuyao merged pull request #1640: URL: https://github.com/apache/ozone/pull/1640 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [ozone] xiaoyuyao commented on pull request #1640: HDDS-4529. Upgrade snakeyaml to 1.26

2020-12-01 Thread GitBox
xiaoyuyao commented on pull request #1640: URL: https://github.com/apache/ozone/pull/1640#issuecomment-736859192 LGTM, +1. Thanks @risdenk for fixing this. This is an automated message from the Apache Git Service. To respond

[GitHub] [ozone] bharatviswa504 commented on a change in pull request #1511: HDDS-4123. Integrate OM Open Key Cleanup Service Into Existing Code

2020-12-01 Thread GitBox
bharatviswa504 commented on a change in pull request #1511: URL: https://github.com/apache/ozone/pull/1511#discussion_r532963547 ## File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOpenKeyCleanupService.java ## @@ -0,0 +1,462 @@ +/* + * Lic

[GitHub] [ozone] xiaoyuyao commented on a change in pull request #1636: HDDS-4496. Separate client and server2server GRPC services of datanode

2020-12-01 Thread GitBox
xiaoyuyao commented on a change in pull request #1636: URL: https://github.com/apache/ozone/pull/1636#discussion_r533752034 ## File path: hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/replication/ReplicationServer.java ## @@ -0,0 +1,147 @@ +/**

[jira] [Resolved] (HDDS-2753) Test OM Volume and Bucket operations are idempotent

2020-12-01 Thread Hanisha Koneru (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hanisha Koneru resolved HDDS-2753. -- Resolution: Not A Problem > Test OM Volume and Bucket operations are idempotent > --

[GitHub] [ozone] errose28 commented on a change in pull request #1613: HDDS-4480. Implement OM Prepare Request/Response.

2020-12-01 Thread GitBox
errose28 commented on a change in pull request #1613: URL: https://github.com/apache/ozone/pull/1613#discussion_r533707771 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/upgrade/OMPrepareRequest.java ## @@ -0,0 +1,196 @@ +/* + * Licen

[GitHub] [ozone] errose28 commented on a change in pull request #1613: HDDS-4480. Implement OM Prepare Request/Response.

2020-12-01 Thread GitBox
errose28 commented on a change in pull request #1613: URL: https://github.com/apache/ozone/pull/1613#discussion_r533706080 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/upgrade/OMPrepareRequest.java ## @@ -0,0 +1,196 @@ +/* + * Licen

[GitHub] [ozone] avijayanhwx commented on a change in pull request #1613: HDDS-4480. Implement OM Prepare Request/Response.

2020-12-01 Thread GitBox
avijayanhwx commented on a change in pull request #1613: URL: https://github.com/apache/ozone/pull/1613#discussion_r533705379 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/upgrade/OMPrepareRequest.java ## @@ -0,0 +1,196 @@ +/* + * Li

[GitHub] [ozone] swagle commented on a change in pull request #1613: HDDS-4480. Implement OM Prepare Request/Response.

2020-12-01 Thread GitBox
swagle commented on a change in pull request #1613: URL: https://github.com/apache/ozone/pull/1613#discussion_r533697329 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/upgrade/OMPrepareRequest.java ## @@ -0,0 +1,196 @@ +/* + * License

[GitHub] [ozone] hanishakoneru commented on pull request #1523: HDDS-4320. Let Ozone input streams implement CanUnbuffer

2020-12-01 Thread GitBox
hanishakoneru commented on pull request #1523: URL: https://github.com/apache/ozone/pull/1523#issuecomment-736795624 Thanks @adoroszlai. LGTM. +1. This is an automated message from the Apache Git Service. To respond to the me

[GitHub] [ozone] errose28 commented on a change in pull request #1613: HDDS-4480. Implement OM Prepare Request/Response.

2020-12-01 Thread GitBox
errose28 commented on a change in pull request #1613: URL: https://github.com/apache/ozone/pull/1613#discussion_r533695961 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/upgrade/OMPrepareRequest.java ## @@ -0,0 +1,196 @@ +/* + * Licen

[GitHub] [ozone] errose28 commented on a change in pull request #1613: HDDS-4480. Implement OM Prepare Request/Response.

2020-12-01 Thread GitBox
errose28 commented on a change in pull request #1613: URL: https://github.com/apache/ozone/pull/1613#discussion_r533695604 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/upgrade/OMPrepareRequest.java ## @@ -0,0 +1,196 @@ +/* + * Licen

[GitHub] [ozone] errose28 commented on a change in pull request #1613: HDDS-4480. Implement OM Prepare Request/Response.

2020-12-01 Thread GitBox
errose28 commented on a change in pull request #1613: URL: https://github.com/apache/ozone/pull/1613#discussion_r533693610 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/upgrade/OMPrepareRequest.java ## @@ -0,0 +1,196 @@ +/* + * Licen

[jira] [Updated] (HDDS-4502) [OFS] Listing volumes under root should return all volumes whenever possible

2020-12-01 Thread Siyao Meng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siyao Meng updated HDDS-4502: - Resolution: Not A Bug Status: Resolved (was: Patch Available) Closing the jira for now. As this s

[GitHub] [ozone] smengcl commented on pull request #1601: HDDS-4404. Datanode can go OOM when a Recon or SCM Server is very slow in processing reports

2020-12-01 Thread GitBox
smengcl commented on pull request #1601: URL: https://github.com/apache/ozone/pull/1601#issuecomment-736765247 Thanks @runzhiwang @avijayanhwx @GlenGeng for the review and comments. CI still has unrelated failures (`TestOzoneManagerHAWithData#testOMRestart` and `acceptance.ozone-om-h

[GitHub] [ozone] smengcl closed pull request #1614: HDDS-4502. [OFS] Listing volumes under root should return all volumes whenever possible

2020-12-01 Thread GitBox
smengcl closed pull request #1614: URL: https://github.com/apache/ozone/pull/1614 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [ozone] swagle commented on a change in pull request #1613: HDDS-4480. Implement OM Prepare Request/Response.

2020-12-01 Thread GitBox
swagle commented on a change in pull request #1613: URL: https://github.com/apache/ozone/pull/1613#discussion_r533634705 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/upgrade/OMPrepareRequest.java ## @@ -0,0 +1,196 @@ +/* + * License

[GitHub] [ozone] swagle commented on a change in pull request #1613: HDDS-4480. Implement OM Prepare Request/Response.

2020-12-01 Thread GitBox
swagle commented on a change in pull request #1613: URL: https://github.com/apache/ozone/pull/1613#discussion_r533631426 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/upgrade/OMPrepareRequest.java ## @@ -0,0 +1,196 @@ +/* + * License

[GitHub] [ozone] swagle commented on a change in pull request #1613: HDDS-4480. Implement OM Prepare Request/Response.

2020-12-01 Thread GitBox
swagle commented on a change in pull request #1613: URL: https://github.com/apache/ozone/pull/1613#discussion_r533629203 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/upgrade/OMPrepareRequest.java ## @@ -0,0 +1,196 @@ +/* + * License

[GitHub] [ozone] avijayanhwx commented on a change in pull request #1645: HDDS-4463. Verify that no node can start when Software Layout Version < Metadata Layout Version.

2020-12-01 Thread GitBox
avijayanhwx commented on a change in pull request #1645: URL: https://github.com/apache/ozone/pull/1645#discussion_r533596046 ## File path: hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneConsts.java ## @@ -43,7 +43,7 @@ public static final String STORAGE_ID =

[GitHub] [ozone] avijayanhwx commented on a change in pull request #1645: HDDS-4463. Verify that no node can start when Software Layout Version < Metadata Layout Version.

2020-12-01 Thread GitBox
avijayanhwx commented on a change in pull request #1645: URL: https://github.com/apache/ozone/pull/1645#discussion_r533596046 ## File path: hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneConsts.java ## @@ -43,7 +43,7 @@ public static final String STORAGE_ID =

[GitHub] [ozone] avijayanhwx opened a new pull request #1646: HDDS-4536. Add more unit tests for OM layout version manager.

2020-12-01 Thread GitBox
avijayanhwx opened a new pull request #1646: URL: https://github.com/apache/ozone/pull/1646 ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) ## What is the link to the Apache JIRA (Please create an issue in ASF JIRA before

[jira] [Updated] (HDDS-4536) Add more unit tests for OM layout version manager.

2020-12-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-4536: - Labels: pull-request-available (was: ) > Add more unit tests for OM layout version manager. > ---

[jira] [Created] (HDDS-4536) Add more unit tests for OM layout version manager.

2020-12-01 Thread Aravindan Vijayan (Jira)
Aravindan Vijayan created HDDS-4536: --- Summary: Add more unit tests for OM layout version manager. Key: HDDS-4536 URL: https://issues.apache.org/jira/browse/HDDS-4536 Project: Hadoop Distributed Data

[jira] [Assigned] (HDDS-4523) Optionally sort pipeline in getContainerWithPipeline

2020-12-01 Thread Stephen O'Donnell (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen O'Donnell reassigned HDDS-4523: --- Assignee: Stephen O'Donnell > Optionally sort pipeline in getContainerWithPipeline >

[jira] [Updated] (HDDS-4473) Reduce number of sortDatanodes RPC calls

2020-12-01 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-4473: --- Labels: (was: pull-request-available) > Reduce number of sortDatanodes RPC calls > -

[jira] [Updated] (HDDS-4473) Reduce number of sortDatanodes RPC calls

2020-12-01 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-4473: --- Fix Version/s: 1.1.0 Resolution: Done Status: Resolved (was: Patch Available

[GitHub] [ozone] adoroszlai merged pull request #1610: HDDS-4473. Reduce number of sortDatanodes RPC calls

2020-12-01 Thread GitBox
adoroszlai merged pull request #1610: URL: https://github.com/apache/ozone/pull/1610 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [ozone] adoroszlai commented on pull request #1610: HDDS-4473. Reduce number of sortDatanodes RPC calls

2020-12-01 Thread GitBox
adoroszlai commented on pull request #1610: URL: https://github.com/apache/ozone/pull/1610#issuecomment-736644270 Thanks @sodonnel for the reviews. This is an automated message from the Apache Git Service. To respond to the m

[jira] [Updated] (HDDS-4463) Verify that no node can start when Software Layout Version < Metadata Layout Version

2020-12-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-4463: - Labels: pull-request-available (was: ) > Verify that no node can start when Software Layout Versi

[GitHub] [ozone] errose28 opened a new pull request #1645: HDDS-4463. Verify that no node can start when Software Layout Version < Metadata Layout Version.

2020-12-01 Thread GitBox
errose28 opened a new pull request #1645: URL: https://github.com/apache/ozone/pull/1645 ## What changes were proposed in this pull request? Add tests for OzoneManager, StorageContainerManager, and DatanodeStateMachine classes that will verify an Exception is thrown on startup when

[jira] [Commented] (HDDS-4534) ConcurrentModificationException in ContainerReportHandler

2020-12-01 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17241599#comment-17241599 ] Wei-Chiu Chuang commented on HDDS-4534: --- Like I reported in HDDS-3918, the issue was

[jira] [Updated] (HDDS-4528) Upgrade slf4j to 1.7.30

2020-12-01 Thread Kevin Risden (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Risden updated HDDS-4528: --- Fix Version/s: 1.1.0 Resolution: Fixed Status: Resolved (was: Patch Available) Than

[GitHub] [ozone] runzhiwang commented on a change in pull request #1643: HDDS-4447. SCMBlockDeletingService should handle ContainerNotFoundExc…

2020-12-01 Thread GitBox
runzhiwang commented on a change in pull request #1643: URL: https://github.com/apache/ozone/pull/1643#discussion_r533395644 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/block/DeletedBlockLogImpl.java ## @@ -354,26 +356,43 @@ public DatanodeDel

[GitHub] [ozone] runzhiwang commented on pull request #1644: HDDS-4535. Use fixed thread pool for closed container replication

2020-12-01 Thread GitBox
runzhiwang commented on pull request #1644: URL: https://github.com/apache/ozone/pull/1644#issuecomment-736541168 > So if queue is not full (and LinkedBlockgingQueue is unbounded by default) the threads will never be created. @elek Please correct me if I am wrong. If the queue is not

[jira] [Resolved] (HDDS-4485) Modification of the operating introduction of ozone ranger.

2020-12-01 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang resolved HDDS-4485. -- Resolution: Fixed > Modification of the operating introduction of ozone ranger. > --

[GitHub] [ozone] runzhiwang merged pull request #1603: HDDS-4485. [DOC] add the authentication rules of the Ozone Ranger.

2020-12-01 Thread GitBox
runzhiwang merged pull request #1603: URL: https://github.com/apache/ozone/pull/1603 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [ozone] runzhiwang commented on pull request #1603: HDDS-4485. [DOC] add the authentication rules of the Ozone Ranger.

2020-12-01 Thread GitBox
runzhiwang commented on pull request #1603: URL: https://github.com/apache/ozone/pull/1603#issuecomment-736538410 @captainzmc Thanks the patch. @cku328 @vivekratnavel Thanks for review. I have merged the patch This is an aut

[jira] [Updated] (HDDS-4535) Use fixed thread pool for closed container replication

2020-12-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-4535: - Labels: pull-request-available (was: ) > Use fixed thread pool for closed container replication

[GitHub] [ozone] elek opened a new pull request #1644: HDDS-4535. Use fixed thread pool for closed container replication

2020-12-01 Thread GitBox
elek opened a new pull request #1644: URL: https://github.com/apache/ozone/pull/1644 ## What changes were proposed in this pull request? Number of threads for closed container replications can be adjusted by the settings `hdds.datanode.replication.streams.limit`. But this number is

[jira] [Created] (HDDS-4535) Use fixed thread pool for closed container replication

2020-12-01 Thread Marton Elek (Jira)
Marton Elek created HDDS-4535: - Summary: Use fixed thread pool for closed container replication Key: HDDS-4535 URL: https://issues.apache.org/jira/browse/HDDS-4535 Project: Hadoop Distributed Data Store

[GitHub] [ozone] timmylicheng commented on pull request #1399: HDDS-3684. Add tests for replication annotation

2020-12-01 Thread GitBox
timmylicheng commented on pull request #1399: URL: https://github.com/apache/ozone/pull/1399#issuecomment-736489822 @amaliujia Looks like we have new conflicts. Could you please take another look? This is an automated messag

[jira] [Updated] (HDDS-4191) Add failover proxy for SCM container client

2020-12-01 Thread Li Cheng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Cheng updated HDDS-4191: --- Resolution: Fixed Status: Resolved (was: Patch Available) > Add failover proxy for SCM container clie

[jira] [Commented] (HDDS-4191) Add failover proxy for SCM container client

2020-12-01 Thread Li Cheng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17241466#comment-17241466 ] Li Cheng commented on HDDS-4191: PR is merged. Resolving > Add failover proxy for SCM con

[GitHub] [ozone] sodonnel commented on pull request #1580: HDDS-1880. Decommission and Maintenance mode in Ozone

2020-12-01 Thread GitBox
sodonnel commented on pull request #1580: URL: https://github.com/apache/ozone/pull/1580#issuecomment-736482735 It seems like the test failures were caused by HDDS-4453 and fixed in HDDS-4429. I have rebased to include HDDS-4429. Once I get a few full CI runs without failure we should be g

[GitHub] [ozone] ChenSammi merged pull request #1514: HDDS-4191 Add failover proxy for SCM container location.

2020-12-01 Thread GitBox
ChenSammi merged pull request #1514: URL: https://github.com/apache/ozone/pull/1514 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [ozone] ChenSammi commented on pull request #1514: HDDS-4191 Add failover proxy for SCM container location.

2020-12-01 Thread GitBox
ChenSammi commented on pull request #1514: URL: https://github.com/apache/ozone/pull/1514#issuecomment-736473202 LGTM + 1. This is an automated message from the Apache Git Service. To respond to the message, please lo

[jira] [Commented] (HDDS-4533) Avoid rewriting pipeline information during PipelineStateManagerV2Impl initialization

2020-12-01 Thread Glen Geng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17241454#comment-17241454 ] Glen Geng commented on HDDS-4533: - OM side notice this problem too: HDDS-4484 > Avoid rew

[GitHub] [ozone] sodonnel commented on a change in pull request #1610: HDDS-4473. Reduce number of sortDatanodes RPC calls

2020-12-01 Thread GitBox
sodonnel commented on a change in pull request #1610: URL: https://github.com/apache/ozone/pull/1610#discussion_r53586 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java ## @@ -2322,21 +2322,22 @@ void sortDatanodes(String

[GitHub] [ozone] adoroszlai commented on a change in pull request #1610: HDDS-4473. Reduce number of sortDatanodes RPC calls

2020-12-01 Thread GitBox
adoroszlai commented on a change in pull request #1610: URL: https://github.com/apache/ozone/pull/1610#discussion_r533323808 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java ## @@ -2322,21 +2322,22 @@ void sortDatanodes(Stri

[GitHub] [ozone] sodonnel commented on pull request #1610: HDDS-4473. Reduce number of sortDatanodes RPC calls

2020-12-01 Thread GitBox
sodonnel commented on pull request #1610: URL: https://github.com/apache/ozone/pull/1610#issuecomment-736393901 > we should use the set of node IDs, instead of the pipeline ID, as cache key. Good idea - I think that will work much better. I checked the latest version and it looks go

[GitHub] [ozone] sodonnel commented on a change in pull request #1610: HDDS-4473. Reduce number of sortDatanodes RPC calls

2020-12-01 Thread GitBox
sodonnel commented on a change in pull request #1610: URL: https://github.com/apache/ozone/pull/1610#discussion_r533310687 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java ## @@ -2322,21 +2322,22 @@ void sortDatanodes(String

[jira] [Updated] (HDDS-4447) SCMBlockDeletingService should handle ContainerNotFoundException

2020-12-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-4447: - Labels: pull-request-available (was: ) > SCMBlockDeletingService should handle ContainerNotFoundE

[GitHub] [ozone] aryangupta1998 opened a new pull request #1643: HDDS-4447. SCMBlockDeletingService should handle ContainerNotFoundExc…

2020-12-01 Thread GitBox
aryangupta1998 opened a new pull request #1643: URL: https://github.com/apache/ozone/pull/1643 …eption. ## What changes were proposed in this pull request? SCMBlockDeletingService terminates task on receiving ContainerNotFoundException. As a result, SCM stops deleting blocks.

[jira] [Commented] (HDDS-4528) Upgrade slf4j to 1.7.30

2020-12-01 Thread Marton Elek (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17241395#comment-17241395 ] Marton Elek commented on HDDS-4528: --- Thanks the contribution [~krisden]. I added you to

[jira] [Assigned] (HDDS-4528) Upgrade slf4j to 1.7.30

2020-12-01 Thread Marton Elek (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Elek reassigned HDDS-4528: - Assignee: Kevin Risden > Upgrade slf4j to 1.7.30 > --- > > Ke

[GitHub] [ozone] adoroszlai commented on pull request #1523: HDDS-4320. Let Ozone input streams implement CanUnbuffer

2020-12-01 Thread GitBox
adoroszlai commented on pull request #1523: URL: https://github.com/apache/ozone/pull/1523#issuecomment-736342125 Thanks @hanishakoneru for the review. I have updated some names and docs according to your comments. This is

[GitHub] [ozone] adoroszlai commented on a change in pull request #1523: HDDS-4320. Let Ozone input streams implement CanUnbuffer

2020-12-01 Thread GitBox
adoroszlai commented on a change in pull request #1523: URL: https://github.com/apache/ozone/pull/1523#discussion_r533191771 ## File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/contract/AbstractContractUnbufferTest.java ## @@ -0,0 +1,159 @@ +/* + * L

[GitHub] [ozone] runzhiwang commented on pull request #1601: HDDS-4404. Datanode can go OOM when a Recon or SCM Server is very slow in processing reports

2020-12-01 Thread GitBox
runzhiwang commented on pull request #1601: URL: https://github.com/apache/ozone/pull/1601#issuecomment-736338966 Waiting CI pass. This is an automated message from the Apache Git Service. To respond to the message, please lo

[GitHub] [ozone] adoroszlai commented on a change in pull request #1523: HDDS-4320. Let Ozone input streams implement CanUnbuffer

2020-12-01 Thread GitBox
adoroszlai commented on a change in pull request #1523: URL: https://github.com/apache/ozone/pull/1523#discussion_r533191712 ## File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/contract/AbstractContractUnbufferTest.java ## @@ -0,0 +1,159 @@ +/* + * L

[GitHub] [ozone] adoroszlai commented on a change in pull request #1523: HDDS-4320. Let Ozone input streams implement CanUnbuffer

2020-12-01 Thread GitBox
adoroszlai commented on a change in pull request #1523: URL: https://github.com/apache/ozone/pull/1523#discussion_r533181203 ## File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestKeyInputStream.java ## @@ -397,4 +410,118 @@ public voi

[jira] [Commented] (HDDS-4533) Avoid rewriting pipeline information during PipelineStateManagerV2Impl initialization

2020-12-01 Thread Glen Geng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17241358#comment-17241358 ] Glen Geng commented on HDDS-4533: - RaftClientRequest can serve both read request and write

[GitHub] [ozone] elek merged pull request #1639: HDDS-4528. Upgrade slf4j to 1.7.30

2020-12-01 Thread GitBox
elek merged pull request #1639: URL: https://github.com/apache/ozone/pull/1639 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[GitHub] [ozone] adoroszlai commented on a change in pull request #1523: HDDS-4320. Let Ozone input streams implement CanUnbuffer

2020-12-01 Thread GitBox
adoroszlai commented on a change in pull request #1523: URL: https://github.com/apache/ozone/pull/1523#discussion_r533171908 ## File path: hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockInputStream.java ## @@ -171,6 +162,23 @@ public synchronized void

[GitHub] [ozone] ChenSammi commented on pull request #1642: HDDS-4519. Return forbidden instead of interval server error from s3g…

2020-12-01 Thread GitBox
ChenSammi commented on pull request #1642: URL: https://github.com/apache/ozone/pull/1642#issuecomment-736321751 Hi, @bharatviswa504 and @elek, would you help to review the patch at your convenient time? This is an automat

[jira] [Updated] (HDDS-4519) Return forbidden instead of interval server error from s3g when user doesn't have the resouce permission

2020-12-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-4519: - Labels: pull-request-available (was: ) > Return forbidden instead of interval server error from s

[GitHub] [ozone] ChenSammi opened a new pull request #1642: HDDS-4519. Return forbidden instead of interval server error from s3g…

2020-12-01 Thread GitBox
ChenSammi opened a new pull request #1642: URL: https://github.com/apache/ozone/pull/1642 https://issues.apache.org/jira/browse/HDDS-4519 Currently s3g will return "internal server error" when user doesn't have the permission to access the resource. This task is to return HTT

[GitHub] [ozone] elek merged pull request #1548: HDDS-4424. Update README with information how to report security issues

2020-12-01 Thread GitBox
elek merged pull request #1548: URL: https://github.com/apache/ozone/pull/1548 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

  1   2   >