[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #1169: HDDS-3930. Fix OMKeyDeletesRequest.

2020-07-07 Thread GitBox
bharatviswa504 commented on a change in pull request #1169: URL: https://github.com/apache/hadoop-ozone/pull/1169#discussion_r451283029 ## File path: hadoop-ozone/interface-client/src/main/proto/OmClientProtocol.proto ## @@ -867,10 +867,10 @@ message DeletedKeys { }

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #1169: HDDS-3930. Fix OMKeyDeletesRequest.

2020-07-07 Thread GitBox
bharatviswa504 commented on a change in pull request #1169: URL: https://github.com/apache/hadoop-ozone/pull/1169#discussion_r451283029 ## File path: hadoop-ozone/interface-client/src/main/proto/OmClientProtocol.proto ## @@ -867,10 +867,10 @@ message DeletedKeys { }

[GitHub] [hadoop-ozone] smengcl commented on a change in pull request #1169: HDDS-3930. Fix OMKeyDeletesRequest.

2020-07-07 Thread GitBox
smengcl commented on a change in pull request #1169: URL: https://github.com/apache/hadoop-ozone/pull/1169#discussion_r451240368 ## File path: hadoop-ozone/interface-client/src/main/proto/OmClientProtocol.proto ## @@ -867,10 +867,10 @@ message DeletedKeys { } message

[GitHub] [hadoop-ozone] cku328 commented on pull request #1171: HDDS-3932. Hide jOOQ logo message from the log output on compile

2020-07-07 Thread GitBox
cku328 commented on pull request #1171: URL: https://github.com/apache/hadoop-ozone/pull/1171#issuecomment-655265688 @avijayanhwx Okay, thanks for the review. I'll retrigger CI check. This is an automated message from

[jira] [Updated] (HDDS-3933) Memory leak because of too many Datanode State Machine Thread

2020-07-07 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-3933: - Attachment: jstack.txt > Memory leak because of too many Datanode State Machine Thread >

[jira] [Updated] (HDDS-3933) Memory leak because of too many Datanode State Machine Thread

2020-07-07 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-3933: - Description: When create 22345th Datanode State Machine Thread, OOM happened. !screenshot-1.png!

[jira] [Updated] (HDDS-3933) Memory leak because of too many Datanode State Machine Thread

2020-07-07 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-3933: - Attachment: screenshot-3.png > Memory leak because of too many Datanode State Machine Thread >

[GitHub] [hadoop-ozone] GlenGeng commented on a change in pull request #1151: HDDS-3191: switch from SCMPipelineManager to PipelineManagerV2Impl

2020-07-07 Thread GitBox
GlenGeng commented on a change in pull request #1151: URL: https://github.com/apache/hadoop-ozone/pull/1151#discussion_r451254135 ## File path: hadoop-ozone/recon/pom.xml ## @@ -108,6 +108,7 @@ pnpm config set store-dir ~/.pnpm-store

[jira] [Assigned] (HDDS-3922) Display the pipeline info on scm web page

2020-07-07 Thread lihanran (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lihanran reassigned HDDS-3922: -- Assignee: lihanran > Display the pipeline info on scm web page >

[GitHub] [hadoop-ozone] iamabug commented on a change in pull request #1175: HDDS-2766. security/SecuringDataNodes.md

2020-07-07 Thread GitBox
iamabug commented on a change in pull request #1175: URL: https://github.com/apache/hadoop-ozone/pull/1175#discussion_r451238736 ## File path: hadoop-hdds/docs/content/security/SecuringDatanodes.zh.md ## @@ -0,0 +1,53 @@ +--- +title: "安全化 Datanode" +date: "2019-April-03"

[GitHub] [hadoop-ozone] iamabug commented on a change in pull request #1175: HDDS-2766. security/SecuringDataNodes.md

2020-07-07 Thread GitBox
iamabug commented on a change in pull request #1175: URL: https://github.com/apache/hadoop-ozone/pull/1175#discussion_r451237935 ## File path: hadoop-hdds/docs/content/security/SecuringDatanodes.zh.md ## @@ -0,0 +1,53 @@ +--- +title: "安全化 Datanode" +date: "2019-April-03"

[GitHub] [hadoop-ozone] iamabug commented on a change in pull request #1175: HDDS-2766. security/SecuringDataNodes.md

2020-07-07 Thread GitBox
iamabug commented on a change in pull request #1175: URL: https://github.com/apache/hadoop-ozone/pull/1175#discussion_r451237218 ## File path: hadoop-hdds/docs/content/security/SecuringDatanodes.zh.md ## @@ -0,0 +1,53 @@ +--- +title: "安全化 Datanode" +date: "2019-April-03"

[jira] [Updated] (HDDS-2766) security/SecuringDataNodes.md

2020-07-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-2766: - Labels: pull-request-available (was: ) > security/SecuringDataNodes.md >

[GitHub] [hadoop-ozone] iamabug opened a new pull request #1175: HDDS-2766. security/SecuringDataNodes.md

2020-07-07 Thread GitBox
iamabug opened a new pull request #1175: URL: https://github.com/apache/hadoop-ozone/pull/1175 ## What changes were proposed in this pull request? translation to doc https://hadoop.apache.org/ozone/docs/0.5.0-beta/security/securingdatanodes.html ## What is the link to the

[jira] [Comment Edited] (HDDS-3918) ConcurrentModificationException in ContainerReportHandler.onMessage

2020-07-07 Thread Xiaoyu Yao (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17153148#comment-17153148 ] Xiaoyu Yao edited comment on HDDS-3918 at 7/8/20, 1:16 AM: --- We have a race

[jira] [Updated] (HDDS-3918) ConcurrentModificationException in ContainerReportHandler.onMessage

2020-07-07 Thread Xiaoyu Yao (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HDDS-3918: - Attachment: TestCME.java > ConcurrentModificationException in ContainerReportHandler.onMessage >

[jira] [Updated] (HDDS-3918) ConcurrentModificationException in ContainerReportHandler.onMessage

2020-07-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-3918: - Labels: pull-request-available (was: ) > ConcurrentModificationException in

[GitHub] [hadoop-ozone] xiaoyuyao opened a new pull request #1174: HDDS-3918. ConcurrentModificationException in ContainerReportHandler.…

2020-07-07 Thread GitBox
xiaoyuyao opened a new pull request #1174: URL: https://github.com/apache/hadoop-ozone/pull/1174 …onMessage. ## What changes were proposed in this pull request? Use thread safe HashSet for NodeStateMap#nodeToContainer map ## What is the link to the Apache JIRA

[jira] [Commented] (HDDS-3918) ConcurrentModificationException in ContainerReportHandler.onMessage

2020-07-07 Thread Xiaoyu Yao (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17153148#comment-17153148 ] Xiaoyu Yao commented on HDDS-3918: -- We have a race condition here on the container set of the

[jira] [Assigned] (HDDS-3853) Container marked as missing on datanode while container directory do exist

2020-07-07 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang reassigned HDDS-3853: Assignee: runzhiwang (was: Shashikant Banerjee) > Container marked as missing on datanode while

[jira] [Updated] (HDDS-3705) [OFS] Implement getTrashRoots for trash cleanup

2020-07-07 Thread Siyao Meng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siyao Meng updated HDDS-3705: - Target Version/s: 0.6.0 (was: 0.7.0) Resolution: Fixed Status: Resolved (was:

[GitHub] [hadoop-ozone] smengcl commented on pull request #1089: HDDS-3705. [OFS] Implement getTrashRoots for trash cleanup

2020-07-07 Thread GitBox
smengcl commented on pull request #1089: URL: https://github.com/apache/hadoop-ozone/pull/1089#issuecomment-655205691 Thanks @xiaoyuyao for the review. Will merge shortly. This is an automated message from the Apache Git

[GitHub] [hadoop-ozone] smengcl merged pull request #1089: HDDS-3705. [OFS] Implement getTrashRoots for trash cleanup

2020-07-07 Thread GitBox
smengcl merged pull request #1089: URL: https://github.com/apache/hadoop-ozone/pull/1089 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #1164: HDDS-3824: OM read requests should make SCM#refreshPipeline outside BUCKET_LOCK

2020-07-07 Thread GitBox
bharatviswa504 commented on a change in pull request #1164: URL: https://github.com/apache/hadoop-ozone/pull/1164#discussion_r451209121 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java ## @@ -659,15 +660,6 @@ public

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1151: HDDS-3191: switch from SCMPipelineManager to PipelineManagerV2Impl

2020-07-07 Thread GitBox
xiaoyuyao commented on a change in pull request #1151: URL: https://github.com/apache/hadoop-ozone/pull/1151#discussion_r451155274 ## File path: hadoop-ozone/recon/pom.xml ## @@ -108,6 +108,7 @@ pnpm config set store-dir ~/.pnpm-store

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1151: HDDS-3191: switch from SCMPipelineManager to PipelineManagerV2Impl

2020-07-07 Thread GitBox
xiaoyuyao commented on a change in pull request #1151: URL: https://github.com/apache/hadoop-ozone/pull/1151#discussion_r451149178 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/ha/SCMHAConfiguration.java ## @@ -78,7 +78,7 @@ description

[GitHub] [hadoop-ozone] xiaoyuyao commented on pull request #1089: HDDS-3705. [OFS] Implement getTrashRoots for trash cleanup

2020-07-07 Thread GitBox
xiaoyuyao commented on pull request #1089: URL: https://github.com/apache/hadoop-ozone/pull/1089#issuecomment-655139139 LGTM, +1 pending CI. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [hadoop-ozone] sonarcloud[bot] commented on pull request #1169: HDDS-3930. Fix OMKeyDeletesRequest.

2020-07-07 Thread GitBox
sonarcloud[bot] commented on pull request #1169: URL: https://github.com/apache/hadoop-ozone/pull/1169#issuecomment-655133377 SonarCloud Quality Gate failed. [](https://sonarcloud.io/project/issues?id=hadoop-ozone=1169=false=BUG)

[GitHub] [hadoop-ozone] arp7 commented on pull request #1149: HDDS-3878. Make OMHA serviceID optional if one (but only one) is defined in the config

2020-07-07 Thread GitBox
arp7 commented on pull request #1149: URL: https://github.com/apache/hadoop-ozone/pull/1149#issuecomment-655129013 Yeah this was a conscious choice. I feel usage of unqualified paths in HDFS can be ambiguous and error-prone when multiple clusters/federation are involved.

[jira] [Commented] (HDDS-3874) ITestRootedOzoneContract tests are flaky

2020-07-07 Thread Siyao Meng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17153089#comment-17153089 ] Siyao Meng commented on HDDS-3874: -- [~elek] I recall seeing once or twice {{it-filesystem-contract}}

[GitHub] [hadoop-ozone] smengcl commented on a change in pull request #1089: HDDS-3705. [OFS] Implement getTrashRoots for trash cleanup

2020-07-07 Thread GitBox
smengcl commented on a change in pull request #1089: URL: https://github.com/apache/hadoop-ozone/pull/1089#discussion_r451129142 ## File path: hadoop-ozone/ozonefs-common/src/main/java/org/apache/hadoop/fs/ozone/BasicRootedOzoneClientAdapterImpl.java ## @@ -516,6 +520,62 @@

[GitHub] [hadoop-ozone] smengcl commented on a change in pull request #1089: HDDS-3705. [OFS] Implement getTrashRoots for trash cleanup

2020-07-07 Thread GitBox
smengcl commented on a change in pull request #1089: URL: https://github.com/apache/hadoop-ozone/pull/1089#discussion_r451126852 ## File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestRootedOzoneFileSystem.java ## @@ -871,4 +876,103 @@ public

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1164: HDDS-3824: OM read requests should make SCM#refreshPipeline outside BUCKET_LOCK

2020-07-07 Thread GitBox
xiaoyuyao commented on a change in pull request #1164: URL: https://github.com/apache/hadoop-ozone/pull/1164#discussion_r451122204 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java ## @@ -1706,36 +1718,41 @@ public

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1164: HDDS-3824: OM read requests should make SCM#refreshPipeline outside BUCKET_LOCK

2020-07-07 Thread GitBox
xiaoyuyao commented on a change in pull request #1164: URL: https://github.com/apache/hadoop-ozone/pull/1164#discussion_r451122204 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java ## @@ -1706,36 +1718,41 @@ public

[GitHub] [hadoop-ozone] bharatviswa504 edited a comment on pull request #1149: HDDS-3878. Make OMHA serviceID optional if one (but only one) is defined in the config

2020-07-07 Thread GitBox
bharatviswa504 edited a comment on pull request #1149: URL: https://github.com/apache/hadoop-ozone/pull/1149#issuecomment-655106253 I think this is a deliberate choice not to pick from config, if it is there. As hive stores the table path in its metastore. So, having a complete path is

[GitHub] [hadoop-ozone] bharatviswa504 edited a comment on pull request #1149: HDDS-3878. Make OMHA serviceID optional if one (but only one) is defined in the config

2020-07-07 Thread GitBox
bharatviswa504 edited a comment on pull request #1149: URL: https://github.com/apache/hadoop-ozone/pull/1149#issuecomment-655106253 I think this is a deliberate choice not to pick from config, if there, as hive stores the table path in its metastore. So, having a complete path is better.

[GitHub] [hadoop-ozone] bharatviswa504 commented on pull request #1149: HDDS-3878. Make OMHA serviceID optional if one (but only one) is defined in the config

2020-07-07 Thread GitBox
bharatviswa504 commented on pull request #1149: URL: https://github.com/apache/hadoop-ozone/pull/1149#issuecomment-655106253 I think this is a deliberate choice not to pick from config, if there, as hive stores the table path in its metastore. So, having a complete path is better.

[jira] [Updated] (HDDS-3878) Make OMHA serviceID optional if one (but only one) is defined in the config

2020-07-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-3878: - Labels: pull-request-available (was: ) > Make OMHA serviceID optional if one (but only one) is

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1089: HDDS-3705. [OFS] Implement getTrashRoots for trash cleanup

2020-07-07 Thread GitBox
xiaoyuyao commented on a change in pull request #1089: URL: https://github.com/apache/hadoop-ozone/pull/1089#discussion_r451112711 ## File path: hadoop-ozone/ozonefs-common/src/main/java/org/apache/hadoop/fs/ozone/BasicRootedOzoneClientAdapterImpl.java ## @@ -516,6 +520,62 @@

[GitHub] [hadoop-ozone] smengcl commented on a change in pull request #1089: HDDS-3705. [OFS] Implement getTrashRoots for trash cleanup

2020-07-07 Thread GitBox
smengcl commented on a change in pull request #1089: URL: https://github.com/apache/hadoop-ozone/pull/1089#discussion_r451113467 ## File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestRootedOzoneFileSystem.java ## @@ -871,4 +876,103 @@ public

[jira] [Updated] (HDDS-3931) Maven warning due to deprecated expression pom.artifactId

2020-07-07 Thread Bharat Viswanadham (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharat Viswanadham updated HDDS-3931: - Fix Version/s: 0.6.0 Resolution: Fixed Status: Resolved (was: Patch

[GitHub] [hadoop-ozone] bharatviswa504 commented on pull request #1172: HDDS-3931. Maven warning due to deprecated expression pom.artifactId

2020-07-07 Thread GitBox
bharatviswa504 commented on pull request #1172: URL: https://github.com/apache/hadoop-ozone/pull/1172#issuecomment-655095351 Thank You @adoroszlai for the contribution. This is an automated message from the Apache Git

[GitHub] [hadoop-ozone] bharatviswa504 merged pull request #1172: HDDS-3931. Maven warning due to deprecated expression pom.artifactId

2020-07-07 Thread GitBox
bharatviswa504 merged pull request #1172: URL: https://github.com/apache/hadoop-ozone/pull/1172 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1089: HDDS-3705. [OFS] Implement getTrashRoots for trash cleanup

2020-07-07 Thread GitBox
xiaoyuyao commented on a change in pull request #1089: URL: https://github.com/apache/hadoop-ozone/pull/1089#discussion_r451103682 ## File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestRootedOzoneFileSystem.java ## @@ -871,4 +876,103 @@

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1089: HDDS-3705. [OFS] Implement getTrashRoots for trash cleanup

2020-07-07 Thread GitBox
xiaoyuyao commented on a change in pull request #1089: URL: https://github.com/apache/hadoop-ozone/pull/1089#discussion_r451103207 ## File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestRootedOzoneFileSystem.java ## @@ -871,4 +876,103 @@

[GitHub] [hadoop-ozone] codecov-commenter commented on pull request #1089: HDDS-3705. [OFS] Implement getTrashRoots for trash cleanup

2020-07-07 Thread GitBox
codecov-commenter commented on pull request #1089: URL: https://github.com/apache/hadoop-ozone/pull/1089#issuecomment-655086823 # [Codecov](https://codecov.io/gh/apache/hadoop-ozone/pull/1089?src=pr=h1) Report > Merging

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1089: HDDS-3705. [OFS] Implement getTrashRoots for trash cleanup

2020-07-07 Thread GitBox
xiaoyuyao commented on a change in pull request #1089: URL: https://github.com/apache/hadoop-ozone/pull/1089#discussion_r451102321 ## File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestRootedOzoneFileSystem.java ## @@ -871,4 +876,103 @@

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1166: HDDS-3914. Remove LevelDB configuration option for DN Metastore

2020-07-07 Thread GitBox
xiaoyuyao commented on a change in pull request #1166: URL: https://github.com/apache/hadoop-ozone/pull/1166#discussion_r451048561 ## File path: hadoop-hdds/framework/src/test/java/org/apache/hadoop/hdds/utils/TestMetadataStore.java ## @@ -166,12 +170,12 @@ public void

[jira] [Updated] (HDDS-3880) Improve OM HA Robot tests

2020-07-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-3880: - Labels: pull-request-available (was: ) > Improve OM HA Robot tests > - >

[GitHub] [hadoop-ozone] hanishakoneru opened a new pull request #1173: HDDS-3880. Improve OM HA Robot test

2020-07-07 Thread GitBox
hanishakoneru opened a new pull request #1173: URL: https://github.com/apache/hadoop-ozone/pull/1173 ## What changes were proposed in this pull request? This Jira aims to address the following: 1. Add robot test for Install Snapshot feature 2. Fix the flakiness in OM HA robot

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1049: HDDS-3662 Decouple finalizeAndDestroyPipeline.

2020-07-07 Thread GitBox
xiaoyuyao commented on a change in pull request #1049: URL: https://github.com/apache/hadoop-ozone/pull/1049#discussion_r451023617 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/PipelineManagerV2Impl.java ## @@ -410,18 +399,29 @@ public

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1049: HDDS-3662 Decouple finalizeAndDestroyPipeline.

2020-07-07 Thread GitBox
xiaoyuyao commented on a change in pull request #1049: URL: https://github.com/apache/hadoop-ozone/pull/1049#discussion_r451024886 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/SCMPipelineManager.java ## @@ -421,18 +438,29 @@ public

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1049: HDDS-3662 Decouple finalizeAndDestroyPipeline.

2020-07-07 Thread GitBox
xiaoyuyao commented on a change in pull request #1049: URL: https://github.com/apache/hadoop-ozone/pull/1049#discussion_r451024282 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/PipelineManagerV2Impl.java ## @@ -310,94 +321,72 @@ public

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1049: HDDS-3662 Decouple finalizeAndDestroyPipeline.

2020-07-07 Thread GitBox
xiaoyuyao commented on a change in pull request #1049: URL: https://github.com/apache/hadoop-ozone/pull/1049#discussion_r451023617 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/PipelineManagerV2Impl.java ## @@ -410,18 +399,29 @@ public

[jira] [Updated] (HDDS-3509) Closing container with unhealthy replica on open pipeline

2020-07-07 Thread Xiaoyu Yao (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HDDS-3509: - Labels: (was: TriagePending) > Closing container with unhealthy replica on open pipeline >

[GitHub] [hadoop-ozone] smengcl commented on pull request #1089: HDDS-3705. [OFS] Implement getTrashRoots for trash cleanup

2020-07-07 Thread GitBox
smengcl commented on pull request #1089: URL: https://github.com/apache/hadoop-ozone/pull/1089#issuecomment-654981393 The new test contaminated the results of other tests in the same test class. Fixing this. This is an

[GitHub] [hadoop-ozone] avijayanhwx commented on pull request #1156: HDDS-3879. Introduce SCM and OM layoutVersion zero to the VERSION file

2020-07-07 Thread GitBox
avijayanhwx commented on pull request #1156: URL: https://github.com/apache/hadoop-ozone/pull/1156#issuecomment-654968796 > I started on the changes to add in the software version too, but in doing that I starting to think about whether we need to actually store it. > > I will

[jira] [Resolved] (HDDS-3910) JooqCodeGenerator interrupted but still alive

2020-07-07 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai resolved HDDS-3910. Fix Version/s: 0.6.0 Target Version/s: (was: 0.7.0) Resolution: Fixed >

[jira] [Updated] (HDDS-3910) JooqCodeGenerator interrupted but still alive

2020-07-07 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-3910: --- Labels: (was: pull-request-available) > JooqCodeGenerator interrupted but still alive >

[GitHub] [hadoop-ozone] adoroszlai merged pull request #1170: HDDS-3910. JooqCodeGenerator interrupted but still alive

2020-07-07 Thread GitBox
adoroszlai merged pull request #1170: URL: https://github.com/apache/hadoop-ozone/pull/1170 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hadoop-ozone] adoroszlai commented on pull request #1170: HDDS-3910. JooqCodeGenerator interrupted but still alive

2020-07-07 Thread GitBox
adoroszlai commented on pull request #1170: URL: https://github.com/apache/hadoop-ozone/pull/1170#issuecomment-654966338 Thanks @avijayanhwx for the review. This is an automated message from the Apache Git Service. To

[GitHub] [hadoop-ozone] avijayanhwx commented on pull request #1170: HDDS-3910. JooqCodeGenerator interrupted but still alive

2020-07-07 Thread GitBox
avijayanhwx commented on pull request #1170: URL: https://github.com/apache/hadoop-ozone/pull/1170#issuecomment-654963861 Thanks for fixing this @cku328. LGTM +1 This is an automated message from the Apache Git Service. To

[jira] [Assigned] (HDDS-3927) Add OZONE_MANAGER_OPTS and OZONE_DATANODE_OPTS

2020-07-07 Thread Siyao Meng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siyao Meng reassigned HDDS-3927: Assignee: Siyao Meng > Add OZONE_MANAGER_OPTS and OZONE_DATANODE_OPTS >

[jira] [Updated] (HDDS-3932) Hide jOOQ logo message from the log output on compile

2020-07-07 Thread Jitendra Nath Pandey (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jitendra Nath Pandey updated HDDS-3932: --- Fix Version/s: (was: 0.6.0) Target Version/s: 0.7.0 (was: 0.6.0) > Hide

[GitHub] [hadoop-ozone] adoroszlai commented on pull request #1170: HDDS-3910. JooqCodeGenerator interrupted but still alive

2020-07-07 Thread GitBox
adoroszlai commented on pull request #1170: URL: https://github.com/apache/hadoop-ozone/pull/1170#issuecomment-654862931 @avijayanhwx please let us know if you have any concerns about this change, otherwise I would like to merge it.

[GitHub] [hadoop-ozone] sodonnel commented on pull request #1162: HDDS-3921. IllegalArgumentException triggered in SCMContainerPlacemen…

2020-07-07 Thread GitBox
sodonnel commented on pull request #1162: URL: https://github.com/apache/hadoop-ozone/pull/1162#issuecomment-654823654 Thanks for this change. I also wonder if there is a bug in the method `isContainerUnderReplicated(...)` which leads to this problem, and results in more processing that

[GitHub] [hadoop-ozone] sodonnel commented on a change in pull request #1162: HDDS-3921. IllegalArgumentException triggered in SCMContainerPlacemen…

2020-07-07 Thread GitBox
sodonnel commented on a change in pull request #1162: URL: https://github.com/apache/hadoop-ozone/pull/1162#discussion_r450820397 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ReplicationManager.java ## @@ -543,11 +543,15 @@ private

[GitHub] [hadoop-ozone] sodonnel commented on a change in pull request #1162: HDDS-3921. IllegalArgumentException triggered in SCMContainerPlacemen…

2020-07-07 Thread GitBox
sodonnel commented on a change in pull request #1162: URL: https://github.com/apache/hadoop-ozone/pull/1162#discussion_r450820397 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ReplicationManager.java ## @@ -543,11 +543,15 @@ private

[GitHub] [hadoop-ozone] sodonnel commented on a change in pull request #1162: HDDS-3921. IllegalArgumentException triggered in SCMContainerPlacemen…

2020-07-07 Thread GitBox
sodonnel commented on a change in pull request #1162: URL: https://github.com/apache/hadoop-ozone/pull/1162#discussion_r450820397 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ReplicationManager.java ## @@ -543,11 +543,15 @@ private

[jira] [Updated] (HDDS-3582) Update Checkstyle rule

2020-07-07 Thread maobaolong (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maobaolong updated HDDS-3582: - Description: Now. the rules of checkstyle is hadoop style, as we all know, hadoop repo started 10 years

[GitHub] [hadoop-ozone] sodonnel commented on pull request #1156: HDDS-3879. Introduce SCM and OM layoutVersion zero to the VERSION file

2020-07-07 Thread GitBox
sodonnel commented on pull request #1156: URL: https://github.com/apache/hadoop-ozone/pull/1156#issuecomment-654791797 I started on the changes to add in the software version too, but in doing that I starting to think about whether we need to actually store it. I will discuss with

[jira] [Commented] (HDDS-3582) Update Checkstyle rule

2020-07-07 Thread maobaolong (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152669#comment-17152669 ] maobaolong commented on HDDS-3582: -- !image-2020-07-07-19-25-59-658.png!   There are another strange

[jira] [Updated] (HDDS-3582) Update Checkstyle rule

2020-07-07 Thread maobaolong (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maobaolong updated HDDS-3582: - Attachment: image-2020-07-07-19-25-59-658.png > Update Checkstyle rule > -- > >

[jira] [Commented] (HDDS-3922) Display the pipeline info on scm web page

2020-07-07 Thread maobaolong (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152665#comment-17152665 ] maobaolong commented on HDDS-3922: -- [~avijayan] I also agree collect all information into Recon is the

[jira] [Commented] (HDDS-3834) We need a edge(master) docs website updated by nightly build

2020-07-07 Thread Marton Elek (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152663#comment-17152663 ] Marton Elek commented on HDDS-3834: --- No idea. INFRA or build@a.o should be asked. 1. We either need a

[jira] [Updated] (HDDS-3933) Memory leak because of too many Datanode State Machine Thread

2020-07-07 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-3933: - Description: When create 22345th Datanode State Machine Thread, OOM happened. !screenshot-1.png!

[jira] [Updated] (HDDS-3933) Memory leak because of too many Datanode State Machine Thread

2020-07-07 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-3933: - Attachment: screenshot-2.png > Memory leak because of too many Datanode State Machine Thread >

[jira] [Updated] (HDDS-3933) Memory leak because of too many Datanode State Machine Thread

2020-07-07 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-3933: - Description: When create 22345th Datanode State Machine Thread, OOM happened. !screenshot-1.png! >

[jira] [Updated] (HDDS-3933) Memory leak because of too many Datanode State Machine Thread

2020-07-07 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-3933: - Attachment: screenshot-1.png > Memory leak because of too many Datanode State Machine Thread >

[jira] [Created] (HDDS-3933) Memory leak because of too many Datanode State Machine Thread

2020-07-07 Thread runzhiwang (Jira)
runzhiwang created HDDS-3933: Summary: Memory leak because of too many Datanode State Machine Thread Key: HDDS-3933 URL: https://issues.apache.org/jira/browse/HDDS-3933 Project: Hadoop Distributed Data

[GitHub] [hadoop-ozone] lokeshj1703 commented on pull request #1121: HDDS-3432. Enable TestBlockDeletion test cases.

2020-07-07 Thread GitBox
lokeshj1703 commented on pull request #1121: URL: https://github.com/apache/hadoop-ozone/pull/1121#issuecomment-654748191 @adoroszlai Thanks for verifying! I see appendEntries timeout in the particular run. The blocks creation in all the datanodes timed out in the run. The failure occurs

[jira] [Assigned] (HDDS-3659) put a new file to exist key with different factor or type don't update the omkeyinfo

2020-07-07 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao reassigned HDDS-3659: -- Assignee: HuangTao > put a new file to exist key with different factor or type don't update the >

[jira] [Commented] (HDDS-3918) ConcurrentModificationException in ContainerReportHandler.onMessage

2020-07-07 Thread Sammi Chen (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152605#comment-17152605 ] Sammi Chen commented on HDDS-3918: -- [~jnp] I usually see it after SCM restart. >

[jira] [Updated] (HDDS-3931) Maven warning due to deprecated expression pom.artifactId

2020-07-07 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-3931: --- Status: Patch Available (was: In Progress) > Maven warning due to deprecated expression

[jira] [Updated] (HDDS-3913) Recon build should ignore proxies

2020-07-07 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-3913: --- Labels: (was: pull-request-available) > Recon build should ignore proxies >

[jira] [Updated] (HDDS-3913) Recon build should ignore proxies

2020-07-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-3913: - Labels: pull-request-available (was: ) > Recon build should ignore proxies >

[jira] [Updated] (HDDS-3913) Recon build should ignore proxies

2020-07-07 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-3913: --- Fix Version/s: 0.6.0 Target Version/s: (was: 0.6.0) Resolution: Fixed

[jira] [Updated] (HDDS-3913) Recon build should ignore proxies

2020-07-07 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-3913: --- Labels: (was: pull-request-available) > Recon build should ignore proxies >

[GitHub] [hadoop-ozone] adoroszlai commented on pull request #1159: HDDS-3913. Recon build should ignore proxies

2020-07-07 Thread GitBox
adoroszlai commented on pull request #1159: URL: https://github.com/apache/hadoop-ozone/pull/1159#issuecomment-654707281 Thanks @vivekratnavel for the fix and @GlenGeng for the review. This is an automated message from the

[GitHub] [hadoop-ozone] adoroszlai merged pull request #1159: HDDS-3913. Recon build should ignore proxies

2020-07-07 Thread GitBox
adoroszlai merged pull request #1159: URL: https://github.com/apache/hadoop-ozone/pull/1159 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hadoop-ozone] adoroszlai commented on pull request #1121: HDDS-3432. Enable TestBlockDeletion test cases.

2020-07-07 Thread GitBox
adoroszlai commented on pull request #1121: URL: https://github.com/apache/hadoop-ozone/pull/1121#issuecomment-654703364 Thanks @lokeshj1703 for updating the patch. I still see 1/20 failure due to timeout at `TestBlockDeletion.testBlockDeletion(TestBlockDeletion.java:174)`.

[jira] [Updated] (HDDS-3931) Maven warning due to deprecated expression pom.artifactId

2020-07-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-3931: - Labels: pull-request-available (was: ) > Maven warning due to deprecated expression

[GitHub] [hadoop-ozone] adoroszlai opened a new pull request #1172: HDDS-3931. Maven warning due to deprecated expression pom.artifactId

2020-07-07 Thread GitBox
adoroszlai opened a new pull request #1172: URL: https://github.com/apache/hadoop-ozone/pull/1172 ## What changes were proposed in this pull request? Fix Maven warning caused by deprecated expression `pom.artifactId`: ``` [INFO] Scanning for projects... [WARNING]

[jira] [Resolved] (HDDS-3707) UUID can be non unique for a huge samples

2020-07-07 Thread maobaolong (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maobaolong resolved HDDS-3707. -- Assignee: maobaolong Resolution: Won't Fix > UUID can be non unique for a huge samples >

[GitHub] [hadoop-ozone] cku328 commented on pull request #1170: HDDS-3910. JooqCodeGenerator interrupted but still alive

2020-07-07 Thread GitBox
cku328 commented on pull request #1170: URL: https://github.com/apache/hadoop-ozone/pull/1170#issuecomment-654680208 Thanks @adoroszlai for the review. This is an automated message from the Apache Git Service. To respond to

[GitHub] [hadoop-ozone] cku328 opened a new pull request #1171: HDDS-3932. Hide jOOQ logo message from the log output on compile

2020-07-07 Thread GitBox
cku328 opened a new pull request #1171: URL: https://github.com/apache/hadoop-ozone/pull/1171 ## What changes were proposed in this pull request? Add system property to hide this self-ad message from the log output. ## What is the link to the Apache JIRA

[jira] [Updated] (HDDS-3932) Hide jOOQ logo message from the log output on compile

2020-07-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-3932: - Labels: pull-request-available (was: ) > Hide jOOQ logo message from the log output on compile >

[jira] [Created] (HDDS-3932) Hide jOOQ logo message from the log output on compile

2020-07-07 Thread Neo Yang (Jira)
Neo Yang created HDDS-3932: -- Summary: Hide jOOQ logo message from the log output on compile Key: HDDS-3932 URL: https://issues.apache.org/jira/browse/HDDS-3932 Project: Hadoop Distributed Data Store

[jira] [Updated] (HDDS-3931) Maven warning due to deprecated expression pom.artifactId

2020-07-07 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-3931: --- Description: {code:title=mvn clean} [INFO] Scanning for projects... [WARNING] [WARNING] Some

[jira] [Created] (HDDS-3931) Maven warning due to deprecated expression pom.artifactId

2020-07-07 Thread Attila Doroszlai (Jira)
Attila Doroszlai created HDDS-3931: -- Summary: Maven warning due to deprecated expression pom.artifactId Key: HDDS-3931 URL: https://issues.apache.org/jira/browse/HDDS-3931 Project: Hadoop Distributed

  1   2   >