[GitHub] [hadoop-ozone] adoroszlai commented on a change in pull request #1454: HDDS-4285. Read is slow due to frequent calls to UGI.getCurrentUser() and getTokens()

adoroszlai commented on a change in pull request #1454: URL: https://github.com/apache/hadoop-ozone/pull/1454#discussion_r496889816 ## File path: hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/storage/ContainerProtocolCalls.java ## @@ -84,7 +86,8 @@ private

[GitHub] [hadoop-ozone] avijayanhwx merged pull request #1430: HDDS-4227. Implement a 'Prepare For Upgrade' step in OM that applies all committed Ratis transactions.

avijayanhwx merged pull request #1430: URL: https://github.com/apache/hadoop-ozone/pull/1430 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hadoop-ozone] avijayanhwx commented on pull request #1430: HDDS-4227. Implement a 'Prepare For Upgrade' step in OM that applies all committed Ratis transactions.

avijayanhwx commented on pull request #1430: URL: https://github.com/apache/hadoop-ozone/pull/1430#issuecomment-700837411 Thank you for the reviews @linyiqun, @fapifta & @swagle. Merging this. This is an automated message

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

bharatviswa504 commented on a change in pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#discussion_r496831469 ## File path: hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmDirectoryInfo.java ## @@ -0,0 +1,266 @@ +/** + *

[GitHub] [hadoop-ozone] linyiqun commented on a change in pull request #1454: HDDS-4285. Read is slow due to frequent calls to UGI.getCurrentUser() and getTokens()

linyiqun commented on a change in pull request #1454: URL: https://github.com/apache/hadoop-ozone/pull/1454#discussion_r496819670 ## File path: hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/storage/ContainerProtocolCalls.java ## @@ -84,7 +86,8 @@ private

[GitHub] [hadoop-ozone] rakeshadr commented on a change in pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

rakeshadr commented on a change in pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#discussion_r496824181 ## File path: hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/file/TestOMDirectoryCreateRequestV1.java ## @@ -0,0

[GitHub] [hadoop-ozone] rakeshadr commented on a change in pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

rakeshadr commented on a change in pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#discussion_r496824421 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMFileRequest.java ## @@ -129,6 +134,131 @@ public

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

bharatviswa504 commented on a change in pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#discussion_r496823617 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMFileRequest.java ## @@ -129,6 +134,131 @@

[GitHub] [hadoop-ozone] rakeshadr commented on a change in pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

rakeshadr commented on a change in pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#discussion_r496821295 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMDirectoryCreateRequestV1.java ## @@ -0,0 +1,323

[GitHub] [hadoop-ozone] rakeshadr commented on a change in pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

rakeshadr commented on a change in pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#discussion_r496820524 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMDirectoryCreateRequestV1.java ## @@ -0,0 +1,312

[GitHub] [hadoop-ozone] rakeshadr commented on a change in pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

rakeshadr commented on a change in pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#discussion_r496819566 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMFileRequest.java ## @@ -129,6 +134,131 @@ public

[GitHub] [hadoop-ozone] rakeshadr commented on a change in pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

rakeshadr commented on a change in pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#discussion_r496818679 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMFileRequest.java ## @@ -129,6 +134,131 @@ public

[GitHub] [hadoop-ozone] rakeshadr commented on a change in pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

rakeshadr commented on a change in pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#discussion_r496818679 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMFileRequest.java ## @@ -129,6 +134,131 @@ public

[GitHub] [hadoop-ozone] rakeshadr commented on a change in pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

rakeshadr commented on a change in pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#discussion_r496815568 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMDirectoryCreateRequestV1.java ## @@ -0,0 +1,323

[GitHub] [hadoop-ozone] rakeshadr commented on a change in pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

rakeshadr commented on a change in pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#discussion_r496813934 ## File path: hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/file/TestOMDirectoryCreateRequestV1.java ## @@ -0,0

[GitHub] [hadoop-ozone] rakeshadr commented on a change in pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

rakeshadr commented on a change in pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#discussion_r496812987 ## File path: hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmDirectoryInfo.java ## @@ -0,0 +1,266 @@ +/** + * Licensed to

[GitHub] [hadoop-ozone] adoroszlai opened a new pull request #1454: HDDS-4285. Read is slow due to the frequent usage of UGI.getCurrentUser Call()

adoroszlai opened a new pull request #1454: URL: https://github.com/apache/hadoop-ozone/pull/1454 ## What changes were proposed in this pull request? Reduce the number of `getCurrentUser()` and `getTokens()` calls performed during some `ContainerProtocolCalls` operations. This is

[GitHub] [hadoop-ozone] captainzmc commented on pull request #1412: HDDS-3751. Ozone sh client support bucket quota option.

captainzmc commented on pull request #1412: URL: https://github.com/apache/hadoop-ozone/pull/1412#issuecomment-700751283 Thanks @adoroszlai’s review, I have already fixed the review issues, can you help take another look？ [CI is normal in my personal

[GitHub] [hadoop-ozone] elek opened a new pull request #1453: HDDS-4290. Enable insight point for SCM heartbeat protocol

elek opened a new pull request #1453: URL: https://github.com/apache/hadoop-ozone/pull/1453 ## What changes were proposed in this pull request? The registration of the already implemented insight-point seems to be missing. ## What is the link to the Apache JIRA

[GitHub] [hadoop-ozone] elek opened a new pull request #1452: HDDS-4288. the icon of hadoop-ozone is bigger than ever

elek opened a new pull request #1452: URL: https://github.com/apache/hadoop-ozone/pull/1452 ## What changes were proposed in this pull request? Logo is too big on doc snapshot (generated by the Jenkins):

[GitHub] [hadoop-ozone] runzhiwang commented on a change in pull request #1371: HDDS-2922. Balance ratis leader distribution in datanodes

runzhiwang commented on a change in pull request #1371: URL: https://github.com/apache/hadoop-ozone/pull/1371#discussion_r496643970 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/SCMPipelineManager.java ## @@ -619,9 +619,9 @@ private

[GitHub] [hadoop-ozone] elek merged pull request #1403: HDDS-4215. update freon doc.

elek merged pull request #1403: URL: https://github.com/apache/hadoop-ozone/pull/1403 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [hadoop-ozone] captainzmc commented on a change in pull request #1412: HDDS-3751. Ozone sh client support bucket quota option.

captainzmc commented on a change in pull request #1412: URL: https://github.com/apache/hadoop-ozone/pull/1412#discussion_r496662068 ## File path: hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rpc/RpcClient.java ## @@ -441,6 +431,8 @@ public void

[GitHub] [hadoop-ozone] runzhiwang commented on a change in pull request #1371: HDDS-2922. Balance ratis leader distribution in datanodes

runzhiwang commented on a change in pull request #1371: URL: https://github.com/apache/hadoop-ozone/pull/1371#discussion_r496627193 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/RatisPipelineProvider.java ## @@ -121,18 +158,24 @@

[GitHub] [hadoop-ozone] runzhiwang commented on a change in pull request #1371: HDDS-2922. Balance ratis leader distribution in datanodes

runzhiwang commented on a change in pull request #1371: URL: https://github.com/apache/hadoop-ozone/pull/1371#discussion_r496643970 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/SCMPipelineManager.java ## @@ -619,9 +619,9 @@ private

[GitHub] [hadoop-ozone] ChenSammi merged pull request #1338: HDDS-4023. Delete closed container after all blocks have been deleted.

ChenSammi merged pull request #1338: URL: https://github.com/apache/hadoop-ozone/pull/1338 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hadoop-ozone] runzhiwang commented on a change in pull request #1371: HDDS-2922. Balance ratis leader distribution in datanodes

runzhiwang commented on a change in pull request #1371: URL: https://github.com/apache/hadoop-ozone/pull/1371#discussion_r496643970 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/SCMPipelineManager.java ## @@ -619,9 +619,9 @@ private

[GitHub] [hadoop-ozone] ChenSammi commented on pull request #1338: HDDS-4023. Delete closed container after all blocks have been deleted.

ChenSammi commented on pull request #1338: URL: https://github.com/apache/hadoop-ozone/pull/1338#issuecomment-700642125 Thanks @sodonnel and @linyiqun for the review. Basically I think report handler is not a good place to handle all the empty container deletion process. It can

[GitHub] [hadoop-ozone] adoroszlai merged pull request #1415: HDDS-4232. Use single thread for KeyDeletingService.

adoroszlai merged pull request #1415: URL: https://github.com/apache/hadoop-ozone/pull/1415 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hadoop-ozone] adoroszlai commented on pull request #1415: HDDS-4232. Use single thread for KeyDeletingService.

adoroszlai commented on pull request #1415: URL: https://github.com/apache/hadoop-ozone/pull/1415#issuecomment-700638453 Thanks @lokeshj1703 for the fix and @amaliujia and @bharatviswa504 for the review. This is an

[GitHub] [hadoop-ozone] adoroszlai commented on a change in pull request #1407: HDDS-4158. Provide a class type for Java based configuration

adoroszlai commented on a change in pull request #1407: URL: https://github.com/apache/hadoop-ozone/pull/1407#discussion_r496629283 ## File path: hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/conf/SimpleConfiguration.java ## @@ -93,4 +98,12 @@ public int getPort() {

[GitHub] [hadoop-ozone] runzhiwang commented on a change in pull request #1371: HDDS-2922. Balance ratis leader distribution in datanodes

runzhiwang commented on a change in pull request #1371: URL: https://github.com/apache/hadoop-ozone/pull/1371#discussion_r496628424 ## File path: hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/protocol/commands/CreatePipelineCommand.java ## @@ -48,16

[GitHub] [hadoop-ozone] runzhiwang commented on a change in pull request #1371: HDDS-2922. Balance ratis leader distribution in datanodes

runzhiwang commented on a change in pull request #1371: URL: https://github.com/apache/hadoop-ozone/pull/1371#discussion_r496628148 ## File path: hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/ratis/XceiverServerRatis.java

[GitHub] [hadoop-ozone] runzhiwang commented on a change in pull request #1371: HDDS-2922. Balance ratis leader distribution in datanodes

runzhiwang commented on a change in pull request #1371: URL: https://github.com/apache/hadoop-ozone/pull/1371#discussion_r496627193 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/RatisPipelineProvider.java ## @@ -121,18 +158,24 @@

[GitHub] [hadoop-ozone] runzhiwang commented on a change in pull request #1371: HDDS-2922. Balance ratis leader distribution in datanodes

runzhiwang commented on a change in pull request #1371: URL: https://github.com/apache/hadoop-ozone/pull/1371#discussion_r496624353 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/PipelineStateManager.java ## @@ -59,7 +59,8 @@ void

[GitHub] [hadoop-ozone] adoroszlai commented on a change in pull request #1412: HDDS-3751. Ozone sh client support bucket quota option.

adoroszlai commented on a change in pull request #1412: URL: https://github.com/apache/hadoop-ozone/pull/1412#discussion_r496617299 ## File path: hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rpc/RpcClient.java ## @@ -441,6 +431,8 @@ public void

[GitHub] [hadoop-ozone] linyiqun edited a comment on pull request #1338: HDDS-4023. Delete closed container after all blocks have been deleted.

linyiqun edited a comment on pull request #1338: URL: https://github.com/apache/hadoop-ozone/pull/1338#issuecomment-700598266 > @linyiqun I do agree that I think this could be handled more cleanly and efficiently in the container report handler. However its probably not much of an

[GitHub] [hadoop-ozone] linyiqun commented on pull request #1338: HDDS-4023. Delete closed container after all blocks have been deleted.

linyiqun commented on pull request #1338: URL: https://github.com/apache/hadoop-ozone/pull/1338#issuecomment-700598266 > @linyiqun I do agree that I think this could be handled more cleanly and efficiently in the container report handler. However its probably not much of an overhead for

[GitHub] [hadoop-ozone] sodonnel commented on pull request #1338: HDDS-4023. Delete closed container after all blocks have been deleted.

sodonnel commented on pull request #1338: URL: https://github.com/apache/hadoop-ozone/pull/1338#issuecomment-700582338 > There is following logic in ReplicatioManager, which will handle the replicas reported during container state is DELETING. Sorry I missed that. You are correct. I

[GitHub] [hadoop-ozone] lokeshj1703 commented on pull request #1415: HDDS-4232. Use single thread for KeyDeletingService.

lokeshj1703 commented on pull request #1415: URL: https://github.com/apache/hadoop-ozone/pull/1415#issuecomment-700573211 /ready This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [hadoop-ozone] linyiqun commented on a change in pull request #1338: HDDS-4023. Delete closed container after all blocks have been deleted.

linyiqun commented on a change in pull request #1338: URL: https://github.com/apache/hadoop-ozone/pull/1338#discussion_r496544696 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ReplicationManager.java ## @@ -320,6 +331,12 @@ private

[GitHub] [hadoop-ozone] adoroszlai commented on pull request #1414: HDDS-4231. Background Service blocks on task results.

adoroszlai commented on pull request #1414: URL: https://github.com/apache/hadoop-ozone/pull/1414#issuecomment-700548576 Thanks @lokeshj1703 for the contribution and @amaliujia and @nandakumar131 for the review. This is an

[GitHub] [hadoop-ozone] adoroszlai merged pull request #1414: HDDS-4231. Background Service blocks on task results.

adoroszlai merged pull request #1414: URL: https://github.com/apache/hadoop-ozone/pull/1414 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hadoop-ozone] GlenGeng commented on a change in pull request #1371: HDDS-2922. Balance ratis leader distribution in datanodes

GlenGeng commented on a change in pull request #1371: URL: https://github.com/apache/hadoop-ozone/pull/1371#discussion_r496533516 ## File path: hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/protocol/commands/CreatePipelineCommand.java ## @@ -39,6 +42,7

[GitHub] [hadoop-ozone] GlenGeng commented on a change in pull request #1371: HDDS-2922. Balance ratis leader distribution in datanodes

GlenGeng commented on a change in pull request #1371: URL: https://github.com/apache/hadoop-ozone/pull/1371#discussion_r496496531 ## File path: hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/protocol/commands/CreatePipelineCommand.java ## @@ -48,16 +52,37

[GitHub] [hadoop-ozone] ChenSammi commented on pull request #1412: HDDS-3751. Ozone sh client support bucket quota option.

ChenSammi commented on pull request #1412: URL: https://github.com/apache/hadoop-ozone/pull/1412#issuecomment-700512727 LGTM, +1. Hi @adoroszlai，I'm not sure if you are satified with the change, would you like to take another look at this patch?

[GitHub] [hadoop-ozone] lokeshj1703 commented on pull request #1414: HDDS-4231. Background Service blocks on task results.

lokeshj1703 commented on pull request #1414: URL: https://github.com/apache/hadoop-ozone/pull/1414#issuecomment-700494136 @nandakumar131 Thanks for the suggestion! I have removed generic reference from both BackgroundTaskQueue and BackgroundTask. BackgroundTask also doesn't require a

[GitHub] [hadoop-ozone] timmylicheng commented on pull request #1346: HDDS-4115. CLI command to show current SCM leader and follower status.

timmylicheng commented on pull request #1346: URL: https://github.com/apache/hadoop-ozone/pull/1346#issuecomment-700491675 +1. Thanks for Rui's contribution. Merging This is an automated message from the Apache Git

[GitHub] [hadoop-ozone] timmylicheng merged pull request #1346: HDDS-4115. CLI command to show current SCM leader and follower status.

timmylicheng merged pull request #1346: URL: https://github.com/apache/hadoop-ozone/pull/1346 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hadoop-ozone] captainzmc commented on a change in pull request #1412: HDDS-3751. Ozone sh client support bucket quota option.

captainzmc commented on a change in pull request #1412: URL: https://github.com/apache/hadoop-ozone/pull/1412#discussion_r496451487 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/bucket/OMBucketCreateRequest.java ## @@ -192,6 +192,10

[GitHub] [hadoop-ozone] runzhiwang commented on pull request #1371: HDDS-2922. Balance ratis leader distribution in datanodes

runzhiwang commented on pull request #1371: URL: https://github.com/apache/hadoop-ozone/pull/1371#issuecomment-700470083 @xiaoyuyao @bshashikant I have updated the patch. Could you help review it again ? Thank you very much.

[GitHub] [hadoop-ozone] amaliujia commented on pull request #1346: HDDS-4115. CLI command to show current SCM leader and follower status.

amaliujia commented on pull request #1346: URL: https://github.com/apache/hadoop-ozone/pull/1346#issuecomment-700441836 Re @bshashikant Agreed. Right now the command itself is unified (for both OM and SCM, we name this command as `roles`). Then we should unify the behavior fo both

[GitHub] [hadoop-ozone] bshashikant commented on pull request #1346: HDDS-4115. CLI command to show current SCM leader and follower status.

bshashikant commented on pull request #1346: URL: https://github.com/apache/hadoop-ozone/pull/1346#issuecomment-700435789 The CLI should be generic for both OM and SCM and probably extended to Datanodes as well. This is an

[GitHub] [hadoop-ozone] bshashikant edited a comment on pull request #1346: HDDS-4115. CLI command to show current SCM leader and follower status.

bshashikant edited a comment on pull request #1346: URL: https://github.com/apache/hadoop-ozone/pull/1346#issuecomment-700435789 I feel the CLI should be common for both OM and SCM and probably extended to Datanodes as well.

[GitHub] [hadoop-ozone] bshashikant commented on a change in pull request #1274: HDDS-3810. Add the logic to distribute open containers among the pipelines of a datanode.

bshashikant commented on a change in pull request #1274: URL: https://github.com/apache/hadoop-ozone/pull/1274#discussion_r496394076 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/SCMContainerManager.java ## @@ -101,12 +103,23 @@

[GitHub] [hadoop-ozone] bshashikant commented on a change in pull request #1274: HDDS-3810. Add the logic to distribute open containers among the pipelines of a datanode.

bshashikant commented on a change in pull request #1274: URL: https://github.com/apache/hadoop-ozone/pull/1274#discussion_r496394076 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/SCMContainerManager.java ## @@ -101,12 +103,23 @@

[GitHub] [hadoop-ozone] GlenGeng commented on a change in pull request #1274: HDDS-3810. Add the logic to distribute open containers among the pipelines of a datanode.

GlenGeng commented on a change in pull request #1274: URL: https://github.com/apache/hadoop-ozone/pull/1274#discussion_r496361400 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/SCMContainerManager.java ## @@ -101,12 +103,23 @@ public

[GitHub] [hadoop-ozone] avijayanhwx commented on a change in pull request #1430: HDDS-4227. Implement a 'Prepare For Upgrade' step in OM that applies all committed Ratis transactions.

avijayanhwx commented on a change in pull request #1430: URL: https://github.com/apache/hadoop-ozone/pull/1430#discussion_r496366104 ## File path: hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/ratis/RatisUpgradeUtils.java ## @@ -0,0 +1,96 @@ +/** + * Licensed to the

[GitHub] [hadoop-ozone] bharatviswa504 opened a new pull request #1451: Hdds 4117

bharatviswa504 opened a new pull request #1451: URL: https://github.com/apache/hadoop-ozone/pull/1451 ## What changes were proposed in this pull request? Normalize Keypath for listKeys. When ozone.om.enable.filesystem.paths, OM normalizes path, and stores the Keyname in the OM DB

[GitHub] [hadoop-ozone] ChenSammi commented on pull request #1338: HDDS-4023. Delete closed container after all blocks have been deleted.

ChenSammi commented on pull request #1338: URL: https://github.com/apache/hadoop-ozone/pull/1338#issuecomment-700409940 > > > This change looks almost good now. I wonder about two final things: > > 1. In `updateContainerStats(...)` do you think we should return if the

[GitHub] [hadoop-ozone] GlenGeng commented on a change in pull request #1274: HDDS-3810. Add the logic to distribute open containers among the pipelines of a datanode.

GlenGeng commented on a change in pull request #1274: URL: https://github.com/apache/hadoop-ozone/pull/1274#discussion_r496361400 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/SCMContainerManager.java ## @@ -101,12 +103,23 @@ public

[GitHub] [hadoop-ozone] GlenGeng commented on a change in pull request #1274: HDDS-3810. Add the logic to distribute open containers among the pipelines of a datanode.

GlenGeng commented on a change in pull request #1274: URL: https://github.com/apache/hadoop-ozone/pull/1274#discussion_r496348325 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/DatanodeInfo.java ## @@ -40,6 +43,8 @@ private long

[GitHub] [hadoop-ozone] ChenSammi commented on pull request #1434: HDDS-3727. Volume space: check quotaUsageInBytes when write key.

ChenSammi commented on pull request #1434: URL: https://github.com/apache/hadoop-ozone/pull/1434#issuecomment-700401426 LGTM +1. This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [hadoop-ozone] ChenSammi merged pull request #1434: HDDS-3727. Volume space: check quotaUsageInBytes when write key.

ChenSammi merged pull request #1434: URL: https://github.com/apache/hadoop-ozone/pull/1434 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hadoop-ozone] swagle commented on a change in pull request #1430: HDDS-4227. Implement a 'Prepare For Upgrade' step in OM that applies all committed Ratis transactions.

swagle commented on a change in pull request #1430: URL: https://github.com/apache/hadoop-ozone/pull/1430#discussion_r496341141 ## File path: hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/ratis/RatisUpgradeUtils.java ## @@ -0,0 +1,96 @@ +/** + * Licensed to the

[GitHub] [hadoop-ozone] fapifta commented on pull request #1430: HDDS-4227. Implement a 'Prepare For Upgrade' step in OM that applies all committed Ratis transactions.

fapifta commented on pull request #1430: URL: https://github.com/apache/hadoop-ozone/pull/1430#issuecomment-700347698 Hi @avijayanhwx sorry for the long silence, the changes look good to me with the follow up items, +1.

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

bharatviswa504 commented on a change in pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#discussion_r496246400 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMDirectoryCreateRequestV1.java ## @@ -0,0

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

bharatviswa504 commented on a change in pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#discussion_r496145730 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMFileRequest.java ## @@ -129,6 +134,131 @@

[GitHub] [hadoop-ozone] hanishakoneru commented on pull request #1298: HDDS-3869. Use different column families for datanode block and metadata

hanishakoneru commented on pull request #1298: URL: https://github.com/apache/hadoop-ozone/pull/1298#issuecomment-700258852 +1 pending CI. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [hadoop-ozone] nandakumar131 commented on a change in pull request #1378: HDDS-4133. Use new ContainerManager in SCM.

nandakumar131 commented on a change in pull request #1378: URL: https://github.com/apache/hadoop-ozone/pull/1378#discussion_r496151525 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ContainerManagerImpl.java ## @@ -242,19 +286,82 @@

[GitHub] [hadoop-ozone] elek merged pull request #1328: HDDS-4102. Normalize Keypath for lookupKey.

elek merged pull request #1328: URL: https://github.com/apache/hadoop-ozone/pull/1328 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [hadoop-ozone] avijayanhwx commented on pull request #1432: HDDS-4252. Add the current layout versions to DN - SCM proto payload.

avijayanhwx commented on pull request #1432: URL: https://github.com/apache/hadoop-ozone/pull/1432#issuecomment-700205929 Since failure is unrelated, and this work is going on in a branch, I am merging these changes. Thanks for the review @linyiqun.

[GitHub] [hadoop-ozone] avijayanhwx merged pull request #1432: HDDS-4252. Add the current layout versions to DN - SCM proto payload.

avijayanhwx merged pull request #1432: URL: https://github.com/apache/hadoop-ozone/pull/1432 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hadoop-ozone] nandakumar131 commented on a change in pull request #1378: HDDS-4133. Use new ContainerManager in SCM.

nandakumar131 commented on a change in pull request #1378: URL: https://github.com/apache/hadoop-ozone/pull/1378#discussion_r496149425 ## File path: hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/container/ContainerID.java ## @@ -41,8 +41,8 @@ * @param id int

[GitHub] [hadoop-ozone] nandakumar131 commented on pull request #1414: HDDS-4231. Background Service blocks on task results.

nandakumar131 commented on pull request #1414: URL: https://github.com/apache/hadoop-ozone/pull/1414#issuecomment-700200428 Overall the patch looks good to me. A minor suggestion, feel free to ignore it. Instead of having a generic reference in `BackgroundTaskQueue` (which

[GitHub] [hadoop-ozone] nandakumar131 commented on a change in pull request #1274: HDDS-3810. Add the logic to distribute open containers among the pipelines of a datanode.

nandakumar131 commented on a change in pull request #1274: URL: https://github.com/apache/hadoop-ozone/pull/1274#discussion_r496111822 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/DatanodeInfo.java ## @@ -121,6 +143,19 @@ public int

[GitHub] [hadoop-ozone] amaliujia commented on pull request #1445: HDDS-4272. Volume namespace: add usedNamespace and update it when create and delete bucket

amaliujia commented on pull request #1445: URL: https://github.com/apache/hadoop-ozone/pull/1445#issuecomment-700130504 @cxorm can you take another look please? Thanks! This is an automated message from the Apache Git

[GitHub] [hadoop-ozone] sodonnel commented on pull request #1338: HDDS-4023. Delete closed container after all blocks have been deleted.

sodonnel commented on pull request #1338: URL: https://github.com/apache/hadoop-ozone/pull/1338#issuecomment-700016623 This change looks almost good now. I wonder about two final things: 1. In `updateContainerStats(...)` do you think we should return if the container is DELETING or

[GitHub] [hadoop-ozone] elek commented on pull request #1419: HDDS-3755. [DESIGN] Storage-class for Ozone

elek commented on pull request #1419: URL: https://github.com/apache/hadoop-ozone/pull/1419#issuecomment-699905781 >> Isn't this design doc a few steps away from coding? A ton of detail is missing around how SCM will manage multiple classes of pipelines and how replication manager will

[GitHub] [hadoop-ozone] ChenSammi commented on a change in pull request #1412: HDDS-3751. Ozone sh client support bucket quota option.

ChenSammi commented on a change in pull request #1412: URL: https://github.com/apache/hadoop-ozone/pull/1412#discussion_r495797897 ## File path: hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/bucket/SetQuotaHandler.java ## @@ -0,0 +1,72 @@ +/* + * Licensed to

[GitHub] [hadoop-ozone] ChenSammi commented on a change in pull request #1412: HDDS-3751. Ozone sh client support bucket quota option.

ChenSammi commented on a change in pull request #1412: URL: https://github.com/apache/hadoop-ozone/pull/1412#discussion_r495777131 ## File path: hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rpc/RpcClient.java ## @@ -334,15 +329,11 @@ public boolean

[GitHub] [hadoop-ozone] ChenSammi commented on a change in pull request #1412: HDDS-3751. Ozone sh client support bucket quota option.

ChenSammi commented on a change in pull request #1412: URL: https://github.com/apache/hadoop-ozone/pull/1412#discussion_r495775842 ## File path: hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rpc/RpcClient.java ## @@ -496,6 +491,28 @@ private static void

[GitHub] [hadoop-ozone] ChenSammi commented on a change in pull request #1412: HDDS-3751. Ozone sh client support bucket quota option.

ChenSammi commented on a change in pull request #1412: URL: https://github.com/apache/hadoop-ozone/pull/1412#discussion_r495775842 ## File path: hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rpc/RpcClient.java ## @@ -496,6 +491,28 @@ private static void

[GitHub] [hadoop-ozone] ChenSammi commented on a change in pull request #1412: HDDS-3751. Ozone sh client support bucket quota option.

ChenSammi commented on a change in pull request #1412: URL: https://github.com/apache/hadoop-ozone/pull/1412#discussion_r495775842 ## File path: hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rpc/RpcClient.java ## @@ -496,6 +491,28 @@ private static void

[GitHub] [hadoop-ozone] nandakumar131 commented on pull request #1438: HDDS-4263. ReplicatiomManager shouldn't retain one healthy replica per origin node Id.

nandakumar131 commented on pull request #1438: URL: https://github.com/apache/hadoop-ozone/pull/1438#issuecomment-699767348 Thanks @maobaolong for the contribution! This is an automated message from the Apache Git Service.

[GitHub] [hadoop-ozone] nandakumar131 merged pull request #1438: HDDS-4263. ReplicatiomManager shouldn't retain one healthy replica per origin node Id.

nandakumar131 merged pull request #1438: URL: https://github.com/apache/hadoop-ozone/pull/1438 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hadoop-ozone] captainzmc edited a comment on pull request #1447: HDDS-4264. Uniform naming conventions of Ozone Shell Options.

captainzmc edited a comment on pull request #1447: URL: https://github.com/apache/hadoop-ozone/pull/1447#issuecomment-698921171 Hi @adoroszlai @elek @ChenSammi Could you help review this PR? This is an automated message

[GitHub] [hadoop-ozone] amaliujia commented on pull request #1346: HDDS-4115. CLI command to show current SCM leader and follower status.

amaliujia commented on pull request #1346: URL: https://github.com/apache/hadoop-ozone/pull/1346#issuecomment-699688323 @timmylicheng comments addressed. Can you take another look? This is an automated message from the

[GitHub] [hadoop-ozone] amaliujia commented on a change in pull request #1346: HDDS-4115. CLI command to show current SCM leader and follower status.

amaliujia commented on a change in pull request #1346: URL: https://github.com/apache/hadoop-ozone/pull/1346#discussion_r495601428 ## File path: hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/ha/MockSCMHAManager.java ## @@ -107,6 +108,14 @@ public void

[GitHub] [hadoop-ozone] amaliujia commented on a change in pull request #1346: HDDS-4115. CLI command to show current SCM leader and follower status.

amaliujia commented on a change in pull request #1346: URL: https://github.com/apache/hadoop-ozone/pull/1346#discussion_r495601341 ## File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/shell/TestScmAdminHA.java ## @@ -0,0 +1,79 @@ +/* + * Licensed

[GitHub] [hadoop-ozone] amaliujia commented on a change in pull request #1346: HDDS-4115. CLI command to show current SCM leader and follower status.

amaliujia commented on a change in pull request #1346: URL: https://github.com/apache/hadoop-ozone/pull/1346#discussion_r495601298 ## File path: hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/scm/GetScmRatisStatusSubcommand.java ## @@ -0,0 +1,46 @@ +/* + *

[GitHub] [hadoop-ozone] mukul1987 merged pull request #1450: HDDS-4282. Improve the emptyDir syntax

mukul1987 merged pull request #1450: URL: https://github.com/apache/hadoop-ozone/pull/1450 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hadoop-ozone] amaliujia commented on pull request #1447: HDDS-4264. Uniform naming conventions of Ozone Shell Options.

amaliujia commented on pull request #1447: URL: https://github.com/apache/hadoop-ozone/pull/1447#issuecomment-699665833 @captainzmc you might try `git commit --allow-empty -m 'trigger new CI check' && git push` to trigger CI which does not trigger PR change notification :-)

[GitHub] [hadoop-ozone] captainzmc closed pull request #1447: HDDS-4264. Uniform naming conventions of Ozone Shell Options.

captainzmc closed pull request #1447: URL: https://github.com/apache/hadoop-ozone/pull/1447 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hadoop-ozone] captainzmc closed pull request #1447: HDDS-4264. Uniform naming conventions of Ozone Shell Options.

captainzmc closed pull request #1447: URL: https://github.com/apache/hadoop-ozone/pull/1447 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hadoop-ozone] captainzmc closed pull request #1447: HDDS-4264. Uniform naming conventions of Ozone Shell Options.

captainzmc closed pull request #1447: URL: https://github.com/apache/hadoop-ozone/pull/1447 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hadoop-ozone] lamber-ken opened a new pull request #1450: HDDS-4282. Improve the emptyDir syntax

lamber-ken opened a new pull request #1450: URL: https://github.com/apache/hadoop-ozone/pull/1450 ## What changes were proposed in this pull request? Improve the emptyDir syntax, from `emptyDir: null` to `emptyDir: {}` ## What is the link to the Apache JIRA

[GitHub] [hadoop-ozone] captainzmc commented on pull request #1434: HDDS-3727. Volume space: check quotaUsageInBytes when write key.

captainzmc commented on pull request #1434: URL: https://github.com/apache/hadoop-ozone/pull/1434#issuecomment-699612552 Thanks @ChenSammi for the review. Review issues has been fixed. This is an automated message from the

[GitHub] [hadoop-ozone] captainzmc removed a comment on pull request #1434: HDDS-3727. Volume space: check quotaUsageInBytes when write key.

captainzmc removed a comment on pull request #1434: URL: https://github.com/apache/hadoop-ozone/pull/1434#issuecomment-698235132 Thanks for @ChenSammi ‘s review. The review issues have been fixed. Could you help take another look?

[GitHub] [hadoop-ozone] timmylicheng closed pull request #1340: HDDS-3188 Add failover proxy for SCM block location.

timmylicheng closed pull request #1340: URL: https://github.com/apache/hadoop-ozone/pull/1340 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

<    3   4   5   6   7   8   9   10   11   12   >