[GitHub] [hadoop-ozone] rakeshadr commented on a change in pull request #1437: HDDS-4222: [OzoneFS optimization] Provide a mechanism for efficient path lookup

2020-09-21 Thread GitBox
rakeshadr commented on a change in pull request #1437: URL: https://github.com/apache/hadoop-ozone/pull/1437#discussion_r492481705 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/cache/OMMetadataCacheFactory.java ## @@ -0,0 +1,120 @@ +/** + *

[GitHub] [hadoop-ozone] rakeshadr commented on a change in pull request #1437: HDDS-4222: [OzoneFS optimization] Provide a mechanism for efficient path lookup

2020-09-21 Thread GitBox
rakeshadr commented on a change in pull request #1437: URL: https://github.com/apache/hadoop-ozone/pull/1437#discussion_r492481300 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/cache/CacheStore.java ## @@ -0,0 +1,72 @@ +/** + * Licensed to

[GitHub] [hadoop-ozone] rakeshadr commented on a change in pull request #1437: HDDS-4222: [OzoneFS optimization] Provide a mechanism for efficient path lookup

2020-09-21 Thread GitBox
rakeshadr commented on a change in pull request #1437: URL: https://github.com/apache/hadoop-ozone/pull/1437#discussion_r492481338 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/cache/DirectoryLRUCacheStore.java ## @@ -0,0 +1,87 @@ +/** + *

[GitHub] [hadoop-ozone] rakeshadr commented on a change in pull request #1437: HDDS-4222: [OzoneFS optimization] Provide a mechanism for efficient path lookup

2020-09-21 Thread GitBox
rakeshadr commented on a change in pull request #1437: URL: https://github.com/apache/hadoop-ozone/pull/1437#discussion_r492480005 ## File path: hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/OMConfigKeys.java ## @@ -246,4 +246,15 @@ private OMConfigKeys() {

[GitHub] [hadoop-ozone] bshashikant commented on a change in pull request #1371: HDDS-2922. Balance ratis leader distribution in datanodes

2020-09-21 Thread GitBox
bshashikant commented on a change in pull request #1371: URL: https://github.com/apache/hadoop-ozone/pull/1371#discussion_r488543079 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/RatisPipelineProvider.java ## @@ -98,8 +105,65 @@

[GitHub] [hadoop-ozone] ChenSammi commented on a change in pull request #1412: HDDS-3751. Ozone sh client support bucket quota option.

2020-09-21 Thread GitBox
ChenSammi commented on a change in pull request #1412: URL: https://github.com/apache/hadoop-ozone/pull/1412#discussion_r492461120 ## File path: hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/BucketArgs.java ## @@ -66,17 +69,23 @@ * @param

[GitHub] [hadoop-ozone] github-actions[bot] commented on pull request #1110: HDDS-3843. Throw the specific exception other than NPE.

2020-09-21 Thread GitBox
github-actions[bot] commented on pull request #1110: URL: https://github.com/apache/hadoop-ozone/pull/1110#issuecomment-696453432 Thank you very much for the patch. I am closing this PR __temporarily__ as there was no activity recently and it is waiting for response from its author.

[GitHub] [hadoop-ozone] aryangupta1998 closed pull request #1439: HDDS-3290. Removing RandomKeyGenerator and TestRandomKeyGenerator.

2020-09-21 Thread GitBox
aryangupta1998 closed pull request #1439: URL: https://github.com/apache/hadoop-ozone/pull/1439 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hadoop-ozone] ChenSammi commented on a change in pull request #1412: HDDS-3751. Ozone sh client support bucket quota option.

2020-09-21 Thread GitBox
ChenSammi commented on a change in pull request #1412: URL: https://github.com/apache/hadoop-ozone/pull/1412#discussion_r492461120 ## File path: hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/BucketArgs.java ## @@ -66,17 +69,23 @@ * @param

[GitHub] [hadoop-ozone] github-actions[bot] closed pull request #1110: HDDS-3843. Throw the specific exception other than NPE.

2020-09-21 Thread GitBox
github-actions[bot] closed pull request #1110: URL: https://github.com/apache/hadoop-ozone/pull/1110 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [hadoop-ozone] elek commented on pull request #1374: HDDS-4185. Remove IncrementalByteBuffer from Ozone client

2020-09-21 Thread GitBox
elek commented on pull request #1374: URL: https://github.com/apache/hadoop-ozone/pull/1374#issuecomment-696169868 Had an offline call with @bshashikant This patch couldn't work with keySize=0 (where key size is unknown). The proposed solution is to adjust the size of

[GitHub] [hadoop-ozone] mukul1987 commented on pull request #1230: HDDS-2949: mkdir : store directory entries in a separate table

2020-09-21 Thread GitBox
mukul1987 commented on pull request #1230: URL: https://github.com/apache/hadoop-ozone/pull/1230#issuecomment-696297773 @rakeshadr Can we close this PR ? as there is a new one available ? This is an automated message from

[GitHub] [hadoop-ozone] rakeshadr commented on a change in pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

2020-09-21 Thread GitBox
rakeshadr commented on a change in pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#discussion_r491969762 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMFileRequest.java ## @@ -129,6 +133,123 @@ public

[GitHub] [hadoop-ozone] avijayanhwx commented on pull request #1432: HDDS-4252. Add the current layout versions to DN - SCM proto payload.

2020-09-21 Thread GitBox
avijayanhwx commented on pull request #1432: URL: https://github.com/apache/hadoop-ozone/pull/1432#issuecomment-696249716 @fapifta, @sodonnel Please review if you can. This is an automated message from the Apache Git

[GitHub] [hadoop-ozone] elek commented on a change in pull request #1421: HDDS-4174. Add current HDDS layout version to Datanode heartbeat/registration

2020-09-21 Thread GitBox
elek commented on a change in pull request #1421: URL: https://github.com/apache/hadoop-ozone/pull/1421#discussion_r491953506 ## File path: hadoop-hdds/interface-server/src/main/proto/ScmServerDatanodeHeartbeatProtocol.proto ## @@ -390,6 +404,14 @@ message

[GitHub] [hadoop-ozone] elek commented on a change in pull request #1419: HDDS-3755. [DESIGN] Storage-class for Ozone

2020-09-21 Thread GitBox
elek commented on a change in pull request #1419: URL: https://github.com/apache/hadoop-ozone/pull/1419#discussion_r492104772 ## File path: hadoop-hdds/docs/content/design/storage-class.md ## @@ -19,10 +19,331 @@ author: Marton Ele See the License for the specific language

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1340: HDDS-3188 Add failover proxy for SCM block location.

2020-09-21 Thread GitBox
xiaoyuyao commented on a change in pull request #1340: URL: https://github.com/apache/hadoop-ozone/pull/1340#discussion_r492294434 ## File path: hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/proxy/SCMClientConfig.java ## @@ -0,0 +1,107 @@ +/* + * Licensed to

[GitHub] [hadoop-ozone] rakeshadr commented on pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

2020-09-21 Thread GitBox
rakeshadr commented on pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#issuecomment-696143619 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hadoop-ozone] arp7 commented on pull request #1419: HDDS-3755. [DESIGN] Storage-class for Ozone

2020-09-21 Thread GitBox
arp7 commented on pull request #1419: URL: https://github.com/apache/hadoop-ozone/pull/1419#issuecomment-696202701 Isn't this design doc a few steps away from coding? A ton of detail is missing around how SCM will manage multiple classes of pipelines and how replication manager will need

[GitHub] [hadoop-ozone] nandakumar131 commented on a change in pull request #1423: HDDS-4244. Container deleted wrong replica cause mis-replicated.

2020-09-21 Thread GitBox
nandakumar131 commented on a change in pull request #1423: URL: https://github.com/apache/hadoop-ozone/pull/1423#discussion_r49199 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ReplicationManager.java ## @@ -660,21 +660,23 @@

[GitHub] [hadoop-ozone] avijayanhwx commented on a change in pull request #1432: HDDS-4252. Add the current layout versions to DN - SCM proto payload.

2020-09-21 Thread GitBox
avijayanhwx commented on a change in pull request #1432: URL: https://github.com/apache/hadoop-ozone/pull/1432#discussion_r492210006 ## File path: hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/DataNodeStorageConfig.java ## @@ -0,0 +1,56

[GitHub] [hadoop-ozone] mukul1987 closed pull request #1230: HDDS-2949: mkdir : store directory entries in a separate table

2020-09-21 Thread GitBox
mukul1987 closed pull request #1230: URL: https://github.com/apache/hadoop-ozone/pull/1230 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hadoop-ozone] nandakumar131 commented on pull request #1423: HDDS-4244. Container deleted wrong replica cause mis-replicated.

2020-09-21 Thread GitBox
nandakumar131 commented on pull request #1423: URL: https://github.com/apache/hadoop-ozone/pull/1423#issuecomment-696007538 How exactly does the new test cases verify that the ReplicationManager is considering Rack Awareness and picks the correct replica for deletion? As far as I can

[GitHub] [hadoop-ozone] bharatviswa504 commented on pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

2020-09-21 Thread GitBox
bharatviswa504 commented on pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#issuecomment-696243290 > > I see we used layOutVersion and tested the change. But this might cause failures to other test like when key and directory with same name, as we have not used

[GitHub] [hadoop-ozone] linyiqun commented on a change in pull request #1437: HDDS-4222: [OzoneFS optimization] Provide a mechanism for efficient path lookup

2020-09-21 Thread GitBox
linyiqun commented on a change in pull request #1437: URL: https://github.com/apache/hadoop-ozone/pull/1437#discussion_r492061634 ## File path: hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/OMConfigKeys.java ## @@ -246,4 +246,15 @@ private OMConfigKeys() {

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

2020-09-21 Thread GitBox
bharatviswa504 commented on a change in pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#discussion_r492215834 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMFileRequest.java ## @@ -156,6 +277,71 @@

[GitHub] [hadoop-ozone] elek commented on pull request #1272: HDDS-2660. Create insight point for datanode container protocol

2020-09-21 Thread GitBox
elek commented on pull request #1272: URL: https://github.com/apache/hadoop-ozone/pull/1272#issuecomment-696158960 > Thanks @elek for updating the patch. Interestingly log -v stopped working, even if I execute log first (as mentioned previously). I double-checked and it worked for

[GitHub] [hadoop-ozone] elek commented on pull request #1419: HDDS-3755. [DESIGN] Storage-class for Ozone

2020-09-21 Thread GitBox
elek commented on pull request #1419: URL: https://github.com/apache/hadoop-ozone/pull/1419#issuecomment-696168676 @umamaheswararao @arp7 Do you have any more comments / questions? If, not, I would suggest to start to create smaller patches from #1208 and merge them one by one...

[GitHub] [hadoop-ozone] runzhiwang commented on a change in pull request #1371: HDDS-2922. Balance ratis leader distribution in datanodes

2020-09-21 Thread GitBox
runzhiwang commented on a change in pull request #1371: URL: https://github.com/apache/hadoop-ozone/pull/1371#discussion_r492416925 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/RatisPipelineProvider.java ## @@ -98,8 +105,65 @@ private

[GitHub] [hadoop-ozone] ChenSammi commented on pull request #1338: HDDS-4023. Delete closed container after all blocks have been deleted.

2020-09-21 Thread GitBox
ChenSammi commented on pull request #1338: URL: https://github.com/apache/hadoop-ozone/pull/1338#issuecomment-695883294 @sodonnel @elek could you help to take another look? This is an automated message from the Apache Git

[GitHub] [hadoop-ozone] arp7 edited a comment on pull request #1419: HDDS-3755. [DESIGN] Storage-class for Ozone

2020-09-21 Thread GitBox
arp7 edited a comment on pull request #1419: URL: https://github.com/apache/hadoop-ozone/pull/1419#issuecomment-696202701 Isn't this design doc a few steps away from coding? A ton of detail is missing around how SCM will manage multiple classes of pipelines and how replication manager

[GitHub] [hadoop-ozone] elek closed pull request #1374: HDDS-4185. Remove IncrementalByteBuffer from Ozone client

2020-09-21 Thread GitBox
elek closed pull request #1374: URL: https://github.com/apache/hadoop-ozone/pull/1374 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [hadoop-ozone] github-actions[bot] closed pull request #1110: HDDS-3843. Throw the specific exception other than NPE.

2020-09-21 Thread GitBox
github-actions[bot] closed pull request #1110: URL: https://github.com/apache/hadoop-ozone/pull/1110 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [hadoop-ozone] github-actions[bot] commented on pull request #1110: HDDS-3843. Throw the specific exception other than NPE.

2020-09-21 Thread GitBox
github-actions[bot] commented on pull request #1110: URL: https://github.com/apache/hadoop-ozone/pull/1110#issuecomment-696453432 Thank you very much for the patch. I am closing this PR __temporarily__ as there was no activity recently and it is waiting for response from its author.

[GitHub] [hadoop-ozone] runzhiwang commented on a change in pull request #1371: HDDS-2922. Balance ratis leader distribution in datanodes

2020-09-21 Thread GitBox
runzhiwang commented on a change in pull request #1371: URL: https://github.com/apache/hadoop-ozone/pull/1371#discussion_r492416925 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/RatisPipelineProvider.java ## @@ -98,8 +105,65 @@ private

[GitHub] [hadoop-ozone] runzhiwang commented on a change in pull request #1371: HDDS-2922. Balance ratis leader distribution in datanodes

2020-09-21 Thread GitBox
runzhiwang commented on a change in pull request #1371: URL: https://github.com/apache/hadoop-ozone/pull/1371#discussion_r492416925 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/RatisPipelineProvider.java ## @@ -98,8 +105,65 @@ private

[GitHub] [hadoop-ozone] runzhiwang commented on a change in pull request #1371: HDDS-2922. Balance ratis leader distribution in datanodes

2020-09-21 Thread GitBox
runzhiwang commented on a change in pull request #1371: URL: https://github.com/apache/hadoop-ozone/pull/1371#discussion_r492416925 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/RatisPipelineProvider.java ## @@ -98,8 +105,65 @@ private

[GitHub] [hadoop-ozone] runzhiwang commented on a change in pull request #1371: HDDS-2922. Balance ratis leader distribution in datanodes

2020-09-21 Thread GitBox
runzhiwang commented on a change in pull request #1371: URL: https://github.com/apache/hadoop-ozone/pull/1371#discussion_r492416925 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/RatisPipelineProvider.java ## @@ -98,8 +105,65 @@ private

[jira] [Commented] (HDDS-541) Ozone Quota support.

2020-09-21 Thread Rui Wang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199638#comment-17199638 ] Rui Wang commented on HDDS-541: --- [~micahzhao] Do you mind me helping on this JIRA? I am seeing lots of open

[jira] [Commented] (HDDS-3091) Assign Object ID from OMKeyInfo in PrefixManagerImpl.addAcl and setAcl

2020-09-21 Thread Rui Wang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199624#comment-17199624 ] Rui Wang commented on HDDS-3091: [~sdeka] do you have any sub-tasks under HDDS-2939 that you think I can

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1340: HDDS-3188 Add failover proxy for SCM block location.

2020-09-21 Thread GitBox
xiaoyuyao commented on a change in pull request #1340: URL: https://github.com/apache/hadoop-ozone/pull/1340#discussion_r492298148 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/protocol/ScmBlockLocationProtocolServerSideTranslatorPB.java ## @@

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1340: HDDS-3188 Add failover proxy for SCM block location.

2020-09-21 Thread GitBox
xiaoyuyao commented on a change in pull request #1340: URL: https://github.com/apache/hadoop-ozone/pull/1340#discussion_r492296581 ## File path: hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/proxy/SCMProxyInfo.java ## @@ -0,0 +1,65 @@ +/** + * Licensed to the

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1340: HDDS-3188 Add failover proxy for SCM block location.

2020-09-21 Thread GitBox
xiaoyuyao commented on a change in pull request #1340: URL: https://github.com/apache/hadoop-ozone/pull/1340#discussion_r492295789 ## File path: hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/proxy/SCMClientConfig.java ## @@ -0,0 +1,107 @@ +/* + * Licensed to

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1340: HDDS-3188 Add failover proxy for SCM block location.

2020-09-21 Thread GitBox
xiaoyuyao commented on a change in pull request #1340: URL: https://github.com/apache/hadoop-ozone/pull/1340#discussion_r492295311 ## File path: hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/proxy/SCMClientConfig.java ## @@ -0,0 +1,107 @@ +/* + * Licensed to

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1340: HDDS-3188 Add failover proxy for SCM block location.

2020-09-21 Thread GitBox
xiaoyuyao commented on a change in pull request #1340: URL: https://github.com/apache/hadoop-ozone/pull/1340#discussion_r492294696 ## File path: hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/proxy/SCMClientConfig.java ## @@ -0,0 +1,107 @@ +/* + * Licensed to

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1340: HDDS-3188 Add failover proxy for SCM block location.

2020-09-21 Thread GitBox
xiaoyuyao commented on a change in pull request #1340: URL: https://github.com/apache/hadoop-ozone/pull/1340#discussion_r492294434 ## File path: hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/proxy/SCMClientConfig.java ## @@ -0,0 +1,107 @@ +/* + * Licensed to

[GitHub] [hadoop-ozone] mukul1987 commented on pull request #1230: HDDS-2949: mkdir : store directory entries in a separate table

2020-09-21 Thread GitBox
mukul1987 commented on pull request #1230: URL: https://github.com/apache/hadoop-ozone/pull/1230#issuecomment-696297773 @rakeshadr Can we close this PR ? as there is a new one available ? This is an automated message from

[GitHub] [hadoop-ozone] mukul1987 closed pull request #1230: HDDS-2949: mkdir : store directory entries in a separate table

2020-09-21 Thread GitBox
mukul1987 closed pull request #1230: URL: https://github.com/apache/hadoop-ozone/pull/1230 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hadoop-ozone] rakeshadr commented on pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

2020-09-21 Thread GitBox
rakeshadr commented on pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#issuecomment-696275956 I've updated another patch addressing @bharatviswa504's comments. This is an automated message from the

[GitHub] [hadoop-ozone] rakeshadr commented on a change in pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

2020-09-21 Thread GitBox
rakeshadr commented on a change in pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#discussion_r492242294 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMFileRequest.java ## @@ -156,6 +277,71 @@ public

[GitHub] [hadoop-ozone] avijayanhwx commented on a change in pull request #1432: HDDS-4252. Add the current layout versions to DN - SCM proto payload.

2020-09-21 Thread GitBox
avijayanhwx commented on a change in pull request #1432: URL: https://github.com/apache/hadoop-ozone/pull/1432#discussion_r492210006 ## File path: hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/DataNodeStorageConfig.java ## @@ -0,0 +1,56

[GitHub] [hadoop-ozone] avijayanhwx commented on pull request #1432: HDDS-4252. Add the current layout versions to DN - SCM proto payload.

2020-09-21 Thread GitBox
avijayanhwx commented on pull request #1432: URL: https://github.com/apache/hadoop-ozone/pull/1432#issuecomment-696249716 @fapifta, @sodonnel Please review if you can. This is an automated message from the Apache Git

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1371: HDDS-2922. Balance ratis leader distribution in datanodes

2020-09-21 Thread GitBox
xiaoyuyao commented on a change in pull request #1371: URL: https://github.com/apache/hadoop-ozone/pull/1371#discussion_r492218302 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/RatisPipelineProvider.java ## @@ -98,8 +105,65 @@ private

[GitHub] [hadoop-ozone] rakeshadr commented on pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

2020-09-21 Thread GitBox
rakeshadr commented on pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#issuecomment-696247887 > As any way we are testing at UT, we can add the entries to KeyTable and implement/simulate these tests. (Similar to Old Requests) Sure, will do that way.

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

2020-09-21 Thread GitBox
bharatviswa504 commented on a change in pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#discussion_r492215834 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMFileRequest.java ## @@ -156,6 +277,71 @@

[GitHub] [hadoop-ozone] rakeshadr commented on pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

2020-09-21 Thread GitBox
rakeshadr commented on pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#issuecomment-696245686 Thanks a lot @bharatviswa504 for the useful review comments and your time. I will upstream next commit.

[GitHub] [hadoop-ozone] rakeshadr commented on a change in pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

2020-09-21 Thread GitBox
rakeshadr commented on a change in pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#discussion_r492213811 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMFileRequest.java ## @@ -156,6 +277,71 @@ public

[GitHub] [hadoop-ozone] bharatviswa504 edited a comment on pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

2020-09-21 Thread GitBox
bharatviswa504 edited a comment on pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#issuecomment-696243290 > > I see we used layOutVersion and tested the change. But this might cause failures to other test like when key and directory with same name, as we have not

[GitHub] [hadoop-ozone] bharatviswa504 commented on pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

2020-09-21 Thread GitBox
bharatviswa504 commented on pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#issuecomment-696243290 > > I see we used layOutVersion and tested the change. But this might cause failures to other test like when key and directory with same name, as we have not used

[GitHub] [hadoop-ozone] rakeshadr commented on a change in pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

2020-09-21 Thread GitBox
rakeshadr commented on a change in pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#discussion_r492205581 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMFileRequest.java ## @@ -129,6 +133,123 @@ public

[GitHub] [hadoop-ozone] rakeshadr commented on a change in pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

2020-09-21 Thread GitBox
rakeshadr commented on a change in pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#discussion_r492185398 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMFileRequest.java ## @@ -254,4 +440,34 @@ public

[GitHub] [hadoop-ozone] arp7 edited a comment on pull request #1419: HDDS-3755. [DESIGN] Storage-class for Ozone

2020-09-21 Thread GitBox
arp7 edited a comment on pull request #1419: URL: https://github.com/apache/hadoop-ozone/pull/1419#issuecomment-696202701 Isn't this design doc a few steps away from coding? A ton of detail is missing around how SCM will manage multiple classes of pipelines and how replication manager

[GitHub] [hadoop-ozone] arp7 commented on pull request #1419: HDDS-3755. [DESIGN] Storage-class for Ozone

2020-09-21 Thread GitBox
arp7 commented on pull request #1419: URL: https://github.com/apache/hadoop-ozone/pull/1419#issuecomment-696202701 Isn't this design doc a few steps away from coding? A ton of detail is missing around how SCM will manage multiple classes of pipelines and how replication manager will need

[GitHub] [hadoop-ozone] elek closed pull request #1374: HDDS-4185. Remove IncrementalByteBuffer from Ozone client

2020-09-21 Thread GitBox
elek closed pull request #1374: URL: https://github.com/apache/hadoop-ozone/pull/1374 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [hadoop-ozone] elek commented on pull request #1374: HDDS-4185. Remove IncrementalByteBuffer from Ozone client

2020-09-21 Thread GitBox
elek commented on pull request #1374: URL: https://github.com/apache/hadoop-ozone/pull/1374#issuecomment-696169868 Had an offline call with @bshashikant This patch couldn't work with keySize=0 (where key size is unknown). The proposed solution is to adjust the size of

[GitHub] [hadoop-ozone] elek commented on pull request #1419: HDDS-3755. [DESIGN] Storage-class for Ozone

2020-09-21 Thread GitBox
elek commented on pull request #1419: URL: https://github.com/apache/hadoop-ozone/pull/1419#issuecomment-696168676 @umamaheswararao @arp7 Do you have any more comments / questions? If, not, I would suggest to start to create smaller patches from #1208 and merge them one by one...

[GitHub] [hadoop-ozone] elek commented on a change in pull request #1419: HDDS-3755. [DESIGN] Storage-class for Ozone

2020-09-21 Thread GitBox
elek commented on a change in pull request #1419: URL: https://github.com/apache/hadoop-ozone/pull/1419#discussion_r492111327 ## File path: hadoop-hdds/docs/content/design/storage-class.md ## @@ -19,10 +19,331 @@ author: Marton Ele See the License for the specific language

[GitHub] [hadoop-ozone] elek commented on a change in pull request #1419: HDDS-3755. [DESIGN] Storage-class for Ozone

2020-09-21 Thread GitBox
elek commented on a change in pull request #1419: URL: https://github.com/apache/hadoop-ozone/pull/1419#discussion_r492110141 ## File path: hadoop-hdds/docs/content/design/storage-class.md ## @@ -19,10 +19,331 @@ author: Marton Ele See the License for the specific language

[GitHub] [hadoop-ozone] elek commented on a change in pull request #1419: HDDS-3755. [DESIGN] Storage-class for Ozone

2020-09-21 Thread GitBox
elek commented on a change in pull request #1419: URL: https://github.com/apache/hadoop-ozone/pull/1419#discussion_r492108250 ## File path: hadoop-hdds/docs/content/design/storage-class.md ## @@ -19,10 +19,331 @@ author: Marton Ele See the License for the specific language

[GitHub] [hadoop-ozone] elek commented on a change in pull request #1419: HDDS-3755. [DESIGN] Storage-class for Ozone

2020-09-21 Thread GitBox
elek commented on a change in pull request #1419: URL: https://github.com/apache/hadoop-ozone/pull/1419#discussion_r492106886 ## File path: hadoop-hdds/docs/content/design/storage-class.md ## @@ -19,10 +19,331 @@ author: Marton Ele See the License for the specific language

[GitHub] [hadoop-ozone] elek commented on a change in pull request #1419: HDDS-3755. [DESIGN] Storage-class for Ozone

2020-09-21 Thread GitBox
elek commented on a change in pull request #1419: URL: https://github.com/apache/hadoop-ozone/pull/1419#discussion_r492104772 ## File path: hadoop-hdds/docs/content/design/storage-class.md ## @@ -19,10 +19,331 @@ author: Marton Ele See the License for the specific language

[GitHub] [hadoop-ozone] elek commented on pull request #1272: HDDS-2660. Create insight point for datanode container protocol

2020-09-21 Thread GitBox
elek commented on pull request #1272: URL: https://github.com/apache/hadoop-ozone/pull/1272#issuecomment-696158960 > Thanks @elek for updating the patch. Interestingly log -v stopped working, even if I execute log first (as mentioned previously). I double-checked and it worked for

[jira] [Updated] (HDDS-2011) TestRandomKeyGenerator fails due to timeout

2020-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-2011: - Labels: pull-request-available (was: ) > TestRandomKeyGenerator fails due to timeout >

[GitHub] [hadoop-ozone] aryangupta1998 opened a new pull request #1440: HDDS-2011. Enable TestRandomKeyGenerator Test Cases.

2020-09-21 Thread GitBox
aryangupta1998 opened a new pull request #1440: URL: https://github.com/apache/hadoop-ozone/pull/1440 ## What changes were proposed in this pull request? Enable TestRandomKeyGenerator Test Cases. ## What is the link to the Apache JIRA

[GitHub] [hadoop-ozone] linyiqun commented on a change in pull request #1437: HDDS-4222: [OzoneFS optimization] Provide a mechanism for efficient path lookup

2020-09-21 Thread GitBox
linyiqun commented on a change in pull request #1437: URL: https://github.com/apache/hadoop-ozone/pull/1437#discussion_r492061634 ## File path: hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/OMConfigKeys.java ## @@ -246,4 +246,15 @@ private OMConfigKeys() {

[GitHub] [hadoop-ozone] rakeshadr commented on pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

2020-09-21 Thread GitBox
rakeshadr commented on pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#issuecomment-696143619 > I see we used layOutVersion and tested the change. But this might cause failures to other test like when key and directory with same name, as we have not used new

[GitHub] [hadoop-ozone] aryangupta1998 closed pull request #1439: HDDS-3290. Removing RandomKeyGenerator and TestRandomKeyGenerator.

2020-09-21 Thread GitBox
aryangupta1998 closed pull request #1439: URL: https://github.com/apache/hadoop-ozone/pull/1439 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[jira] [Updated] (HDDS-3290) Remove deprecated RandomKeyGenerator

2020-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-3290: - Labels: TriagePending pull-request-available (was: TriagePending) > Remove deprecated

[GitHub] [hadoop-ozone] aryangupta1998 opened a new pull request #1439: HDDS-3290. Removing RandomKeyGenerator and TestRandomKeyGenerator.

2020-09-21 Thread GitBox
aryangupta1998 opened a new pull request #1439: URL: https://github.com/apache/hadoop-ozone/pull/1439 ## What changes were proposed in this pull request? Removing RandomKeyGenerator and TestRandomKeyGenerator. We already have key generators with the new structure. Hence, I think we

[GitHub] [hadoop-ozone] rakeshadr commented on a change in pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

2020-09-21 Thread GitBox
rakeshadr commented on a change in pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#discussion_r491976839 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMDirectoryCreateRequestV1.java ## @@ -0,0 +1,312

[GitHub] [hadoop-ozone] rakeshadr commented on a change in pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

2020-09-21 Thread GitBox
rakeshadr commented on a change in pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#discussion_r491975260 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMDirectoryCreateRequestV1.java ## @@ -0,0 +1,312

[jira] [Assigned] (HDDS-2011) TestRandomKeyGenerator fails due to timeout

2020-09-21 Thread Aryan Gupta (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aryan Gupta reassigned HDDS-2011: - Assignee: Aryan Gupta > TestRandomKeyGenerator fails due to timeout >

[GitHub] [hadoop-ozone] rakeshadr commented on a change in pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

2020-09-21 Thread GitBox
rakeshadr commented on a change in pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#discussion_r491970938 ## File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneDirectory.java ## @@ -0,0 +1,200 @@ +/* + *

[GitHub] [hadoop-ozone] rakeshadr commented on a change in pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

2020-09-21 Thread GitBox
rakeshadr commented on a change in pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#discussion_r491970625 ## File path: hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmDirectoryInfo.java ## @@ -0,0 +1,269 @@ +/** + * Licensed to

[GitHub] [hadoop-ozone] rakeshadr commented on a change in pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

2020-09-21 Thread GitBox
rakeshadr commented on a change in pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#discussion_r491970382 ## File path: hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmDirectoryInfo.java ## @@ -0,0 +1,269 @@ +/** + * Licensed to

[GitHub] [hadoop-ozone] rakeshadr commented on a change in pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

2020-09-21 Thread GitBox
rakeshadr commented on a change in pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#discussion_r491970097 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMFileRequest.java ## @@ -129,6 +133,123 @@ public

[GitHub] [hadoop-ozone] rakeshadr commented on a change in pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

2020-09-21 Thread GitBox
rakeshadr commented on a change in pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#discussion_r491970254 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMFileRequest.java ## @@ -129,6 +133,123 @@ public

[GitHub] [hadoop-ozone] rakeshadr commented on a change in pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

2020-09-21 Thread GitBox
rakeshadr commented on a change in pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#discussion_r491969762 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMFileRequest.java ## @@ -129,6 +133,123 @@ public

[GitHub] [hadoop-ozone] elek commented on a change in pull request #1421: HDDS-4174. Add current HDDS layout version to Datanode heartbeat/registration

2020-09-21 Thread GitBox
elek commented on a change in pull request #1421: URL: https://github.com/apache/hadoop-ozone/pull/1421#discussion_r491953506 ## File path: hadoop-hdds/interface-server/src/main/proto/ScmServerDatanodeHeartbeatProtocol.proto ## @@ -390,6 +404,14 @@ message

[jira] [Updated] (HDDS-4263) ReplicatiomManager shouldn't retain one healthy replica per origin node Id.

2020-09-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-4263: - Labels: pull-request-available (was: ) > ReplicatiomManager shouldn't retain one healthy replica

[GitHub] [hadoop-ozone] maobaolong opened a new pull request #1438: HDDS-4263. ReplicatiomManager shouldn't retain one healthy replica per origin node Id.

2020-09-21 Thread GitBox
maobaolong opened a new pull request #1438: URL: https://github.com/apache/hadoop-ozone/pull/1438 ## What changes were proposed in this pull request? ReplicatiomManager shouldn't retain one healthy replica per origin node Id. ## What is the link to the Apache JIRA

[GitHub] [hadoop-ozone] nandakumar131 commented on pull request #1423: HDDS-4244. Container deleted wrong replica cause mis-replicated.

2020-09-21 Thread GitBox
nandakumar131 commented on pull request #1423: URL: https://github.com/apache/hadoop-ozone/pull/1423#issuecomment-696007538 How exactly does the new test cases verify that the ReplicationManager is considering Rack Awareness and picks the correct replica for deletion? As far as I can

[jira] [Updated] (HDDS-4239) Ozone support truncate operation

2020-09-21 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-4239: - Attachment: (was: Ozone Truncate Design-v3.pdf) > Ozone support truncate operation >

[jira] [Updated] (HDDS-4239) Ozone support truncate operation

2020-09-21 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-4239: - Attachment: Ozone Truncate Design.pdf > Ozone support truncate operation >

[jira] [Updated] (HDDS-4239) Ozone support truncate operation

2020-09-21 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-4239: - Attachment: (was: Ozone Truncate Design-v2.pdf) > Ozone support truncate operation >

[jira] [Updated] (HDDS-4239) Ozone support truncate operation

2020-09-21 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-4239: - Attachment: (was: Ozone Truncate Design-v1.pdf) > Ozone support truncate operation >

[GitHub] [hadoop-ozone] nandakumar131 commented on a change in pull request #1423: HDDS-4244. Container deleted wrong replica cause mis-replicated.

2020-09-21 Thread GitBox
nandakumar131 commented on a change in pull request #1423: URL: https://github.com/apache/hadoop-ozone/pull/1423#discussion_r49199 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ReplicationManager.java ## @@ -660,21 +660,23 @@

[jira] [Commented] (HDDS-4244) container deleted wrong replica cause mis-replicated

2020-09-21 Thread Nanda kumar (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199272#comment-17199272 ] Nanda kumar commented on HDDS-4244: --- Please add a proper description to the jira so that people looking

[jira] [Created] (HDDS-4263) ReplicatiomManager shouldn't retain one healthy replica per origin node Id.

2020-09-21 Thread maobaolong (Jira)
maobaolong created HDDS-4263: Summary: ReplicatiomManager shouldn't retain one healthy replica per origin node Id. Key: HDDS-4263 URL: https://issues.apache.org/jira/browse/HDDS-4263 Project: Hadoop