[GitHub] [hadoop-ozone] elek commented on pull request #1396: HDDS-4150. recon.api.TestEndpoints test is flaky

2020-09-11 Thread GitBox
elek commented on pull request #1396: URL: https://github.com/apache/hadoop-ozone/pull/1396#issuecomment-691426288 Sorry, I am late in the game to review it, but it shouldn't be too late to say: thanks a lot to fix this flakiness. ---

[jira] [Commented] (HDDS-4209) S3A Filesystem does not work with Ozone S3

2020-09-11 Thread Bharat Viswanadham (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17194562#comment-17194562 ] Bharat Viswanadham commented on HDDS-4209: -- Posted a draft PR, when discussing wi

[jira] [Updated] (HDDS-4209) S3A Filesystem does not work with Ozone S3

2020-09-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-4209: - Labels: pull-request-available (was: ) > S3A Filesystem does not work with Ozone S3 > ---

[GitHub] [hadoop-ozone] bharatviswa504 opened a new pull request #1418: HDDS-4209. S3A Filesystem does not work with Ozone S3.

2020-09-11 Thread GitBox
bharatviswa504 opened a new pull request #1418: URL: https://github.com/apache/hadoop-ozone/pull/1418 ## What changes were proposed in this pull request? When Key is 0 bytes, and ending with "/" call createDirectory API. In this way below scenario can work from S3A. ``` hdf

[jira] [Created] (HDDS-4235) Ozone client FS path validation is not present in OFS.

2020-09-11 Thread Aravindan Vijayan (Jira)
Aravindan Vijayan created HDDS-4235: --- Summary: Ozone client FS path validation is not present in OFS. Key: HDDS-4235 URL: https://issues.apache.org/jira/browse/HDDS-4235 Project: Hadoop Distributed D

[GitHub] [hadoop-ozone] avijayanhwx commented on pull request #1405: HDDS-4143. Implement a factory for OM Requests that returns an instance based on layout version.

2020-09-11 Thread GitBox
avijayanhwx commented on pull request #1405: URL: https://github.com/apache/hadoop-ozone/pull/1405#issuecomment-691330372 > Hi @avijayanhwx, thank you for working on this. > Please find some comments below. > > > First and foremost I would like to comment on the LayoutVersionI

[GitHub] [hadoop-ozone] amaliujia commented on pull request #1410: HDDS-3102. ozone getconf command should use the GenericCli parent class

2020-09-11 Thread GitBox
amaliujia commented on pull request #1410: URL: https://github.com/apache/hadoop-ozone/pull/1410#issuecomment-691324990 note that because the `getconf.robot` is under `basic` directory, it will be picked up by `compose/ozone/test.sh` ---

[GitHub] [hadoop-ozone] amaliujia commented on pull request #1410: HDDS-3102. ozone getconf command should use the GenericCli parent class

2020-09-11 Thread GitBox
amaliujia commented on pull request #1410: URL: https://github.com/apache/hadoop-ozone/pull/1410#issuecomment-691320364 @elek Thanks for your help! I have added a robot test. Can you take a look please? This is an aut

[jira] [Commented] (HDDS-4209) S3A Filesystem does not work with Ozone S3

2020-09-11 Thread Bharat Viswanadham (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17194513#comment-17194513 ] Bharat Viswanadham commented on HDDS-4209: -- Looked in to S3A code, when directory

[jira] [Comment Edited] (HDDS-4209) S3A Filesystem does not work with Ozone S3

2020-09-11 Thread Bharat Viswanadham (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17193033#comment-17193033 ] Bharat Viswanadham edited comment on HDDS-4209 at 9/11/20, 8:16 PM:

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #1361: HDDS-4155. Directory and filename can end up with same name in a path.

2020-09-11 Thread GitBox
bharatviswa504 commented on a change in pull request #1361: URL: https://github.com/apache/hadoop-ozone/pull/1361#discussion_r487246329 ## File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneFSWithObjectStoreCreate.java ## @@ -303,6 +304,

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #1361: HDDS-4155. Directory and filename can end up with same name in a path.

2020-09-11 Thread GitBox
bharatviswa504 commented on a change in pull request #1361: URL: https://github.com/apache/hadoop-ozone/pull/1361#discussion_r487246329 ## File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneFSWithObjectStoreCreate.java ## @@ -303,6 +304,

[GitHub] [hadoop-ozone] amaliujia commented on a change in pull request #1415: HDDS-4232. Use single thread for KeyDeletingService.

2020-09-11 Thread GitBox
amaliujia commented on a change in pull request #1415: URL: https://github.com/apache/hadoop-ozone/pull/1415#discussion_r487185841 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyDeletingService.java ## @@ -66,9 +66,6 @@ private static f

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #1398: HDDS-4210. ResolveBucket during checkAcls fails.

2020-09-11 Thread GitBox
bharatviswa504 commented on a change in pull request #1398: URL: https://github.com/apache/hadoop-ozone/pull/1398#discussion_r487181977 ## File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOzoneManagerHAWithData.java ## @@ -69,6 +70,38 @@ pu

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #1398: HDDS-4210. ResolveBucket during checkAcls fails.

2020-09-11 Thread GitBox
bharatviswa504 commented on a change in pull request #1398: URL: https://github.com/apache/hadoop-ozone/pull/1398#discussion_r487178258 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java ## @@ -3523,6 +3533,47 @@ public Resolved

[GitHub] [hadoop-ozone] timmylicheng closed pull request #1340: HDDS-3188 Add failover proxy for SCM block location.

2020-09-11 Thread GitBox
timmylicheng closed pull request #1340: URL: https://github.com/apache/hadoop-ozone/pull/1340 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[jira] [Resolved] (HDDS-4119) Improve performance of the BufferPool management of Ozone client

2020-09-11 Thread Marton Elek (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Elek resolved HDDS-4119. --- Fix Version/s: 1.1.0 Resolution: Fixed > Improve performance of the BufferPool management of Oz

[GitHub] [hadoop-ozone] elek merged pull request #1336: HDDS-4119. Improve performance of the BufferPool management of Ozone client

2020-09-11 Thread GitBox
elek merged pull request #1336: URL: https://github.com/apache/hadoop-ozone/pull/1336 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [hadoop-ozone] elek commented on pull request #1336: HDDS-4119. Improve performance of the BufferPool management of Ozone client

2020-09-11 Thread GitBox
elek commented on pull request #1336: URL: https://github.com/apache/hadoop-ozone/pull/1336#issuecomment-691125675 Thanks the review @bshashikant and @adoroszlai I am merging it now... This is an automated message fro

[jira] [Commented] (HDDS-4222) [OzoneFS optimization] Provide a mechanism for efficient path lookup

2020-09-11 Thread Yiqun Lin (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17194239#comment-17194239 ] Yiqun Lin commented on HDDS-4222: - [~rakeshr] , thanks for the explanation, it's very clea

[jira] [Commented] (HDDS-4222) [OzoneFS optimization] Provide a mechanism for efficient path lookup

2020-09-11 Thread Rakesh Radhakrishnan (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17194221#comment-17194221 ] Rakesh Radhakrishnan commented on HDDS-4222: {quote}If we delete dir 512/a, it

[GitHub] [hadoop-ozone] lokeshj1703 commented on pull request #1415: HDDS-4232. Use single thread for KeyDeletingService and OpenKeyCleanupService.

2020-09-11 Thread GitBox
lokeshj1703 commented on pull request #1415: URL: https://github.com/apache/hadoop-ozone/pull/1415#issuecomment-690983128 > [HDDS-4120](https://issues.apache.org/jira/browse/HDDS-4120) This Jira changes the behavior of openKeyCleanup service, and does not call SCM directly, it moves the ke

[jira] [Updated] (HDDS-4232) Use single thread for KeyDeletingService

2020-09-11 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HDDS-4232: -- Description: KeyDeletingService scan the keys from a particular rocksdb table and sends deletion request

[jira] [Updated] (HDDS-4232) Use single thread for KeyDeletingService

2020-09-11 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HDDS-4232: -- Summary: Use single thread for KeyDeletingService (was: Use single thread for KeyDeletingService and Op

[GitHub] [hadoop-ozone] lokeshj1703 commented on a change in pull request #1415: HDDS-4232. Use single thread for KeyDeletingService and OpenKeyCleanupService.

2020-09-11 Thread GitBox
lokeshj1703 commented on a change in pull request #1415: URL: https://github.com/apache/hadoop-ozone/pull/1415#discussion_r486905798 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyDeletingService.java ## @@ -66,9 +66,6 @@ private static

[GitHub] [hadoop-ozone] bshashikant merged pull request #1408: HDDS-4217. Remove test TestOzoneContainerRatis

2020-09-11 Thread GitBox
bshashikant merged pull request #1408: URL: https://github.com/apache/hadoop-ozone/pull/1408 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[jira] [Resolved] (HDDS-4218) Remove test TestRatisManager

2020-09-11 Thread Shashikant Banerjee (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee resolved HDDS-4218. --- Fix Version/s: 1.1.0 Resolution: Fixed > Remove test TestRatisManager > ---

[jira] [Resolved] (HDDS-4217) Remove test TestOzoneContainerRatis

2020-09-11 Thread Shashikant Banerjee (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee resolved HDDS-4217. --- Fix Version/s: 1.1.0 Resolution: Fixed > Remove test TestOzoneContainerRatis >

[GitHub] [hadoop-ozone] bshashikant merged pull request #1409: HDDS-4218. Remove test TestRatisManager

2020-09-11 Thread GitBox
bshashikant merged pull request #1409: URL: https://github.com/apache/hadoop-ozone/pull/1409 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hadoop-ozone] lokeshj1703 commented on pull request #1415: HDDS-4232. Use single thread for KeyDeletingService and OpenKeyCleanupService.

2020-09-11 Thread GitBox
lokeshj1703 commented on pull request #1415: URL: https://github.com/apache/hadoop-ozone/pull/1415#issuecomment-690940801 The PR requires change in 1414 to address the test failure. This is an automated message from the Apach

[GitHub] [hadoop-ozone] lokeshj1703 commented on pull request #1415: HDDS-4232. Use single thread for KeyDeletingService and OpenKeyCleanupService.

2020-09-11 Thread GitBox
lokeshj1703 commented on pull request #1415: URL: https://github.com/apache/hadoop-ozone/pull/1415#issuecomment-690940004 /pending This is an automated message from the Apache Git Service. To respond to the message, please l

[GitHub] [hadoop-ozone] lokeshj1703 commented on pull request #1414: HDDS-4231. Background Service blocks on task results.

2020-09-11 Thread GitBox
lokeshj1703 commented on pull request #1414: URL: https://github.com/apache/hadoop-ozone/pull/1414#issuecomment-690939847 Pushed another commit to fix a unit test. This is an automated message from the Apache Git Service. To

[GitHub] [hadoop-ozone] lokeshj1703 commented on a change in pull request #1414: HDDS-4231. Background Service blocks on task results.

2020-09-11 Thread GitBox
lokeshj1703 commented on a change in pull request #1414: URL: https://github.com/apache/hadoop-ozone/pull/1414#discussion_r486841264 ## File path: hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/utils/BackgroundService.java ## @@ -62,11 +56,11 @@ public BackgroundServi

[GitHub] [hadoop-ozone] elek commented on pull request #1410: HDDS-3102. ozone getconf command should use the GenericCli parent class

2020-09-11 Thread GitBox
elek commented on pull request #1410: URL: https://github.com/apache/hadoop-ozone/pull/1410#issuecomment-690929244 The easiest way IMHO to test cli is adding a robot test. A simple example is here: https://github.com/apache/hadoop-ozone/blob/master/hadoop-ozone/dist/src/main/

[GitHub] [hadoop-ozone] lamber-ken removed a comment on pull request #1417: HDDS-4324. Add important comment to ListVolumes logic

2020-09-11 Thread GitBox
lamber-ken removed a comment on pull request #1417: URL: https://github.com/apache/hadoop-ozone/pull/1417#issuecomment-690928333 help This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [hadoop-ozone] lamber-ken commented on pull request #1417: HDDS-4324. Add important comment to ListVolumes logic

2020-09-11 Thread GitBox
lamber-ken commented on pull request #1417: URL: https://github.com/apache/hadoop-ozone/pull/1417#issuecomment-690928333 help This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [hadoop-ozone] ChenSammi merged pull request #1337: HDDS-4129. change MAX_QUOTA_IN_BYTES to Long.MAX_VALUE.

2020-09-11 Thread GitBox
ChenSammi merged pull request #1337: URL: https://github.com/apache/hadoop-ozone/pull/1337 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [hadoop-ozone] ChenSammi commented on pull request #1337: HDDS-4129. change MAX_QUOTA_IN_BYTES to Long.MAX_VALUE.

2020-09-11 Thread GitBox
ChenSammi commented on pull request #1337: URL: https://github.com/apache/hadoop-ozone/pull/1337#issuecomment-690919896 +1. After a thoroughly thinking, we choose a conservative way. Thanks @captainzmc for the contribution and @adoroszlai for the review. -