[GitHub] [hadoop-ozone] lokeshj1703 commented on pull request #1459: HDDS-4297. Allow multiple transactions per container to be sent for deletion by SCM.

2020-10-19 Thread GitBox
lokeshj1703 commented on pull request #1459: URL: https://github.com/apache/hadoop-ozone/pull/1459#issuecomment-712178445 @bshashikant @linyiqun Thanks for reviewing the PR! I have committed it to master branch. This is an a

[GitHub] [hadoop-ozone] linyiqun commented on pull request #1459: HDDS-4297. Allow multiple transactions per container to be sent for deletion by SCM.

2020-10-19 Thread GitBox
linyiqun commented on pull request #1459: URL: https://github.com/apache/hadoop-ozone/pull/1459#issuecomment-712021874 Good to know the testing result. +1 for this change. Thanks @lokeshj1703 . This is an automated message fr

[GitHub] [hadoop-ozone] lokeshj1703 commented on a change in pull request #1502: HDDS-4354. Avoid spamming of logs because of deleted transactions.

2020-10-19 Thread GitBox
lokeshj1703 commented on a change in pull request #1502: URL: https://github.com/apache/hadoop-ozone/pull/1502#discussion_r507591562 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/block/DeletedBlockLogImpl.java ## @@ -130,9 +130,7 @@ public void

[GitHub] [hadoop-ozone] cxorm commented on pull request #1301: HDDS-3882. Update modification time when updating volume/bucket/key ACLs

2020-10-19 Thread GitBox
cxorm commented on pull request #1301: URL: https://github.com/apache/hadoop-ozone/pull/1301#issuecomment-711871349 > Thanks @cxorm for the update. The latest PR LGTM overall, a few comments added inline. Thank you @xiaoyuyao for the careful review. The fix was added to addressed

[GitHub] [hadoop-ozone] lokeshj1703 commented on pull request #1459: HDDS-4297. Allow multiple transactions per container to be sent for deletion by SCM.

2020-10-19 Thread GitBox
lokeshj1703 commented on pull request #1459: URL: https://github.com/apache/hadoop-ozone/pull/1459#issuecomment-711778460 @linyiqun currently we allow block deletion for only closed containers. For open containers deletion needs to be done via ratis pipeline which is not currently implemen

[GitHub] [hadoop-ozone] aryangupta1998 closed pull request #1440: HDDS-2011. Enable TestRandomKeyGenerator Test Cases.

2020-10-18 Thread GitBox
aryangupta1998 closed pull request #1440: URL: https://github.com/apache/hadoop-ozone/pull/1440 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [hadoop-ozone] runzhiwang commented on pull request #1371: HDDS-2922. Balance ratis leader distribution in datanodes

2020-10-18 Thread GitBox
runzhiwang commented on pull request #1371: URL: https://github.com/apache/hadoop-ozone/pull/1371#issuecomment-711495208 @ChenSammi Thanks for merging it, @bshashikant @xiaoyuyao @GlenGeng Thanks for review. This is an autom

[GitHub] [hadoop-ozone] ChenSammi commented on pull request #1371: HDDS-2922. Balance ratis leader distribution in datanodes

2020-10-18 Thread GitBox
ChenSammi commented on pull request #1371: URL: https://github.com/apache/hadoop-ozone/pull/1371#issuecomment-711494972 LGTM + 1. Thanks @runzhiwang for the contribution and @bshashikant @xiaoyuyao @GlenGeng for the review.

[GitHub] [hadoop-ozone] ChenSammi merged pull request #1371: HDDS-2922. Balance ratis leader distribution in datanodes

2020-10-18 Thread GitBox
ChenSammi merged pull request #1371: URL: https://github.com/apache/hadoop-ozone/pull/1371 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [hadoop-ozone] runzhiwang opened a new pull request #1504: HDDS-3714. [Design] Ozone support truncate operation

2020-10-18 Thread GitBox
runzhiwang opened a new pull request #1504: URL: https://github.com/apache/hadoop-ozone/pull/1504 ## What changes were proposed in this pull request? This is a design doc, which is moved from google doc to make it easier to track the progress. ## What is the link to the Apache

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #1498: HDDS-4339. Skip aws signature when it's absent in header.

2020-10-18 Thread GitBox
bharatviswa504 commented on a change in pull request #1498: URL: https://github.com/apache/hadoop-ozone/pull/1498#discussion_r507293372 ## File path: hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/OzoneClientProducer.java ## @@ -116,6 +116,13 @@ private OzoneC

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #1498: HDDS-4339. Skip aws signature when it's absent in header.

2020-10-18 Thread GitBox
bharatviswa504 commented on a change in pull request #1498: URL: https://github.com/apache/hadoop-ozone/pull/1498#discussion_r507293372 ## File path: hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/OzoneClientProducer.java ## @@ -116,6 +116,13 @@ private OzoneC

[GitHub] [hadoop-ozone] linyiqun commented on a change in pull request #1503: HDDS-4332: ListFileStatus - do lookup in directory and file tables

2020-10-18 Thread GitBox
linyiqun commented on a change in pull request #1503: URL: https://github.com/apache/hadoop-ozone/pull/1503#discussion_r507038746 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java ## @@ -2205,6 +2272,167 @@ private void listS

[GitHub] [hadoop-ozone] linyiqun commented on a change in pull request #1503: HDDS-4332: ListFileStatus - do lookup in directory and file tables

2020-10-18 Thread GitBox
linyiqun commented on a change in pull request #1503: URL: https://github.com/apache/hadoop-ozone/pull/1503#discussion_r507038746 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java ## @@ -2205,6 +2272,167 @@ private void listS

[GitHub] [hadoop-ozone] linyiqun commented on a change in pull request #1503: HDDS-4332: ListFileStatus - do lookup in directory and file tables

2020-10-18 Thread GitBox
linyiqun commented on a change in pull request #1503: URL: https://github.com/apache/hadoop-ozone/pull/1503#discussion_r506883082 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java ## @@ -2205,6 +2272,167 @@ private void listS

[GitHub] [hadoop-ozone] amaliujia commented on pull request #1482: HDDS-4317. Copy UserVolumeInfo to PersistedUserVolumeInfo in interface-storage

2020-10-17 Thread GitBox
amaliujia commented on pull request #1482: URL: https://github.com/apache/hadoop-ozone/pull/1482#issuecomment-711083779 @elek friendly ping on this PR~ This is an automated message from the Apache Git Service. To respond to t

[GitHub] [hadoop-ozone] linyiqun commented on a change in pull request #1503: HDDS-4332: ListFileStatus - do lookup in directory and file tables

2020-10-17 Thread GitBox
linyiqun commented on a change in pull request #1503: URL: https://github.com/apache/hadoop-ozone/pull/1503#discussion_r506954736 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java ## @@ -2205,6 +2272,167 @@ private void listS

[GitHub] [hadoop-ozone] linyiqun commented on a change in pull request #1503: HDDS-4332: ListFileStatus - do lookup in directory and file tables

2020-10-17 Thread GitBox
linyiqun commented on a change in pull request #1503: URL: https://github.com/apache/hadoop-ozone/pull/1503#discussion_r506792006 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMFileRequest.java ## @@ -583,4 +588,112 @@ public s

[GitHub] [hadoop-ozone] linyiqun commented on a change in pull request #1503: HDDS-4332: ListFileStatus - do lookup in directory and file tables

2020-10-17 Thread GitBox
linyiqun commented on a change in pull request #1503: URL: https://github.com/apache/hadoop-ozone/pull/1503#discussion_r506887372 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java ## @@ -2205,6 +2272,167 @@ private void listS

[GitHub] [hadoop-ozone] linyiqun commented on a change in pull request #1503: HDDS-4332: ListFileStatus - do lookup in directory and file tables

2020-10-17 Thread GitBox
linyiqun commented on a change in pull request #1503: URL: https://github.com/apache/hadoop-ozone/pull/1503#discussion_r506854548 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java ## @@ -2205,6 +2272,167 @@ private void listS

[GitHub] [hadoop-ozone] linyiqun commented on a change in pull request #1503: HDDS-4332: ListFileStatus - do lookup in directory and file tables

2020-10-17 Thread GitBox
linyiqun commented on a change in pull request #1503: URL: https://github.com/apache/hadoop-ozone/pull/1503#discussion_r506790946 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java ## @@ -1831,6 +1838,62 @@ private OzoneFileSt

[GitHub] [hadoop-ozone] rakeshadr opened a new pull request #1503: HDDS-4332: ListFileStatus - do lookup in directory and file tables

2020-10-16 Thread GitBox
rakeshadr opened a new pull request #1503: URL: https://github.com/apache/hadoop-ozone/pull/1503 ## What changes were proposed in this pull request? https://issues.apache.org/jira/browse/HDDS-4332 ## What is the link to the Apache JIRA This task is to perform look up of

[GitHub] [hadoop-ozone] mukul1987 opened a new pull request #1502: HDDS-4354. Avoid spamming of logs because of deleted transactions.

2020-10-16 Thread GitBox
mukul1987 opened a new pull request #1502: URL: https://github.com/apache/hadoop-ozone/pull/1502 ## What changes were proposed in this pull request? Avoid spamming of logs because of deleted transactions. ## What is the link to the Apache JIRA https://issues.apache.org/jira/

[GitHub] [hadoop-ozone] runzhiwang commented on a change in pull request #1419: HDDS-3755. [DESIGN] Storage-class for Ozone

2020-10-15 Thread GitBox
runzhiwang commented on a change in pull request #1419: URL: https://github.com/apache/hadoop-ozone/pull/1419#discussion_r505970425 ## File path: hadoop-hdds/docs/content/design/storage-class.md ## @@ -19,10 +19,331 @@ author: Marton Ele See the License for the specific lang

[GitHub] [hadoop-ozone] maobaolong commented on pull request #1407: HDDS-4158. Provide a class type for Java based configuration

2020-10-15 Thread GitBox
maobaolong commented on pull request #1407: URL: https://github.com/apache/hadoop-ozone/pull/1407#issuecomment-709296180 Thank @elek for trigger ci checks for me, now it getting green, could you please to merge this PR? This

[GitHub] [hadoop-ozone] maobaolong commented on pull request #1110: HDDS-3843. Throw the specific exception other than NPE.

2020-10-15 Thread GitBox
maobaolong commented on pull request #1110: URL: https://github.com/apache/hadoop-ozone/pull/1110#issuecomment-709294091 @bharatviswa504 Thanks for bring this PR back, as s3g without enable the security, so I think EndpointBase#SignatureProcessor shouldn't be inject as AWSSignatureProcesso

[GitHub] [hadoop-ozone] flirmnave commented on pull request #1490: HDDS-4269. Ozone DataNode thinks a volume is failed if an unexpected file is in the HDDS root directory.

2020-10-15 Thread GitBox
flirmnave commented on pull request #1490: URL: https://github.com/apache/hadoop-ozone/pull/1490#issuecomment-709086213 Thanks @cxorm for reviewing and committing it, and @jojochuang for the review. This is an automated mess

[GitHub] [hadoop-ozone] cxorm merged pull request #1490: HDDS-4269. Ozone DataNode thinks a volume is failed if an unexpected file is in the HDDS root directory.

2020-10-15 Thread GitBox
cxorm merged pull request #1490: URL: https://github.com/apache/hadoop-ozone/pull/1490 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [hadoop-ozone] cxorm commented on pull request #1490: HDDS-4269. Ozone DataNode thinks a volume is failed if an unexpected file is in the HDDS root directory.

2020-10-15 Thread GitBox
cxorm commented on pull request #1490: URL: https://github.com/apache/hadoop-ozone/pull/1490#issuecomment-709020567 Thanks @flirmnave for the work, +1. And thanks @jojochuang for the report and review. I have committed it. -

[GitHub] [hadoop-ozone] cxorm commented on pull request #1445: HDDS-4272. Volume namespace: add usedNamespace and update it when create and delete bucket

2020-10-15 Thread GitBox
cxorm commented on pull request #1445: URL: https://github.com/apache/hadoop-ozone/pull/1445#issuecomment-709002336 Thank you @amaliujia for the fixes addressed comments. The work is great to me, but I think we would commit it after HDDS-4308. ---

[GitHub] [hadoop-ozone] amaliujia edited a comment on pull request #1445: HDDS-4272. Volume namespace: add usedNamespace and update it when create and delete bucket

2020-10-15 Thread GitBox
amaliujia edited a comment on pull request #1445: URL: https://github.com/apache/hadoop-ozone/pull/1445#issuecomment-708945714 @ChenSammi thanks for your pointer. I find the way to create linked bucket from there: just construct a bucket args by setting source volume name and bucket name

[GitHub] [hadoop-ozone] amaliujia edited a comment on pull request #1445: HDDS-4272. Volume namespace: add usedNamespace and update it when create and delete bucket

2020-10-15 Thread GitBox
amaliujia edited a comment on pull request #1445: URL: https://github.com/apache/hadoop-ozone/pull/1445#issuecomment-708945714 @ChenSammi thanks for your pointer. I find the way to create linked bucket from there: just construct a bucket args by setting source volume name and bucket name.

[GitHub] [hadoop-ozone] amaliujia commented on pull request #1445: HDDS-4272. Volume namespace: add usedNamespace and update it when create and delete bucket

2020-10-15 Thread GitBox
amaliujia commented on pull request #1445: URL: https://github.com/apache/hadoop-ozone/pull/1445#issuecomment-708945714 @ChenSammi thanks for your pointer. I find the way to create linked bucket there: just construct a bucket args by setting source volume name and bucket name. Have

[GitHub] [hadoop-ozone] ChenSammi commented on pull request #1445: HDDS-4272. Volume namespace: add usedNamespace and update it when create and delete bucket

2020-10-14 Thread GitBox
ChenSammi commented on pull request #1445: URL: https://github.com/apache/hadoop-ozone/pull/1445#issuecomment-708879034 It seems there is no example UT so far. You can refer to links.robot for CLI examples, and HDDS-3612. T

[GitHub] [hadoop-ozone] hanishakoneru merged pull request #1493: HDDS-4327. Potential resource leakage using BatchOperation.

2020-10-14 Thread GitBox
hanishakoneru merged pull request #1493: URL: https://github.com/apache/hadoop-ozone/pull/1493 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[GitHub] [hadoop-ozone] hanishakoneru commented on pull request #1493: HDDS-4327. Potential resource leakage using BatchOperation.

2020-10-14 Thread GitBox
hanishakoneru commented on pull request #1493: URL: https://github.com/apache/hadoop-ozone/pull/1493#issuecomment-708718652 Thanks @bharatviswa504. +1 for merge. This is an automated message from the Apache Git Service. To re

[GitHub] [hadoop-ozone] bharatviswa504 closed pull request #1228: HDDS-3995. Fix s3g met NPE exception while write file by multiPartUpload

2020-10-14 Thread GitBox
bharatviswa504 closed pull request #1228: URL: https://github.com/apache/hadoop-ozone/pull/1228 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [hadoop-ozone] bharatviswa504 commented on pull request #1228: HDDS-3995. Fix s3g met NPE exception while write file by multiPartUpload

2020-10-14 Thread GitBox
bharatviswa504 commented on pull request #1228: URL: https://github.com/apache/hadoop-ozone/pull/1228#issuecomment-708672417 Closing this, it is committed thru #1499 This is an automated message from the Apache Git Service.

[GitHub] [hadoop-ozone] bharatviswa504 commented on pull request #1499: HDDS-3995. Fix s3g met NPE exception while write file by multiPartUpload

2020-10-14 Thread GitBox
bharatviswa504 commented on pull request #1499: URL: https://github.com/apache/hadoop-ozone/pull/1499#issuecomment-708671730 Thank You @GlenGeng for the contribution. This is an automated message from the Apache Git Service.

[GitHub] [hadoop-ozone] bharatviswa504 merged pull request #1499: HDDS-3995. Fix s3g met NPE exception while write file by multiPartUpload

2020-10-14 Thread GitBox
bharatviswa504 merged pull request #1499: URL: https://github.com/apache/hadoop-ozone/pull/1499 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #1493: HDDS-4327. Potential resource leakage using BatchOperation.

2020-10-14 Thread GitBox
bharatviswa504 commented on a change in pull request #1493: URL: https://github.com/apache/hadoop-ozone/pull/1493#discussion_r504974154 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/SCMContainerManager.java ## @@ -397,33 +399,29 @@ pub

[GitHub] [hadoop-ozone] sodonnel opened a new pull request #1501: HDDS-4323. Add integration tests for putting nodes into maintenance and fix any issues uncovered in the tests

2020-10-14 Thread GitBox
sodonnel opened a new pull request #1501: URL: https://github.com/apache/hadoop-ozone/pull/1501 ## What changes were proposed in this pull request? Add a series of intergration tests to prove nodes can enter and leave maintenance correctly and address any issues in the code when addi

[GitHub] [hadoop-ozone] sodonnel opened a new pull request #1500: HDDS-4324. DatanodeAdminMonitor no longers needs maintenance end time to be passed

2020-10-14 Thread GitBox
sodonnel opened a new pull request #1500: URL: https://github.com/apache/hadoop-ozone/pull/1500 ## What changes were proposed in this pull request? An earlier change moved the maintenance endtime into the NodeStatus object. However when adding a node to the decommission monitor the e

[GitHub] [hadoop-ozone] sodonnel merged pull request #1495: HDDS-4343: ReplicationManager.handleOverReplicatedContainer() does not handle unhealthyReplicas properly.

2020-10-14 Thread GitBox
sodonnel merged pull request #1495: URL: https://github.com/apache/hadoop-ozone/pull/1495 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [hadoop-ozone] asfgit merged pull request #1496: Decom merge master oct14

2020-10-14 Thread GitBox
asfgit merged pull request #1496: URL: https://github.com/apache/hadoop-ozone/pull/1496 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [hadoop-ozone] GlenGeng commented on pull request #1499: HDDS-3995. Fix s3g met NPE exception while write file by multiPartUpload

2020-10-14 Thread GitBox
GlenGeng commented on pull request #1499: URL: https://github.com/apache/hadoop-ozone/pull/1499#issuecomment-708522091 @bharatviswa504 For the S3G NPE case, please check this patch. This is an automated message from the Apach

[GitHub] [hadoop-ozone] hanishakoneru commented on a change in pull request #1493: HDDS-4327. Potential resource leakage using BatchOperation.

2020-10-14 Thread GitBox
hanishakoneru commented on a change in pull request #1493: URL: https://github.com/apache/hadoop-ozone/pull/1493#discussion_r504812062 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/SCMContainerManager.java ## @@ -397,33 +399,29 @@ publ

[GitHub] [hadoop-ozone] GlenGeng opened a new pull request #1499: HDDS-3995. Fix s3g met NPE exception while write file by multiPartUpload

2020-10-14 Thread GitBox
GlenGeng opened a new pull request #1499: URL: https://github.com/apache/hadoop-ozone/pull/1499 ## What changes were proposed in this pull request? Fix s3g met NPE exception while write file by multiPartUpload. For more detail, please refer to conversions of https://github.com

[GitHub] [hadoop-ozone] linyiqun edited a comment on pull request #1459: HDDS-4297. Allow multiple transactions per container to be sent for deletion by SCM.

2020-10-14 Thread GitBox
linyiqun edited a comment on pull request #1459: URL: https://github.com/apache/hadoop-ozone/pull/1459#issuecomment-708509231 @lokeshj1703 , seems the original purpose of letting only one transaction per container to delete on datanode is to make container block deletion more balanced. So

[GitHub] [hadoop-ozone] linyiqun commented on pull request #1459: HDDS-4297. Allow multiple transactions per container to be sent for deletion by SCM.

2020-10-14 Thread GitBox
linyiqun commented on pull request #1459: URL: https://github.com/apache/hadoop-ozone/pull/1459#issuecomment-708509231 @lokeshj1703 , seems the original purpose of letting only one transaction per container to delete on datanode is to make container block deletion more balanced. So the con

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #1228: HDDS-3995. Fix s3g met NPE exception while write file by multiPartUpload

2020-10-14 Thread GitBox
bharatviswa504 commented on a change in pull request #1228: URL: https://github.com/apache/hadoop-ozone/pull/1228#discussion_r504746102 ## File path: tencent/python-client/OzoneS3.py ## @@ -0,0 +1,89 @@ +import boto3 +from botocore.config import Config +from botocore.exceptions

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #1228: HDDS-3995. Fix s3g met NPE exception while write file by multiPartUpload

2020-10-14 Thread GitBox
bharatviswa504 commented on a change in pull request #1228: URL: https://github.com/apache/hadoop-ozone/pull/1228#discussion_r504746102 ## File path: tencent/python-client/OzoneS3.py ## @@ -0,0 +1,89 @@ +import boto3 +from botocore.config import Config +from botocore.exceptions

[GitHub] [hadoop-ozone] codecov-io commented on pull request #1496: Decom merge master oct14

2020-10-14 Thread GitBox
codecov-io commented on pull request #1496: URL: https://github.com/apache/hadoop-ozone/pull/1496#issuecomment-708435644 # [Codecov](https://codecov.io/gh/apache/hadoop-ozone/pull/1496?src=pr&el=h1) Report > :exclamation: No coverage uploaded for pull request base (`HDDS-1880-Decom@f74

[GitHub] [hadoop-ozone] captainzmc closed pull request #1497: HDDS-4345. Replace the deprecated Lock method

2020-10-14 Thread GitBox
captainzmc closed pull request #1497: URL: https://github.com/apache/hadoop-ozone/pull/1497 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[GitHub] [hadoop-ozone] timmylicheng opened a new pull request #1498: HDDS-4339. Skip aws signature when it's absent in header.

2020-10-14 Thread GitBox
timmylicheng opened a new pull request #1498: URL: https://github.com/apache/hadoop-ozone/pull/1498 ## What changes were proposed in this pull request? Skip aws signature retrieval in AWSSignatureProcessor. It was causing NPE when signature is absent in header. (Please fill in cha

[GitHub] [hadoop-ozone] captainzmc opened a new pull request #1497: HDDS-4345. Delete the deprecated Lock method

2020-10-14 Thread GitBox
captainzmc opened a new pull request #1497: URL: https://github.com/apache/hadoop-ozone/pull/1497 ## What changes were proposed in this pull request? ![image](https://user-images.githubusercontent.com/13825159/95983129-b06dea80-0e53-11eb-806c-bd113c75f570.png) ![image](https://us

[GitHub] [hadoop-ozone] runzhiwang commented on pull request #1371: HDDS-2922. Balance ratis leader distribution in datanodes

2020-10-14 Thread GitBox
runzhiwang commented on pull request #1371: URL: https://github.com/apache/hadoop-ozone/pull/1371#issuecomment-708327004 @xiaoyuyao Could you help merge this patch ? Thanks a lot. This is an automated message from the Apache

[GitHub] [hadoop-ozone] linyiqun commented on a change in pull request #1495: HDDS-4343: ReplicationManager.handleOverReplicatedContainer() does not handle unhealthyReplicas properly.

2020-10-14 Thread GitBox
linyiqun commented on a change in pull request #1495: URL: https://github.com/apache/hadoop-ozone/pull/1495#discussion_r504584726 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ReplicationManager.java ## @@ -746,8 +746,9 @@ private void

[GitHub] [hadoop-ozone] sodonnel commented on pull request #1495: HDDS-4343: ReplicationManager.handleOverReplicatedContainer() does not handle unhealthyReplicas properly.

2020-10-14 Thread GitBox
sodonnel commented on pull request #1495: URL: https://github.com/apache/hadoop-ozone/pull/1495#issuecomment-708312356 @GlenGeng Thanks for highlighting this problem and well done for finding it. The change you have made looks good to me, +1 from my side. -

[GitHub] [hadoop-ozone] sodonnel commented on a change in pull request #1495: HDDS-4343: ReplicationManager.handleOverReplicatedContainer() does not handle unhealthyReplicas properly.

2020-10-14 Thread GitBox
sodonnel commented on a change in pull request #1495: URL: https://github.com/apache/hadoop-ozone/pull/1495#discussion_r504569348 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ReplicationManager.java ## @@ -746,8 +746,9 @@ private void

[GitHub] [hadoop-ozone] linyiqun commented on a change in pull request #1495: HDDS-4343: ReplicationManager.handleOverReplicatedContainer() does not handle unhealthyReplicas properly.

2020-10-14 Thread GitBox
linyiqun commented on a change in pull request #1495: URL: https://github.com/apache/hadoop-ozone/pull/1495#discussion_r504539793 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ReplicationManager.java ## @@ -746,8 +746,9 @@ private void

[GitHub] [hadoop-ozone] linyiqun commented on a change in pull request #1495: HDDS-4343: ReplicationManager.handleOverReplicatedContainer() does not handle unhealthyReplicas properly.

2020-10-14 Thread GitBox
linyiqun commented on a change in pull request #1495: URL: https://github.com/apache/hadoop-ozone/pull/1495#discussion_r504539793 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ReplicationManager.java ## @@ -746,8 +746,9 @@ private void

[GitHub] [hadoop-ozone] GlenGeng commented on pull request #1495: HDDS-4343: ReplicationManager.handleOverReplicatedContainer() does not handle unhealthyReplicas properly.

2020-10-14 Thread GitBox
GlenGeng commented on pull request #1495: URL: https://github.com/apache/hadoop-ozone/pull/1495#issuecomment-708261600 @sodonnel Hi Stephen, Please take a look at this PR. Thanks! This is an automated message from the Apache

[GitHub] [hadoop-ozone] sodonnel opened a new pull request #1496: Decom merge master oct14

2020-10-14 Thread GitBox
sodonnel opened a new pull request #1496: URL: https://github.com/apache/hadoop-ozone/pull/1496 ## What changes were proposed in this pull request? Merge the master branch into the Decommission branch. This was a clean merge except for a minor conflict on TestDataUtil.java, which was

[GitHub] [hadoop-ozone] GlenGeng opened a new pull request #1495: HDDS-4343: ReplicationManager.handleOverReplicatedContainer() does not handle unhealthyReplicas properly.

2020-10-14 Thread GitBox
GlenGeng opened a new pull request #1495: URL: https://github.com/apache/hadoop-ozone/pull/1495 What changes were proposed in this pull request? ``` // If there are unhealthy replicas, then we should remove them even if it // makes the container violate the placemen

[GitHub] [hadoop-ozone] sodonnel merged pull request #1484: HDDS-4322. Add integration tests for Decommission and resolve issues detected by the tests.

2020-10-14 Thread GitBox
sodonnel merged pull request #1484: URL: https://github.com/apache/hadoop-ozone/pull/1484 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [hadoop-ozone] maobaolong opened a new pull request #1228: HDDS-3995. Fix s3g met NPE exception while write file by multiPartUpload

2020-10-13 Thread GitBox
maobaolong opened a new pull request #1228: URL: https://github.com/apache/hadoop-ozone/pull/1228 ## What changes were proposed in this pull request? Fix s3g met NPE exception while write file by multiPartUpload ## What is the link to the Apache JIRA HDDS-3995 ##

[GitHub] [hadoop-ozone] maobaolong closed pull request #1228: HDDS-3995. Fix s3g met NPE exception while write file by multiPartUpload

2020-10-13 Thread GitBox
maobaolong closed pull request #1228: URL: https://github.com/apache/hadoop-ozone/pull/1228 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[GitHub] [hadoop-ozone] captainzmc commented on pull request #1489: HDDS-4308. Fix issue with quota update

2020-10-13 Thread GitBox
captainzmc commented on pull request #1489: URL: https://github.com/apache/hadoop-ozone/pull/1489#issuecomment-708182129 Hi @bharatviswa504, Please help to review this PR. This is an automated message from the Apache Git Serv

[GitHub] [hadoop-ozone] GlenGeng commented on pull request #1228: HDDS-3995. Fix s3g met NPE exception while write file by multiPartUpload

2020-10-13 Thread GitBox
GlenGeng commented on pull request #1228: URL: https://github.com/apache/hadoop-ozone/pull/1228#issuecomment-708134810 +1, LGTM. This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [hadoop-ozone] maobaolong commented on pull request #1228: HDDS-3995. Fix s3g met NPE exception while write file by multiPartUpload

2020-10-13 Thread GitBox
maobaolong commented on pull request #1228: URL: https://github.com/apache/hadoop-ozone/pull/1228#issuecomment-708134370 @GlenGeng @bharatviswa504 Thanks for your suggestion, I push a new commit to fix this. PTAL This

[GitHub] [hadoop-ozone] maobaolong commented on a change in pull request #1228: HDDS-3995. Fix s3g met NPE exception while write file by multiPartUpload

2020-10-13 Thread GitBox
maobaolong commented on a change in pull request #1228: URL: https://github.com/apache/hadoop-ozone/pull/1228#discussion_r504378327 ## File path: hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/ObjectEndpoint.java ## @@ -562,13 +562,18 @@ private Respo

[GitHub] [hadoop-ozone] GlenGeng commented on a change in pull request #1228: HDDS-3995. Fix s3g met NPE exception while write file by multiPartUpload

2020-10-13 Thread GitBox
GlenGeng commented on a change in pull request #1228: URL: https://github.com/apache/hadoop-ozone/pull/1228#discussion_r504374249 ## File path: hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/ObjectEndpoint.java ## @@ -562,13 +562,18 @@ private Respons

[GitHub] [hadoop-ozone] avijayanhwx merged pull request #1492: HDDS-4314. OM Layout Version Manager init throws silent CNF error in integration tests.

2020-10-13 Thread GitBox
avijayanhwx merged pull request #1492: URL: https://github.com/apache/hadoop-ozone/pull/1492 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hadoop-ozone] bharatviswa504 closed pull request #1418: HDDS-4209. S3A Filesystem does not work with Ozone S3.

2020-10-13 Thread GitBox
bharatviswa504 closed pull request #1418: URL: https://github.com/apache/hadoop-ozone/pull/1418 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [hadoop-ozone] bharatviswa504 commented on pull request #1418: HDDS-4209. S3A Filesystem does not work with Ozone S3.

2020-10-13 Thread GitBox
bharatviswa504 commented on pull request #1418: URL: https://github.com/apache/hadoop-ozone/pull/1418#issuecomment-707983374 Closing this for now, as we donot support S3A FileSystem in filesystem path enabled mode. Need to document this.

[GitHub] [hadoop-ozone] bharatviswa504 commented on pull request #1110: HDDS-3843. Throw the specific exception other than NPE.

2020-10-13 Thread GitBox
bharatviswa504 commented on pull request #1110: URL: https://github.com/apache/hadoop-ozone/pull/1110#issuecomment-707979758 I see with injection, we see error in logs but we don't see proper error code at the S3 client. ![Screen Shot 2020-10-13 at 1 08 09 PM](https://user-images.gi

[GitHub] [hadoop-ozone] bharatviswa504 commented on pull request #1110: HDDS-3843. Throw the specific exception other than NPE.

2020-10-13 Thread GitBox
bharatviswa504 commented on pull request #1110: URL: https://github.com/apache/hadoop-ozone/pull/1110#issuecomment-707967571 @maobaolong Once we are in similar behavior with AWS S3 we can get this in. I agree that NPE does not look right here. -

[GitHub] [hadoop-ozone] maobaolong opened a new pull request #1110: HDDS-3843. Throw the specific exception other than NPE.

2020-10-13 Thread GitBox
maobaolong opened a new pull request #1110: URL: https://github.com/apache/hadoop-ozone/pull/1110 ## What changes were proposed in this pull request? NPE shouldn't appear in the log. ## What is the link to the Apache JIRA HDDS-3843 ## How was this patch tested?

[GitHub] [hadoop-ozone] hanishakoneru opened a new pull request #1494: HDDS-4330. Bootstrap new OM node

2020-10-13 Thread GitBox
hanishakoneru opened a new pull request #1494: URL: https://github.com/apache/hadoop-ozone/pull/1494 ## What changes were proposed in this pull request? In a ratis enabled OM cluster, add support to bootstrap a new OM node and add it to OM ratis ring.  First step would be to u

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #1228: HDDS-3995. Fix s3g met NPE exception while write file by multiPartUpload

2020-10-13 Thread GitBox
bharatviswa504 commented on a change in pull request #1228: URL: https://github.com/apache/hadoop-ozone/pull/1228#discussion_r504192790 ## File path: hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/ObjectEndpoint.java ## @@ -562,13 +562,18 @@ private R

[GitHub] [hadoop-ozone] bharatviswa504 opened a new pull request #1493: HDDS-4327. Potential resource leakage using BatchOperation.

2020-10-13 Thread GitBox
bharatviswa504 opened a new pull request #1493: URL: https://github.com/apache/hadoop-ozone/pull/1493 ## What changes were proposed in this pull request? Potential resource leakage using BatchOperation. Use try enclosed resource/close the batch once after it usage is completed.

[GitHub] [hadoop-ozone] aryangupta1998 removed a comment on pull request #1487: HDDS-4318. Disable single node pipeline creation by default in Ozone.

2020-10-13 Thread GitBox
aryangupta1998 removed a comment on pull request #1487: URL: https://github.com/apache/hadoop-ozone/pull/1487#issuecomment-707740142 /retest This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [hadoop-ozone] rakeshadr commented on a change in pull request #1473: HDDS-4266: CreateFile : store parent dir entries into DirTable and file entry into separate FileTable

2020-10-13 Thread GitBox
rakeshadr commented on a change in pull request #1473: URL: https://github.com/apache/hadoop-ozone/pull/1473#discussion_r504133459 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMFileRequest.java ## @@ -466,27 +465,22 @@ public

[GitHub] [hadoop-ozone] bharatviswa504 merged pull request #1473: HDDS-4266: CreateFile : store parent dir entries into DirTable and file entry into separate FileTable

2020-10-13 Thread GitBox
bharatviswa504 merged pull request #1473: URL: https://github.com/apache/hadoop-ozone/pull/1473 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #1473: HDDS-4266: CreateFile : store parent dir entries into DirTable and file entry into separate FileTable

2020-10-13 Thread GitBox
bharatviswa504 commented on a change in pull request #1473: URL: https://github.com/apache/hadoop-ozone/pull/1473#discussion_r504127124 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMFileRequest.java ## @@ -466,27 +465,22 @@ pu

[GitHub] [hadoop-ozone] avijayanhwx commented on pull request #1492: HDDS-4314. OM Layout Version Manager init throws silent CNF error in integration tests.

2020-10-13 Thread GitBox
avijayanhwx commented on pull request #1492: URL: https://github.com/apache/hadoop-ozone/pull/1492#issuecomment-707882048 @fapifta / @prashantpogde Please review. This is an automated message from the Apache Git Service. To r

[GitHub] [hadoop-ozone] avijayanhwx opened a new pull request #1492: HDDS-4314. OM Layout Version Manager init throws silent CNF error in integration tests.

2020-10-13 Thread GitBox
avijayanhwx opened a new pull request #1492: URL: https://github.com/apache/hadoop-ozone/pull/1492 ## What changes were proposed in this pull request? Fix class scanner initialization such that it does not scan test packages. ## What is the link to the Apache JIRA https://issues

[GitHub] [hadoop-ozone] rakeshadr commented on pull request #1473: HDDS-4266: CreateFile : store parent dir entries into DirTable and file entry into separate FileTable

2020-10-13 Thread GitBox
rakeshadr commented on pull request #1473: URL: https://github.com/apache/hadoop-ozone/pull/1473#issuecomment-707874090 > Thank You for the updated patch. > I have one comment, (resolved fixed comments), rest LGTM. Thanks @bharatviswa504 for the detailed reviews. I've updated the p

[GitHub] [hadoop-ozone] bharatviswa504 merged pull request #1435: HDDS-4122. Implement OM Delete Expired Open Key Request and Response

2020-10-13 Thread GitBox
bharatviswa504 merged pull request #1435: URL: https://github.com/apache/hadoop-ozone/pull/1435 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [hadoop-ozone] bharatviswa504 commented on pull request #1435: HDDS-4122. Implement OM Delete Expired Open Key Request and Response

2020-10-13 Thread GitBox
bharatviswa504 commented on pull request #1435: URL: https://github.com/apache/hadoop-ozone/pull/1435#issuecomment-707853993 Thank You @errose28 for the contribution and @avijayanhwx for the review This is an automated messag

[GitHub] [hadoop-ozone] flirmnave commented on a change in pull request #1490: HDDS-4269. Ozone DataNode thinks a volume is failed if an unexpected file is in the HDDS root directory.

2020-10-13 Thread GitBox
flirmnave commented on a change in pull request #1490: URL: https://github.com/apache/hadoop-ozone/pull/1490#discussion_r503964885 ## File path: hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/utils/HddsVolumeUtil.java ## @@ -212,6 +212,9 @

[GitHub] [hadoop-ozone] github-actions[bot] commented on pull request #1487: HDDS-4318. Disable single node pipeline creation by default in Ozone.

2020-10-13 Thread GitBox
github-actions[bot] commented on pull request #1487: URL: https://github.com/apache/hadoop-ozone/pull/1487#issuecomment-707740455 To re-run CI checks, please follow these steps with the source branch checked out: ``` git commit --allow-empty -m 'trigger new CI check' git push ``

[GitHub] [hadoop-ozone] aryangupta1998 commented on pull request #1487: HDDS-4318. Disable single node pipeline creation by default in Ozone.

2020-10-13 Thread GitBox
aryangupta1998 commented on pull request #1487: URL: https://github.com/apache/hadoop-ozone/pull/1487#issuecomment-707740142 /retest This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [hadoop-ozone] jojochuang commented on a change in pull request #1490: HDDS-4269. Ozone DataNode thinks a volume is failed if an unexpected file is in the HDDS root directory.

2020-10-13 Thread GitBox
jojochuang commented on a change in pull request #1490: URL: https://github.com/apache/hadoop-ozone/pull/1490#discussion_r503922135 ## File path: hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/utils/HddsVolumeUtil.java ## @@ -212,6 +212,9

[GitHub] [hadoop-ozone] sodonnel closed pull request #1461: Merge Master into HDDS-1880-Decom

2020-10-13 Thread GitBox
sodonnel closed pull request #1461: URL: https://github.com/apache/hadoop-ozone/pull/1461 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [hadoop-ozone] sodonnel commented on pull request #1461: Merge Master into HDDS-1880-Decom

2020-10-13 Thread GitBox
sodonnel commented on pull request #1461: URL: https://github.com/apache/hadoop-ozone/pull/1461#issuecomment-707693651 This PR was merged manually from the CLI to preserve the history. Therefore I am closing this PR. This is

[GitHub] [hadoop-ozone] sodonnel opened a new pull request #1491: HDDS-4340. Add Operational State to the datanode list command

2020-10-13 Thread GitBox
sodonnel opened a new pull request #1491: URL: https://github.com/apache/hadoop-ozone/pull/1491 ## What changes were proposed in this pull request? The existing CLI command `ozone admin datanode list` provides output like: ``` bash-4.2$ ozone admin datanode list Datanode:

[GitHub] [hadoop-ozone] sodonnel merged pull request #1488: HDDS-4336. ContainerInfo does not persist BCSID (sequenceId) leading to failed replica reports

2020-10-13 Thread GitBox
sodonnel merged pull request #1488: URL: https://github.com/apache/hadoop-ozone/pull/1488 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [hadoop-ozone] flirmnave opened a new pull request #1490: HDDS-4269. Ozone DataNode thinks a volume is failed if an unexpected file is in the HDDS root directory.

2020-10-13 Thread GitBox
flirmnave opened a new pull request #1490: URL: https://github.com/apache/hadoop-ozone/pull/1490 ## What changes were proposed in this pull request? Adding a log makes it easy to track the problem. ## What is the link to the Apache JIRA https://issues.apache.org/jira/bro

<    1   2   3   4   5   6   7   8   9   10   >