[jira] [Updated] (HDDS-3567) Make replicationfactor can be configurable to any number

2020-05-15 Thread maobaolong (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maobaolong updated HDDS-3567: - Description: As there is a feature request that Ozone should support any replication number of a file,

[GitHub] [hadoop-ozone] maobaolong commented on pull request #920: HDDS-3581. Make replicationfactor can be set as a number

2020-05-15 Thread GitBox
maobaolong commented on pull request #920: URL: https://github.com/apache/hadoop-ozone/pull/920#issuecomment-629564381 This is a document relates this PR, i am writing it, feel free to take a look and left some commets. https://docs.qq.com/doc/DV2N6bWdCcnJVc3Rk Any comments will

[GitHub] [hadoop-ozone] maobaolong commented on pull request #920: HDDS-3581. Make replicationfactor can be set as a number

2020-05-15 Thread GitBox
maobaolong commented on pull request #920: URL: https://github.com/apache/hadoop-ozone/pull/920#issuecomment-629563949 @xiaoyuyao This PR aimed to refactor the protocol and relates code to support using integer to specify the count of replication. Now in the proto files, it use enum type

[GitHub] [hadoop-ozone] maobaolong commented on pull request #927: HDDS-3597. using protobuf maven plugin instead of the legacy protoc executable file

2020-05-15 Thread GitBox
maobaolong commented on pull request #927: URL: https://github.com/apache/hadoop-ozone/pull/927#issuecomment-629553291 @elek Would you like to take a look at this PR?This is a improvement what i aways want to do. This is an

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #855: HDDS-3474. Create transactionInfo Table in OmMetadataManager.

2020-05-15 Thread GitBox
bharatviswa504 commented on a change in pull request #855: URL: https://github.com/apache/hadoop-ozone/pull/855#discussion_r426091313 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmMetadataManagerImpl.java ## @@ -131,6 +136,8 @@ public

[GitHub] [hadoop-ozone] avijayanhwx commented on pull request #885: HDDS-3514. Fix memory leak of RaftServerImpl

2020-05-15 Thread GitBox
avijayanhwx commented on pull request #885: URL: https://github.com/apache/hadoop-ozone/pull/885#issuecomment-629542686 /pending This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [hadoop-ozone] hanishakoneru commented on a change in pull request #855: HDDS-3474. Create transactionInfo Table in OmMetadataManager.

2020-05-15 Thread GitBox
hanishakoneru commented on a change in pull request #855: URL: https://github.com/apache/hadoop-ozone/pull/855#discussion_r426072963 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmMetadataManagerImpl.java ## @@ -131,6 +136,8 @@ public

[GitHub] [hadoop-ozone] avijayanhwx edited a comment on pull request #924: HDDS-3595. Add a maven proto file backward compatibility checker in Ozone.

2020-05-15 Thread GitBox
avijayanhwx edited a comment on pull request #924: URL: https://github.com/apache/hadoop-ozone/pull/924#issuecomment-629529457 > I found the plugin to be useful to detect backward compatibility issues. However I still am not completely sure about LICENSE and checkstyle and other related

[GitHub] [hadoop-ozone] avijayanhwx commented on pull request #924: HDDS-3595. Add a maven proto file backward compatibility checker in Ozone.

2020-05-15 Thread GitBox
avijayanhwx commented on pull request #924: URL: https://github.com/apache/hadoop-ozone/pull/924#issuecomment-629529457 > I found the plugin to be useful to detect backward compatibility issues. However I still am not completely sure about LICENSE and checkstyle and other related issue.

[GitHub] [hadoop-ozone] avijayanhwx edited a comment on pull request #924: HDDS-3595. Add a maven proto file backward compatibility checker in Ozone.

2020-05-15 Thread GitBox
avijayanhwx edited a comment on pull request #924: URL: https://github.com/apache/hadoop-ozone/pull/924#issuecomment-629529457 > I found the plugin to be useful to detect backward compatibility issues. However I still am not completely sure about LICENSE and checkstyle and other related

[GitHub] [hadoop-ozone] smengcl commented on pull request #865: HDDS-2969. Implement ofs://: Add contract test

2020-05-15 Thread GitBox
smengcl commented on pull request #865: URL: https://github.com/apache/hadoop-ozone/pull/865#issuecomment-629511537 Opened #928 for HA contract tests. This is an automated message from the Apache Git Service. To respond to

[GitHub] [hadoop-ozone] smengcl opened a new pull request #928: HDDS-3559. Implement ofs://: Add contract test for HA

2020-05-15 Thread GitBox
smengcl opened a new pull request #928: URL: https://github.com/apache/hadoop-ozone/pull/928 ## What changes were proposed in this pull request? OFS: Add contract test for HA scenario. ## What is the link to the Apache JIRA

[jira] [Updated] (HDDS-3559) Datanode doesn't handle java heap OutOfMemory exception

2020-05-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-3559: - Labels: pull-request-available (was: ) > Datanode doesn't handle java heap OutOfMemory exception

[jira] [Updated] (HDDS-3599) Implement ofs://: Add contract test for HA

2020-05-15 Thread Siyao Meng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siyao Meng updated HDDS-3599: - Description: Add contract tests for HA as well. Since adding HA contract tests will be another ~10 new

[jira] [Created] (HDDS-3599) Implement ofs://: Add contract test for HA

2020-05-15 Thread Siyao Meng (Jira)
Siyao Meng created HDDS-3599: Summary: Implement ofs://: Add contract test for HA Key: HDDS-3599 URL: https://issues.apache.org/jira/browse/HDDS-3599 Project: Hadoop Distributed Data Store Issue

[jira] [Commented] (HDDS-3590) Recon UI: Add visualization for file size distribution

2020-05-15 Thread Vivek Ratnavel Subramanian (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17108665#comment-17108665 ] Vivek Ratnavel Subramanian commented on HDDS-3590: -- I have attached the initial version

[jira] [Updated] (HDDS-3590) Recon UI: Add visualization for file size distribution

2020-05-15 Thread Vivek Ratnavel Subramanian (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vivek Ratnavel Subramanian updated HDDS-3590: - Attachment: Recon UI_ Visualizing file size distribution.pdf > Recon UI:

[jira] [Created] (HDDS-3598) Allow existing buckets to enable trash ability.

2020-05-15 Thread YiSheng Lien (Jira)
YiSheng Lien created HDDS-3598: -- Summary: Allow existing buckets to enable trash ability. Key: HDDS-3598 URL: https://issues.apache.org/jira/browse/HDDS-3598 Project: Hadoop Distributed Data Store

[jira] [Commented] (HDDS-3567) Make replicationfactor can be configurable to any number

2020-05-15 Thread Arpit Agarwal (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17108598#comment-17108598 ] Arpit Agarwal commented on HDDS-3567: - Hi [~maobaolong], do you propose supporting different

[jira] [Updated] (HDDS-3597) using protobuf maven plugin instead of the legacy protoc executable file

2020-05-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-3597: - Labels: pull-request-available (was: ) > using protobuf maven plugin instead of the legacy

[GitHub] [hadoop-ozone] maobaolong opened a new pull request #927: HDDS-3597. using protobuf maven plugin instead of the legacy protoc executable file

2020-05-15 Thread GitBox
maobaolong opened a new pull request #927: URL: https://github.com/apache/hadoop-ozone/pull/927 ## What changes were proposed in this pull request? With this PR, we united using protobuf-maven-plugin to generate java file from proto file, we never have to install protoc

[jira] [Created] (HDDS-3597) using protobuf maven plugin instead of the legacy protoc executable file

2020-05-15 Thread maobaolong (Jira)
maobaolong created HDDS-3597: Summary: using protobuf maven plugin instead of the legacy protoc executable file Key: HDDS-3597 URL: https://issues.apache.org/jira/browse/HDDS-3597 Project: Hadoop

[jira] [Updated] (HDDS-3596) Clean up unused code after HDDS-2940

2020-05-15 Thread Siyao Meng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siyao Meng updated HDDS-3596: - Assignee: Siyao Meng Status: Patch Available (was: Open) > Clean up unused code after HDDS-2940 >

[jira] [Updated] (HDDS-3596) Clean up unused code after HDDS-2940

2020-05-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-3596: - Labels: pull-request-available (was: ) > Clean up unused code after HDDS-2940 >

[jira] [Updated] (HDDS-3596) Clean up unused code after HDDS-2940

2020-05-15 Thread Siyao Meng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siyao Meng updated HDDS-3596: - Description: It seems some snippets of code should be removed as HDDS-2940 is committed. For example

[GitHub] [hadoop-ozone] smengcl opened a new pull request #926: HDDS-3596. Clean up unused code after HDDS-2940

2020-05-15 Thread GitBox
smengcl opened a new pull request #926: URL: https://github.com/apache/hadoop-ozone/pull/926 ## What changes were proposed in this pull request? It seems some snippets of code should be removed as HDDS-2940 is committed. For example

[GitHub] [hadoop-ozone] smengcl commented on a change in pull request #906: HDDS-3494. Implement ofs://: Support volume and bucket deletion

2020-05-15 Thread GitBox
smengcl commented on a change in pull request #906: URL: https://github.com/apache/hadoop-ozone/pull/906#discussion_r425983500 ## File path: hadoop-ozone/ozonefs/src/main/java/org/apache/hadoop/fs/ozone/BasicRootedOzoneFileSystem.java ## @@ -469,15 +478,72 @@ public boolean

[jira] [Updated] (HDDS-3567) Make replicationfactor can be configurable to any number

2020-05-15 Thread maobaolong (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maobaolong updated HDDS-3567: - Description: As there is a feature request that Ozone should support any replication number of a file, we

[jira] [Created] (HDDS-3596) Clean up unused code after HDDS-2940

2020-05-15 Thread Siyao Meng (Jira)
Siyao Meng created HDDS-3596: Summary: Clean up unused code after HDDS-2940 Key: HDDS-3596 URL: https://issues.apache.org/jira/browse/HDDS-3596 Project: Hadoop Distributed Data Store Issue Type:

[GitHub] [hadoop-ozone] smengcl commented on pull request #906: HDDS-3494. Implement ofs://: Support volume and bucket deletion

2020-05-15 Thread GitBox
smengcl commented on pull request #906: URL: https://github.com/apache/hadoop-ozone/pull/906#issuecomment-629416263 @umamaheswararao Looks like a lot of flaky tests since the last commit is only javadoc change. WIll trigger a retest empty commit just in case.

[GitHub] [hadoop-ozone] vivekratnavel commented on pull request #923: HDDS-3391. Delegate admin ACL checks to Ozone authorizer plugin.

2020-05-15 Thread GitBox
vivekratnavel commented on pull request #923: URL: https://github.com/apache/hadoop-ozone/pull/923#issuecomment-629389500 @xiaoyuyao Thanks for working on this! +1 LGTM except for the CI test failures. This is an

[GitHub] [hadoop-ozone] hanishakoneru opened a new pull request #925: OM HA can be started with 3 isolated LEADER instead of one OM ring

2020-05-15 Thread GitBox
hanishakoneru opened a new pull request #925: URL: https://github.com/apache/hadoop-ozone/pull/925 ## What changes were proposed in this pull request? If DNS is unreliable, OM HA cluster could end up with all OMs starting their own single node Ratis servers. This could happen

[GitHub] [hadoop-ozone] xiaoyuyao commented on pull request #923: HDDS-3391. Delegate admin ACL checks to Ozone authorizer plugin.

2020-05-15 Thread GitBox
xiaoyuyao commented on pull request #923: URL: https://github.com/apache/hadoop-ozone/pull/923#issuecomment-629353349 Thanks @captainzmc for the confirmation. I will clear the CI issues today. This is an automated message

[jira] [Updated] (HDDS-2416) Ozone Trash Feature

2020-05-15 Thread YiSheng Lien (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YiSheng Lien updated HDDS-2416: --- Description: This Jira is a proposal to add a new feature to Ozone that provides a user with the

[jira] [Comment Edited] (HDDS-3367) Trash Support for Ozone FileSystem

2020-05-15 Thread YiSheng Lien (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17108326#comment-17108326 ] YiSheng Lien edited comment on HDDS-3367 at 5/15/20, 2:26 PM: -- Hi

[GitHub] [hadoop-ozone] maobaolong commented on pull request #921: HDDS-3583. Loosen some rule restrictions of checkstyle

2020-05-15 Thread GitBox
maobaolong commented on pull request #921: URL: https://github.com/apache/hadoop-ozone/pull/921#issuecomment-629265339 @sodonnel Yeah it is necessary This is an automated message from the Apache Git Service. To respond to

[jira] [Commented] (HDDS-3367) Trash Support for Ozone FileSystem

2020-05-15 Thread YiSheng Lien (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17108326#comment-17108326 ] YiSheng Lien commented on HDDS-3367: Hi [~sadanand_shenoy], thank you for the patch. I think we could

[GitHub] [hadoop-ozone] mukul1987 commented on pull request #924: HDDS-3595. Add a maven proto file backward compatibility checker in Ozone.

2020-05-15 Thread GitBox
mukul1987 commented on pull request #924: URL: https://github.com/apache/hadoop-ozone/pull/924#issuecomment-629252681 I found the plugin to be useful to detect backward compatibility issues. However I still am not completely sure about LICENSE and checkstyle and other related issue.

[jira] [Updated] (HDDS-3595) Add a maven proto file backward compatibility checker into Ozone.

2020-05-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-3595: - Labels: pull-request-available (was: ) > Add a maven proto file backward compatibility checker

[GitHub] [hadoop-ozone] mukul1987 opened a new pull request #924: HDDS-3595. Add a maven proto file backward compatibility checker in Ozone.

2020-05-15 Thread GitBox
mukul1987 opened a new pull request #924: URL: https://github.com/apache/hadoop-ozone/pull/924 ## What changes were proposed in this pull request? This patch adds a protobuf backward compatibility checker into Ozone. ## What is the link to the Apache JIRA

[GitHub] [hadoop-ozone] captainzmc edited a comment on pull request #923: HDDS-3391. Delegate admin ACL checks to Ozone authorizer plugin.

2020-05-15 Thread GitBox
captainzmc edited a comment on pull request #923: URL: https://github.com/apache/hadoop-ozone/pull/923#issuecomment-629236440 Thanks xiaoyu for contributing this. cc @ChenSammi @timmylicheng I tested the PR locally and it looks good. Administrators can access all volumes, buckets,

[jira] [Commented] (HDDS-3572) Make sure ozone. administrators have access to all keys when acl enabled.

2020-05-15 Thread mingchao zhao (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17108291#comment-17108291 ] mingchao zhao commented on HDDS-3572: - This issue will be fixed in this jira. > Make sure ozone.

[GitHub] [hadoop-ozone] captainzmc commented on pull request #923: HDDS-3391. Delegate admin ACL checks to Ozone authorizer plugin.

2020-05-15 Thread GitBox
captainzmc commented on pull request #923: URL: https://github.com/apache/hadoop-ozone/pull/923#issuecomment-629236440 Thanks xiaoyu for contributing this. I tested the PR locally and it looks good. Administrators can access all volumes, buckets, and keys. Its permissions are no longer

[jira] [Created] (HDDS-3595) Add a maven proto file backward compatibility checker into Ozone.

2020-05-15 Thread Mukul Kumar Singh (Jira)
Mukul Kumar Singh created HDDS-3595: --- Summary: Add a maven proto file backward compatibility checker into Ozone. Key: HDDS-3595 URL: https://issues.apache.org/jira/browse/HDDS-3595 Project: Hadoop

[jira] [Created] (HDDS-3594) ManagedChannels are leaked in XceiverClientGrpc manager

2020-05-15 Thread Rakesh Radhakrishnan (Jira)
Rakesh Radhakrishnan created HDDS-3594: -- Summary: ManagedChannels are leaked in XceiverClientGrpc manager Key: HDDS-3594 URL: https://issues.apache.org/jira/browse/HDDS-3594 Project: Hadoop

[GitHub] [hadoop-ozone] sodonnel commented on pull request #921: HDDS-3583. Loosen some rule restrictions of checkstyle

2020-05-15 Thread GitBox
sodonnel commented on pull request #921: URL: https://github.com/apache/hadoop-ozone/pull/921#issuecomment-629127925 I am in favour of this change, but for a change like this, I feel we need a few more +1's and also wait to see if there are any objections. We will try to discuss it on the

[jira] [Assigned] (HDDS-3196) Pipeline mutation needs to applyTransaction before writing to DB

2020-05-15 Thread Li Cheng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Cheng reassigned HDDS-3196: -- Assignee: Li Cheng > Pipeline mutation needs to applyTransaction before writing to DB >

[jira] [Assigned] (HDDS-3584) Support write 2 replication

2020-05-15 Thread maobaolong (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maobaolong reassigned HDDS-3584: Assignee: maobaolong > Support write 2 replication > --- > >

[jira] [Created] (HDDS-3593) Set a configurable maximum limit of available pipeline

2020-05-15 Thread maobaolong (Jira)
maobaolong created HDDS-3593: Summary: Set a configurable maximum limit of available pipeline Key: HDDS-3593 URL: https://issues.apache.org/jira/browse/HDDS-3593 Project: Hadoop Distributed Data Store

[jira] [Resolved] (HDDS-3473) ozone chunkinfo CLI should display block file path info

2020-05-15 Thread Shashikant Banerjee (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee resolved HDDS-3473. --- Fix Version/s: 0.6.0 Resolution: Fixed Thanks [~sadanand_shenoy] for working

[GitHub] [hadoop-ozone] bshashikant merged pull request #886: HDDS-3473. Ozone chunkinfo CLI should display block file path info

2020-05-15 Thread GitBox
bshashikant merged pull request #886: URL: https://github.com/apache/hadoop-ozone/pull/886 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hadoop-ozone] captainzmc commented on a change in pull request #918: WIP HDDS-3572. Make sure ozone. administrators have access to all keys when acl enabled.

2020-05-15 Thread GitBox
captainzmc commented on a change in pull request #918: URL: https://github.com/apache/hadoop-ozone/pull/918#discussion_r425563994 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java ## @@ -1589,6 +1593,15 @@ public boolean

[jira] [Updated] (HDDS-3591) ozone.administrators support dynamic changes through the cli.

2020-05-15 Thread mingchao zhao (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mingchao zhao updated HDDS-3591: Summary: ozone.administrators support dynamic changes through the cli. (was: ozone.

[jira] [Created] (HDDS-3592) Pipeline evict policy

2020-05-15 Thread maobaolong (Jira)
maobaolong created HDDS-3592: Summary: Pipeline evict policy Key: HDDS-3592 URL: https://issues.apache.org/jira/browse/HDDS-3592 Project: Hadoop Distributed Data Store Issue Type: Sub-task

[jira] [Resolved] (HDDS-3555) Failed to create bucket by s3g

2020-05-15 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang resolved HDDS-3555. -- Resolution: Resolved > Failed to create bucket by s3g > -- > >

[GitHub] [hadoop-ozone] xiaoyuyao commented on pull request #918: WIP HDDS-3572. Make sure ozone. administrators have access to all keys when acl enabled.

2020-05-15 Thread GitBox
xiaoyuyao commented on pull request #918: URL: https://github.com/apache/hadoop-ozone/pull/918#issuecomment-629051052 @captainzmc Please check: https://github.com/apache/hadoop-ozone/pull/923 Thanks! This is an automated

[jira] [Updated] (HDDS-3391) Delegate admin ACL checks to Ozone authorizer plugin

2020-05-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-3391: - Labels: pull-request-available (was: ) > Delegate admin ACL checks to Ozone authorizer plugin >

[GitHub] [hadoop-ozone] xiaoyuyao opened a new pull request #923: HDDS-3391. Delegate admin ACL checks to Ozone authorizer plugin.

2020-05-15 Thread GitBox
xiaoyuyao opened a new pull request #923: URL: https://github.com/apache/hadoop-ozone/pull/923 ## What changes were proposed in this pull request? Delegate admin check (volume create and list) to authorizer plugin. Bypass admin acl check for ozone native authorizer. ## What