[jira] [Updated] (HDDS-3678) Remove usage of DFSUtil.addPBProtocol method

2020-05-29 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-3678: --- Status: Patch Available (was: Open) > Remove usage of DFSUtil.addPBProtocol method >

[GitHub] [hadoop-ozone] avijayanhwx commented on a change in pull request #994: HDDS-3082. Refactor recon missing containers task to detect under, over and mis-replicated containers.

2020-05-29 Thread GitBox
avijayanhwx commented on a change in pull request #994: URL: https://github.com/apache/hadoop-ozone/pull/994#discussion_r432785173 ## File path: hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/fsck/ContainerHealthTask.java ## @@ -0,0 +1,345 @@ +/* + * Licensed t

[GitHub] [hadoop-ozone] vivekratnavel commented on a change in pull request #988: HDDS-3681. Recon: Add support to store file size counts in each volum…

2020-05-29 Thread GitBox
vivekratnavel commented on a change in pull request #988: URL: https://github.com/apache/hadoop-ozone/pull/988#discussion_r432754315 ## File path: hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/FileSizeCountTask.java ## @@ -49,33 +52,28 @@ private stat

[GitHub] [hadoop-ozone] smengcl commented on a change in pull request #941: HDDS-3574. Implement ofs://: Override getTrashRoot

2020-05-29 Thread GitBox
smengcl commented on a change in pull request #941: URL: https://github.com/apache/hadoop-ozone/pull/941#discussion_r432731528 ## File path: hadoop-ozone/ozonefs/src/main/java/org/apache/hadoop/fs/ozone/OFSPath.java ## @@ -238,4 +248,23 @@ static String getTempMountBucketNameO

[GitHub] [hadoop-ozone] smengcl commented on a change in pull request #941: HDDS-3574. Implement ofs://: Override getTrashRoot

2020-05-29 Thread GitBox
smengcl commented on a change in pull request #941: URL: https://github.com/apache/hadoop-ozone/pull/941#discussion_r432730947 ## File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/shell/TestOzoneShellHA.java ## @@ -442,6 +446,27 @@ public void test

[jira] [Updated] (HDDS-3309) Add TimedOutTestsListener to surefire and add timeout to integration tests

2020-05-29 Thread Arpit Agarwal (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDDS-3309: Resolution: Fixed Status: Resolved (was: Patch Available) > Add TimedOutTestsListener to su

[GitHub] [hadoop-ozone] arp7 commented on pull request #941: HDDS-3574. Implement ofs://: Override getTrashRoot

2020-05-29 Thread GitBox
arp7 commented on pull request #941: URL: https://github.com/apache/hadoop-ozone/pull/941#issuecomment-636177915 > But we might need to set the ACL on /.Trash// to that user only though. @xiaoyuyao Sounds good, we can do this in a followup jira. ---

[GitHub] [hadoop-ozone] arp7 edited a comment on pull request #941: HDDS-3574. Implement ofs://: Override getTrashRoot

2020-05-29 Thread GitBox
arp7 edited a comment on pull request #941: URL: https://github.com/apache/hadoop-ozone/pull/941#issuecomment-636177915 > But we might need to set the ACL on /.Trash// to that user only though. @xiaoyuyao Sounds good, we can do this in a followup jira.

[jira] [Resolved] (HDDS-3382) OzoneManager fails to apply log index because of FNFE

2020-05-29 Thread Arpit Agarwal (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal resolved HDDS-3382. - Resolution: Duplicate > OzoneManager fails to apply log index because of FNFE > --

[GitHub] [hadoop-ozone] smengcl commented on a change in pull request #941: HDDS-3574. Implement ofs://: Override getTrashRoot

2020-05-29 Thread GitBox
smengcl commented on a change in pull request #941: URL: https://github.com/apache/hadoop-ozone/pull/941#discussion_r432709744 ## File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/shell/TestOzoneShellHA.java ## @@ -442,6 +446,27 @@ public void test

[GitHub] [hadoop-ozone] vivekratnavel commented on a change in pull request #994: HDDS-3082. Refactor recon missing containers task to detect under, over and mis-replicated containers.

2020-05-29 Thread GitBox
vivekratnavel commented on a change in pull request #994: URL: https://github.com/apache/hadoop-ozone/pull/994#discussion_r432689814 ## File path: hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/fsck/ContainerHealthTask.java ## @@ -0,0 +1,345 @@ +/* + * Licensed

[GitHub] [hadoop-ozone] elek commented on pull request #989: HDDS-3546. use shade plugin to build legacy jar

2020-05-29 Thread GitBox
elek commented on pull request #989: URL: https://github.com/apache/hadoop-ozone/pull/989#issuecomment-636147143 Thanks to open this issue @Simon0806 I agree, that it's a problem that the defaults are not merged currently and we should fix it. 1. Small comments: As I see the

[GitHub] [hadoop-ozone] vivekratnavel commented on a change in pull request #988: HDDS-3681. Recon: Add support to store file size counts in each volum…

2020-05-29 Thread GitBox
vivekratnavel commented on a change in pull request #988: URL: https://github.com/apache/hadoop-ozone/pull/988#discussion_r432670797 ## File path: hadoop-ozone/recon-codegen/src/main/java/org/hadoop/ozone/recon/schema/UtilizationSchemaDefinition.java ## @@ -83,11 +86,17 @@ pri

[GitHub] [hadoop-ozone] avijayanhwx commented on a change in pull request #988: HDDS-3681. Recon: Add support to store file size counts in each volum…

2020-05-29 Thread GitBox
avijayanhwx commented on a change in pull request #988: URL: https://github.com/apache/hadoop-ozone/pull/988#discussion_r432623933 ## File path: hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/UtilizationEndpoint.java ## @@ -34,7 +34,7 @@ */ @Path("/utili

[GitHub] [hadoop-ozone] avijayanhwx commented on a change in pull request #988: HDDS-3681. Recon: Add support to store file size counts in each volum…

2020-05-29 Thread GitBox
avijayanhwx commented on a change in pull request #988: URL: https://github.com/apache/hadoop-ozone/pull/988#discussion_r432610066 ## File path: hadoop-ozone/recon-codegen/src/main/java/org/hadoop/ozone/recon/schema/UtilizationSchemaDefinition.java ## @@ -83,11 +86,17 @@ priva

[GitHub] [hadoop-ozone] mukul1987 commented on pull request #911: HDDS-3564. Update Ozone to latest Ratis Snapshot (0.6.0-3596a58-SNAPSHOT).

2020-05-29 Thread GitBox
mukul1987 commented on pull request #911: URL: https://github.com/apache/hadoop-ozone/pull/911#issuecomment-636073584 @adoroszlai, sure I will create a new patch with the latest ratis snapshot This is an automated message fro

[jira] [Updated] (HDDS-3082) Refactor recon missing containers task to detect under, over and mis-replicated containers

2020-05-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-3082: - Labels: pull-request-available (was: ) > Refactor recon missing containers task to detect under,

[GitHub] [hadoop-ozone] sodonnel opened a new pull request #994: HDDS-3082. Refactor recon missing containers task to detect under, over and mis-replicated containers.

2020-05-29 Thread GitBox
sodonnel opened a new pull request #994: URL: https://github.com/apache/hadoop-ozone/pull/994 ## What changes were proposed in this pull request? The current Recon "Missing Containers Task" only highlights missing containers in the cluster. It is desired for it to also detect unde

[jira] [Updated] (HDDS-3082) Refactor recon missing containers task to detect under, over and mis-replicated containers

2020-05-29 Thread Stephen O'Donnell (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen O'Donnell updated HDDS-3082: Summary: Refactor recon missing containers task to detect under, over and mis-replicated con

[jira] [Updated] (HDDS-3542) ozone chunkinfo CLI cannot connect to OM when run from NON OM leader client node

2020-05-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-3542: - Labels: pull-request-available (was: ) > ozone chunkinfo CLI cannot connect to OM when run from N

[GitHub] [hadoop-ozone] sadanand48 opened a new pull request #993: HDDS-3542.Ozone chunkinfo CLI cannot connect to OM when run from non-om node.

2020-05-29 Thread GitBox
sadanand48 opened a new pull request #993: URL: https://github.com/apache/hadoop-ozone/pull/993 ## What changes were proposed in this pull request? The chunkinfo tool was not connecting to OM on a non-om leader client node .The fix here is to use existing ozoneManagerClient instance in

[GitHub] [hadoop-ozone] sadanand48 commented on pull request #945: HDDS-3622. Implement rocksdb tool to parse scm db

2020-05-29 Thread GitBox
sadanand48 commented on pull request #945: URL: https://github.com/apache/hadoop-ozone/pull/945#issuecomment-636032834 Thanks @elek for the comments. I have made the changes you suggested. Also, i am now choosing the DBDefinition according to the directory name. In this patch , i have done

[jira] [Commented] (HDDS-2939) Ozone FS namespace

2020-05-29 Thread Istvan Fajth (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17119653#comment-17119653 ] Istvan Fajth commented on HDDS-2939: Hi [~sdeka] and [~rakeshr], I have gone through

[jira] [Commented] (HDDS-3644) Failed to delete chunk file due to chunk size mismatch

2020-05-29 Thread Mukul Kumar Singh (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17119651#comment-17119651 ] Mukul Kumar Singh commented on HDDS-3644: - [~adoroszlai] [~Sammi], I think with th

[GitHub] [hadoop-ozone] ChenSammi commented on a change in pull request #976: HDDS-3672. Ozone fs failed to list intermediate directory.

2020-05-29 Thread GitBox
ChenSammi commented on a change in pull request #976: URL: https://github.com/apache/hadoop-ozone/pull/976#discussion_r432238251 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java ## @@ -2037,11 +2037,11 @@ private void listSt

[jira] [Updated] (HDDS-3627) Remove FilteredClassloader and replace with maven based hadoop2/hadoop3 ozonefs generation

2020-05-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-3627: - Labels: pull-request-available (was: ) > Remove FilteredClassloader and replace with maven based

[GitHub] [hadoop-ozone] elek opened a new pull request #992: HDDS-3627. Remove FilteredClassloader and replace with maven based hadoop2/hadoop3 ozonefs generation

2020-05-29 Thread GitBox
elek opened a new pull request #992: URL: https://github.com/apache/hadoop-ozone/pull/992 ## What changes were proposed in this pull request? As described in the [parent issue](https://issues.apache.org/jira/browse/HDDS-3458), the final step is to create a Hadoop independent shaded

[jira] [Updated] (HDDS-3686) Insert an argument of ozone shell to accept jvm arguments

2020-05-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-3686: - Labels: pull-request-available (was: ) > Insert an argument of ozone shell to accept jvm argument

[GitHub] [hadoop-ozone] maobaolong opened a new pull request #991: HDDS-3686. Insert an argument of ozone shell to accept jvm arguments

2020-05-29 Thread GitBox
maobaolong opened a new pull request #991: URL: https://github.com/apache/hadoop-ozone/pull/991 ## What changes were proposed in this pull request? This is a great feature, i believe everyone would like this new "--jvmargs" argument, we can specify some arguments you like, mostly, i

[jira] [Assigned] (HDDS-3687) logLevel cannot change the log level of a given logger

2020-05-29 Thread maobaolong (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maobaolong reassigned HDDS-3687: Assignee: (was: maobaolong) > logLevel cannot change the log level of a given logger >

[jira] [Commented] (HDDS-3687) logLevel cannot change the log level of a given logger

2020-05-29 Thread maobaolong (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17119438#comment-17119438 ] maobaolong commented on HDDS-3687: -- Hi all, [~leosun08] is working on this tickets. > lo

[GitHub] [hadoop-ozone] captainzmc commented on pull request #814: HDDS-3286. BasicOzoneFileSystem support batchDelete.

2020-05-29 Thread GitBox
captainzmc commented on pull request #814: URL: https://github.com/apache/hadoop-ozone/pull/814#issuecomment-635870828 This PR can continue to be reviewed. cc @timmylicheng @ChenSammi @xiaoyuyao Thanks. This is an automated

[GitHub] [hadoop-ozone] captainzmc edited a comment on pull request #814: HDDS-3286. BasicOzoneFileSystem support batchDelete.

2020-05-29 Thread GitBox
captainzmc edited a comment on pull request #814: URL: https://github.com/apache/hadoop-ozone/pull/814#issuecomment-635474849 A new commit was committed to fix the review problem: 1. Use multiple KeyArgs to delete in batches. 2. Add the entire batch operation interface to ensure compat

[jira] [Assigned] (HDDS-3687) logLevel cannot change the log level of a given logger

2020-05-29 Thread maobaolong (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maobaolong reassigned HDDS-3687: Assignee: (was: maobaolong) > logLevel cannot change the log level of a given logger >

[jira] [Commented] (HDDS-3644) Failed to delete chunk file due to chunk size mismatch

2020-05-29 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17119426#comment-17119426 ] Attila Doroszlai commented on HDDS-3644: bq. If I change the datanode code, do not

[jira] [Assigned] (HDDS-3687) logLevel cannot change the log level of a given logger

2020-05-29 Thread maobaolong (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maobaolong reassigned HDDS-3687: Assignee: maobaolong > logLevel cannot change the log level of a given logger > ---

[jira] [Assigned] (HDDS-3687) logLevel cannot change the log level of a given logger

2020-05-29 Thread maobaolong (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maobaolong reassigned HDDS-3687: Assignee: maobaolong > logLevel cannot change the log level of a given logger > ---

[jira] [Created] (HDDS-3687) logLevel cannot change the log level of a given logger

2020-05-29 Thread maobaolong (Jira)
maobaolong created HDDS-3687: Summary: logLevel cannot change the log level of a given logger Key: HDDS-3687 URL: https://issues.apache.org/jira/browse/HDDS-3687 Project: Hadoop Distributed Data Store

[GitHub] [hadoop-ozone] elek commented on pull request #944: HDDS-3609. Avoid to use Hadoop3.x IOUtils in Ozone Client

2020-05-29 Thread GitBox
elek commented on pull request #944: URL: https://github.com/apache/hadoop-ozone/pull/944#issuecomment-635861656 It turned out that using IOUtils is unsafe. With using Hadoop 2.7.7: With the final patch I got: ``` nm_1| java.lang.NoSuchMethodError: org.apache.common

[jira] [Assigned] (HDDS-3644) Failed to delete chunk file due to chunk size mismatch

2020-05-29 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai reassigned HDDS-3644: -- Assignee: Attila Doroszlai > Failed to delete chunk file due to chunk size mismatch > -

[jira] [Commented] (HDDS-3499) Address compatibility issue by SCM DB instances change

2020-05-29 Thread Li Cheng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17119406#comment-17119406 ] Li Cheng commented on HDDS-3499: Hey I've tested in a testing cluster and it works for us.

[jira] [Created] (HDDS-3686) Insert an argument of ozone shell to accept jvm arguments

2020-05-29 Thread maobaolong (Jira)
maobaolong created HDDS-3686: Summary: Insert an argument of ozone shell to accept jvm arguments Key: HDDS-3686 URL: https://issues.apache.org/jira/browse/HDDS-3686 Project: Hadoop Distributed Data Store

[jira] [Assigned] (HDDS-3667) If we gracefully stop datanode it would be better to notify scm and recon to unregister

2020-05-29 Thread maobaolong (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maobaolong reassigned HDDS-3667: Assignee: maobaolong > If we gracefully stop datanode it would be better to notify scm and recon to

[GitHub] [hadoop-ozone] timmylicheng commented on a change in pull request #980: HDDS-3196 New PipelineManager interface to persist to RatisServer.

2020-05-29 Thread GitBox
timmylicheng commented on a change in pull request #980: URL: https://github.com/apache/hadoop-ozone/pull/980#discussion_r432308891 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/PipelineStateManagerV2.java ## @@ -0,0 +1,88 @@ +/* + * Li