[GitHub] [hadoop] bilaharith commented on a change in pull request #2549: Hadoop 17428. ABFS: Implementation for getContentSummary

bilaharith commented on a change in pull request #2549: URL: https://github.com/apache/hadoop/pull/2549#discussion_r554922036 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystem.java ## @@ -363,6 +367,22 @@ public boolean del

[GitHub] [hadoop] bilaharith commented on a change in pull request #2549: Hadoop 17428. ABFS: Implementation for getContentSummary

bilaharith commented on a change in pull request #2549: URL: https://github.com/apache/hadoop/pull/2549#discussion_r554923224 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/ContentSummaryProcessor.java ## @@ -0,0 +1,119 @@ +/** + *

[GitHub] [hadoop] bilaharith commented on a change in pull request #2549: Hadoop 17428. ABFS: Implementation for getContentSummary

bilaharith commented on a change in pull request #2549: URL: https://github.com/apache/hadoop/pull/2549#discussion_r554925918 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/ContentSummaryProcessor.java ## @@ -0,0 +1,119 @@ +/** + *

[GitHub] [hadoop] bilaharith commented on a change in pull request #2549: Hadoop 17428. ABFS: Implementation for getContentSummary

bilaharith commented on a change in pull request #2549: URL: https://github.com/apache/hadoop/pull/2549#discussion_r554927643 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/ContentSummaryProcessor.java ## @@ -0,0 +1,119 @@ +/** + *

[GitHub] [hadoop] bilaharith commented on a change in pull request #2549: Hadoop 17428. ABFS: Implementation for getContentSummary

bilaharith commented on a change in pull request #2549: URL: https://github.com/apache/hadoop/pull/2549#discussion_r554927643 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/ContentSummaryProcessor.java ## @@ -0,0 +1,119 @@ +/** + *

[GitHub] [hadoop] bilaharith commented on a change in pull request #2549: Hadoop 17428. ABFS: Implementation for getContentSummary

bilaharith commented on a change in pull request #2549: URL: https://github.com/apache/hadoop/pull/2549#discussion_r554932129 ## File path: hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestGetContentSummary.java ## @@ -0,0 +1,204 @@ +/** + * L

[GitHub] [hadoop] bilaharith commented on a change in pull request #2549: Hadoop 17428. ABFS: Implementation for getContentSummary

bilaharith commented on a change in pull request #2549: URL: https://github.com/apache/hadoop/pull/2549#discussion_r554932969 ## File path: hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestGetContentSummary.java ## @@ -0,0 +1,204 @@ +/** + * L

[GitHub] [hadoop] bilaharith commented on a change in pull request #2549: Hadoop 17428. ABFS: Implementation for getContentSummary

bilaharith commented on a change in pull request #2549: URL: https://github.com/apache/hadoop/pull/2549#discussion_r554934952 ## File path: hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestGetContentSummary.java ## @@ -0,0 +1,204 @@ +/** + * L

[GitHub] [hadoop] bilaharith commented on a change in pull request #2549: Hadoop 17428. ABFS: Implementation for getContentSummary

bilaharith commented on a change in pull request #2549: URL: https://github.com/apache/hadoop/pull/2549#discussion_r554935350 ## File path: hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestGetContentSummary.java ## @@ -0,0 +1,204 @@ +/** + * L

[GitHub] [hadoop] hadoop-yetus commented on pull request #2583: HDFS-15549. Improve DISK/ARCHIVE movement if they are on same filesystem

hadoop-yetus commented on pull request #2583: URL: https://github.com/apache/hadoop/pull/2583#issuecomment-757908931 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[jira] [Updated] (HADOOP-16947) Stale record should be remove when MutableRollingAverages generating aggregate data.

[ https://issues.apache.org/jira/browse/HADOOP-16947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibin Huang updated HADOOP-16947: -- Attachment: HADOOP-16947-012.patch > Stale record should be remove when MutableRollingAverage

[jira] [Updated] (HADOOP-16947) Stale record should be remove when MutableRollingAverages generating aggregate data.

[ https://issues.apache.org/jira/browse/HADOOP-16947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibin Huang updated HADOOP-16947: -- Attachment: (was: HADOOP-16947-012.patch) > Stale record should be remove when MutableRol

[jira] [Updated] (HADOOP-16492) Support HuaweiCloud Object Storage as a Hadoop Backend File System

[ https://issues.apache.org/jira/browse/HADOOP-16492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brahma Reddy Battula updated HADOOP-16492: -- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Release Note: Added supp

[GitHub] [hadoop] amahussein commented on pull request #2581: YARN-10553. Refactor TestDistributedShell

amahussein commented on pull request #2581: URL: https://github.com/apache/hadoop/pull/2581#issuecomment-757950935 > Thanks for the update, @amahussein. +1. Thanks for the review @iwasakims and @goiri This is an auto

[GitHub] [hadoop] steveloughran commented on pull request #2594: HADOOP-17451. IOStatistics test failures in S3A code.

steveloughran commented on pull request #2594: URL: https://github.com/apache/hadoop/pull/2594#issuecomment-757952165 last yetus run had execution problems. Rebase to trunk and resubmitting This is an automated message from t

[jira] [Work logged] (HADOOP-17451) intermittent failure of S3A tests which make assertions on statistics/IOStatistics

[ https://issues.apache.org/jira/browse/HADOOP-17451?focusedWorklogId=534232&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534232 ] ASF GitHub Bot logged work on HADOOP-17451: --- Author: ASF GitHub B

[GitHub] [hadoop] amahussein commented on a change in pull request #2598: HDFS-15762. TestMultipleNNPortQOP#testMultipleNNPortOverwriteDownStre…

amahussein commented on a change in pull request #2598: URL: https://github.com/apache/hadoop/pull/2598#discussion_r555047228 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestMultipleNNPortQOP.java ## @@ -251,62 +252,48 @@ public void test

[GitHub] [hadoop] amahussein commented on a change in pull request #2598: HDFS-15762. TestMultipleNNPortQOP#testMultipleNNPortOverwriteDownStre…

amahussein commented on a change in pull request #2598: URL: https://github.com/apache/hadoop/pull/2598#discussion_r555048475 ## File path: hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/sasl/SaslDataTransferClient.java ## @@

[jira] [Commented] (HADOOP-17463) Replace currentTimeMillis with monotonicNow in elapsed time

[ https://issues.apache.org/jira/browse/HADOOP-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17262646#comment-17262646 ] Ahmed Hussein commented on HADOOP-17463: I created YARN-10566 to fix the change

[GitHub] [hadoop] amahussein commented on pull request #2560: HADOOP-17438. Increase docker memory limit in Jenkins.

amahussein commented on pull request #2560: URL: https://github.com/apache/hadoop/pull/2560#issuecomment-757970031 @aajisaka , @ericbadger Any thoughts whether we should move fwd with this PR? This is an automated message

[GitHub] [hadoop] amahussein edited a comment on pull request #2560: HADOOP-17438. Increase docker memory limit in Jenkins.

amahussein edited a comment on pull request #2560: URL: https://github.com/apache/hadoop/pull/2560#issuecomment-757970031 @aajisaka , @ericbadger Any thoughts whether we should move fwd with this PR? The error still exists as per January 10th build https://ci-hadoop.apache.org/job/ha

[jira] [Work logged] (HADOOP-17438) Increase docker memory limit in Jenkins

[ https://issues.apache.org/jira/browse/HADOOP-17438?focusedWorklogId=534242&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534242 ] ASF GitHub Bot logged work on HADOOP-17438: --- Author: ASF GitHub B

[jira] [Work logged] (HADOOP-17438) Increase docker memory limit in Jenkins

[ https://issues.apache.org/jira/browse/HADOOP-17438?focusedWorklogId=534243&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534243 ] ASF GitHub Bot logged work on HADOOP-17438: --- Author: ASF GitHub B

[GitHub] [hadoop] amahussein edited a comment on pull request #2560: HADOOP-17438. Increase docker memory limit in Jenkins.

amahussein edited a comment on pull request #2560: URL: https://github.com/apache/hadoop/pull/2560#issuecomment-757970031 @aajisaka , @ericbadger Any thoughts whether we should move fwd with this PR? The error still exists as per January 10th build https://ci-hadoop.apache.org/job/ha

[jira] [Work logged] (HADOOP-17438) Increase docker memory limit in Jenkins

[ https://issues.apache.org/jira/browse/HADOOP-17438?focusedWorklogId=534246&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534246 ] ASF GitHub Bot logged work on HADOOP-17438: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on a change in pull request #2530: HADOOP-17414. Magic committer files don't have the count of bytes written collected by spark

steveloughran commented on a change in pull request #2530: URL: https://github.com/apache/hadoop/pull/2530#discussion_r555075093 ## File path: hadoop-common-project/hadoop-common/src/main/resources/core-default.xml ## @@ -1873,11 +1873,9 @@ fs.s3a.committer.magic.enable

[jira] [Work logged] (HADOOP-17414) Magic committer files don't have the count of bytes written collected by spark

[ https://issues.apache.org/jira/browse/HADOOP-17414?focusedWorklogId=534255&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534255 ] ASF GitHub Bot logged work on HADOOP-17414: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on a change in pull request #2530: HADOOP-17414. Magic committer files don't have the count of bytes written collected by spark

steveloughran commented on a change in pull request #2530: URL: https://github.com/apache/hadoop/pull/2530#discussion_r555080742 ## File path: hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/HeaderProcessing.java ## @@ -0,0 +1,300 @@ +/* + * Licensed to the

[jira] [Work logged] (HADOOP-17414) Magic committer files don't have the count of bytes written collected by spark

[ https://issues.apache.org/jira/browse/HADOOP-17414?focusedWorklogId=534261&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534261 ] ASF GitHub Bot logged work on HADOOP-17414: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus commented on pull request #2594: HADOOP-17451. IOStatistics test failures in S3A code.

hadoop-yetus commented on pull request #2594: URL: https://github.com/apache/hadoop/pull/2594#issuecomment-757991015 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[jira] [Work logged] (HADOOP-17451) intermittent failure of S3A tests which make assertions on statistics/IOStatistics

[ https://issues.apache.org/jira/browse/HADOOP-17451?focusedWorklogId=534266&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534266 ] ASF GitHub Bot logged work on HADOOP-17451: --- Author: ASF GitHub B

[GitHub] [hadoop] mukund-thakur commented on a change in pull request #2594: HADOOP-17451. IOStatistics test failures in S3A code.

mukund-thakur commented on a change in pull request #2594: URL: https://github.com/apache/hadoop/pull/2594#discussion_r555093592 ## File path: hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AInstrumentation.java ## @@ -1470,7 +1456,7 @@ public void blockUploa

[jira] [Work logged] (HADOOP-17451) intermittent failure of S3A tests which make assertions on statistics/IOStatistics

[ https://issues.apache.org/jira/browse/HADOOP-17451?focusedWorklogId=534272&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534272 ] ASF GitHub Bot logged work on HADOOP-17451: --- Author: ASF GitHub B

[GitHub] [hadoop] touchida commented on a change in pull request #2598: HDFS-15762. TestMultipleNNPortQOP#testMultipleNNPortOverwriteDownStre…

touchida commented on a change in pull request #2598: URL: https://github.com/apache/hadoop/pull/2598#discussion_r555139610 ## File path: hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/sasl/SaslDataTransferClient.java ## @@ -4

[jira] [Commented] (HADOOP-16524) Automatic keystore reloading for HttpServer2

[ https://issues.apache.org/jira/browse/HADOOP-16524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17262765#comment-17262765 ] Ahmed Hussein commented on HADOOP-16524: [~stack] Can you please take a look at

[GitHub] [hadoop] amahussein commented on pull request #2609: HADOOP-16524. Reloading SSL keystore for both DataNode and NameNode (…

amahussein commented on pull request #2609: URL: https://github.com/apache/hadoop/pull/2609#issuecomment-758071912 @saintstack Thanks for your contribution. This PR caused failures to other unit tests. Can you please take a look to the list of failures. https://issues.apache.org/jira

[GitHub] [hadoop] amahussein commented on pull request #2470: HADOOP-16524. Reloading SSL keystore for both DataNode and NameNode

amahussein commented on pull request #2470: URL: https://github.com/apache/hadoop/pull/2470#issuecomment-758072073 @saintstack Thanks for your contribution. This PR caused failures to other unit tests. Can you please take a look to the list of failures. https://issues.apache.org/jira

[jira] [Work logged] (HADOOP-16524) Automatic keystore reloading for HttpServer2

[ https://issues.apache.org/jira/browse/HADOOP-16524?focusedWorklogId=534327&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534327 ] ASF GitHub Bot logged work on HADOOP-16524: --- Author: ASF GitHub B

[jira] [Work logged] (HADOOP-16524) Automatic keystore reloading for HttpServer2

[ https://issues.apache.org/jira/browse/HADOOP-16524?focusedWorklogId=534328&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534328 ] ASF GitHub Bot logged work on HADOOP-16524: --- Author: ASF GitHub B

[jira] [Commented] (HADOOP-16524) Automatic keystore reloading for HttpServer2

[ https://issues.apache.org/jira/browse/HADOOP-16524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17262777#comment-17262777 ] Michael Stack commented on HADOOP-16524: Sorry about that. Thanks for the ping.

[jira] [Commented] (HADOOP-16524) Automatic keystore reloading for HttpServer2

[ https://issues.apache.org/jira/browse/HADOOP-16524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17262785#comment-17262785 ] Michael Stack commented on HADOOP-16524: Reverted from trunk and branch-3.3. >

[jira] [Reopened] (HADOOP-16524) Automatic keystore reloading for HttpServer2

[ https://issues.apache.org/jira/browse/HADOOP-16524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack reopened HADOOP-16524: Reopening to revert change until fix failing yarn test. > Automatic keystore reloading for H

[GitHub] [hadoop] steveloughran commented on a change in pull request #2530: HADOOP-17414. Magic committer files don't have the count of bytes written collected by spark

steveloughran commented on a change in pull request #2530: URL: https://github.com/apache/hadoop/pull/2530#discussion_r555204308 ## File path: hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/Constants.java ## @@ -1048,4 +1048,10 @@ private Constants() { publi

[jira] [Work logged] (HADOOP-17414) Magic committer files don't have the count of bytes written collected by spark

[ https://issues.apache.org/jira/browse/HADOOP-17414?focusedWorklogId=534352&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534352 ] ASF GitHub Bot logged work on HADOOP-17414: --- Author: ASF GitHub B

[GitHub] [hadoop] goiri commented on a change in pull request #2605: HDFS-15423 RBF: WebHDFS create shouldn't choose DN from all sub-clusters

goiri commented on a change in pull request #2605: URL: https://github.com/apache/hadoop/pull/2605#discussion_r555222518 ## File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestRouterWebHdfsMethods.java ## @@ -0,0 +1,

[GitHub] [hadoop] mehakmeet commented on pull request #2604: HADOOP-17272. ABFS Streams to support IOStatistics API

mehakmeet commented on pull request #2604: URL: https://github.com/apache/hadoop/pull/2604#issuecomment-758120852 Checkstyle issues are due to static imports having ".*", but these are enums, should I go this route or add the class name before their name(adds a few bits in the code, but av

[jira] [Work logged] (HADOOP-17272) ABFS Streams to support IOStatistics API

[ https://issues.apache.org/jira/browse/HADOOP-17272?focusedWorklogId=534381&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534381 ] ASF GitHub Bot logged work on HADOOP-17272: --- Author: ASF GitHub B

[jira] [Commented] (HADOOP-16947) Stale record should be remove when MutableRollingAverages generating aggregate data.

[ https://issues.apache.org/jira/browse/HADOOP-16947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17262834#comment-17262834 ] Hadoop QA commented on HADOOP-16947: | (x) *{color:red}-1 overall{color}* | \\ \\ |

[GitHub] [hadoop] steveloughran commented on a change in pull request #2530: HADOOP-17414. Magic committer files don't have the count of bytes written collected by spark

steveloughran commented on a change in pull request #2530: URL: https://github.com/apache/hadoop/pull/2530#discussion_r555242848 ## File path: hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java ## @@ -4103,56 +4111,7 @@ private ObjectMetadata cl

[jira] [Work logged] (HADOOP-17414) Magic committer files don't have the count of bytes written collected by spark

[ https://issues.apache.org/jira/browse/HADOOP-17414?focusedWorklogId=534387&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534387 ] ASF GitHub Bot logged work on HADOOP-17414: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on a change in pull request #2530: HADOOP-17414. Magic committer files don't have the count of bytes written collected by spark

steveloughran commented on a change in pull request #2530: URL: https://github.com/apache/hadoop/pull/2530#discussion_r555245870 ## File path: hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/HeaderProcessing.java ## @@ -0,0 +1,300 @@ +/* + * Licensed to the

[jira] [Work logged] (HADOOP-17414) Magic committer files don't have the count of bytes written collected by spark

[ https://issues.apache.org/jira/browse/HADOOP-17414?focusedWorklogId=534392&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534392 ] ASF GitHub Bot logged work on HADOOP-17414: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on a change in pull request #2530: HADOOP-17414. Magic committer files don't have the count of bytes written collected by spark

steveloughran commented on a change in pull request #2530: URL: https://github.com/apache/hadoop/pull/2530#discussion_r555247027 ## File path: hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/committer_architecture.md ## @@ -1312,6 +1312,16 @@ On `close()`, summary d

[jira] [Work logged] (HADOOP-17414) Magic committer files don't have the count of bytes written collected by spark

[ https://issues.apache.org/jira/browse/HADOOP-17414?focusedWorklogId=534394&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534394 ] ASF GitHub Bot logged work on HADOOP-17414: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on a change in pull request #2530: HADOOP-17414. Magic committer files don't have the count of bytes written collected by spark

steveloughran commented on a change in pull request #2530: URL: https://github.com/apache/hadoop/pull/2530#discussion_r555252337 ## File path: hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java ## @@ -330,6 +331,11 @@ */ private DirectoryP

[jira] [Work logged] (HADOOP-17414) Magic committer files don't have the count of bytes written collected by spark

[ https://issues.apache.org/jira/browse/HADOOP-17414?focusedWorklogId=534399&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534399 ] ASF GitHub Bot logged work on HADOOP-17414: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on a change in pull request #2530: HADOOP-17414. Magic committer files don't have the count of bytes written collected by spark

steveloughran commented on a change in pull request #2530: URL: https://github.com/apache/hadoop/pull/2530#discussion_r555253967 ## File path: hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/HeaderProcessing.java ## @@ -0,0 +1,300 @@ +/* + * Licensed to the

[jira] [Work logged] (HADOOP-17414) Magic committer files don't have the count of bytes written collected by spark

[ https://issues.apache.org/jira/browse/HADOOP-17414?focusedWorklogId=534402&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534402 ] ASF GitHub Bot logged work on HADOOP-17414: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on pull request #2530: HADOOP-17414. Magic committer files don't have the count of bytes written collected by spark

steveloughran commented on pull request #2530: URL: https://github.com/apache/hadoop/pull/2530#issuecomment-758139586 > Looks good to me overall. Using XAttr is smart and extensible for solving problems like this - and it breaks nothing! thanks,. biggest risk is actually the copy cod

[jira] [Work logged] (HADOOP-17414) Magic committer files don't have the count of bytes written collected by spark

[ https://issues.apache.org/jira/browse/HADOOP-17414?focusedWorklogId=534405&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534405 ] ASF GitHub Bot logged work on HADOOP-17414: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on pull request #2530: HADOOP-17414. Magic committer files don't have the count of bytes written collected by spark

steveloughran commented on pull request #2530: URL: https://github.com/apache/hadoop/pull/2530#issuecomment-758146786 Next PR to come in * tries to address all review comments * adds stats gathering * adds almost all the AWS headers (everything but some of the encryption stuff) as

[jira] [Work logged] (HADOOP-17414) Magic committer files don't have the count of bytes written collected by spark

[ https://issues.apache.org/jira/browse/HADOOP-17414?focusedWorklogId=534421&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534421 ] ASF GitHub Bot logged work on HADOOP-17414: --- Author: ASF GitHub B

[GitHub] [hadoop] amahussein opened a new pull request #2612: YARN-10566. Elapsed time should be measured monotonicNow

amahussein opened a new pull request #2612: URL: https://github.com/apache/hadoop/pull/2612 For Yarn module, Use `Time.monotonicNow()` instead of `System.currentTimeMillis()` to measure elapsed time. This is an autom

[GitHub] [hadoop] steveloughran commented on pull request #2530: HADOOP-17414. Magic committer files don't have the count of bytes written collected by spark

steveloughran commented on pull request #2530: URL: https://github.com/apache/hadoop/pull/2530#issuecomment-758165380 (testing in progress) This is an automated message from the Apache Git Service. To respond to the message,

[jira] [Work logged] (HADOOP-17414) Magic committer files don't have the count of bytes written collected by spark

[ https://issues.apache.org/jira/browse/HADOOP-17414?focusedWorklogId=534465&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534465 ] ASF GitHub Bot logged work on HADOOP-17414: --- Author: ASF GitHub B

[GitHub] [hadoop] fengnanli commented on pull request #2605: HDFS-15423 RBF: WebHDFS create shouldn't choose DN from all sub-clusters

fengnanli commented on pull request #2605: URL: https://github.com/apache/hadoop/pull/2605#issuecomment-758186706 @goiri Thanks for the further review. I found these TestRouterWebHdfsContractXX are flaky with my change and I think they are related with the fact that for the MiniRouterDF

[GitHub] [hadoop] fengnanli edited a comment on pull request #2605: HDFS-15423 RBF: WebHDFS create shouldn't choose DN from all sub-clusters

fengnanli edited a comment on pull request #2605: URL: https://github.com/apache/hadoop/pull/2605#issuecomment-758186706 @goiri Thanks for the further review. I found these TestRouterWebHdfsContractXX are flaky with my change and I think they are related with the fact that for the MiniR

[GitHub] [hadoop] goiri commented on pull request #2605: HDFS-15423 RBF: WebHDFS create shouldn't choose DN from all sub-clusters

goiri commented on pull request #2605: URL: https://github.com/apache/hadoop/pull/2605#issuecomment-758187791 > @goiri Thanks for the further review. > I found these TestRouterWebHdfsContractXX are flaky with my change and I think they are related with the fact that for the MiniRouterDFS

[GitHub] [hadoop] goiri merged pull request #2507: HDFS-15702. Fix intermittent falilure of TestDecommission#testAllocAndIBRWhileDecommission.

goiri merged pull request #2507: URL: https://github.com/apache/hadoop/pull/2507 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hadoop] hadoop-yetus commented on pull request #2604: HADOOP-17272. ABFS Streams to support IOStatistics API

hadoop-yetus commented on pull request #2604: URL: https://github.com/apache/hadoop/pull/2604#issuecomment-758220929 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[jira] [Work logged] (HADOOP-17272) ABFS Streams to support IOStatistics API

[ https://issues.apache.org/jira/browse/HADOOP-17272?focusedWorklogId=534536&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534536 ] ASF GitHub Bot logged work on HADOOP-17272: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus commented on pull request #2605: HDFS-15423 RBF: WebHDFS create shouldn't choose DN from all sub-clusters

hadoop-yetus commented on pull request #2605: URL: https://github.com/apache/hadoop/pull/2605#issuecomment-758245360 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] hadoop-yetus commented on pull request #2530: HADOOP-17414. Magic committer files don't have the count of bytes written collected by spark

hadoop-yetus commented on pull request #2530: URL: https://github.com/apache/hadoop/pull/2530#issuecomment-758262157 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[jira] [Work logged] (HADOOP-17414) Magic committer files don't have the count of bytes written collected by spark

[ https://issues.apache.org/jira/browse/HADOOP-17414?focusedWorklogId=534572&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534572 ] ASF GitHub Bot logged work on HADOOP-17414: --- Author: ASF GitHub B

[GitHub] [hadoop] iwasakims commented on pull request #2507: HDFS-15702. Fix intermittent falilure of TestDecommission#testAllocAndIBRWhileDecommission.

iwasakims commented on pull request #2507: URL: https://github.com/apache/hadoop/pull/2507#issuecomment-758285519 Thanks, @amahussein and @goiri. This is an automated message from the Apache Git Service. To respond to the mes

[GitHub] [hadoop] eric-badger commented on pull request #2560: HADOOP-17438. Increase docker memory limit in Jenkins.

eric-badger commented on pull request #2560: URL: https://github.com/apache/hadoop/pull/2560#issuecomment-758288185 It doesn't really hurt anything to ask for more memory unless we tell it to touch all pages or unless we fill all of the memory. If we fill all of the memory, then I guess th

[jira] [Work logged] (HADOOP-17438) Increase docker memory limit in Jenkins

[ https://issues.apache.org/jira/browse/HADOOP-17438?focusedWorklogId=534604&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534604 ] ASF GitHub Bot logged work on HADOOP-17438: --- Author: ASF GitHub B

[GitHub] [hadoop] mehakmeet edited a comment on pull request #2604: HADOOP-17272. ABFS Streams to support IOStatistics API

mehakmeet edited a comment on pull request #2604: URL: https://github.com/apache/hadoop/pull/2604#issuecomment-758120852 Checkstyle issues are due to static imports having ".*", but these are enums, should I go this route or add the class name before their name(adds a few bits in the code,

[jira] [Work logged] (HADOOP-17272) ABFS Streams to support IOStatistics API

[ https://issues.apache.org/jira/browse/HADOOP-17272?focusedWorklogId=534605&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534605 ] ASF GitHub Bot logged work on HADOOP-17272: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus commented on pull request #2583: HDFS-15549. Improve DISK/ARCHIVE movement if they are on same filesystem

hadoop-yetus commented on pull request #2583: URL: https://github.com/apache/hadoop/pull/2583#issuecomment-758325969 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] hadoop-yetus commented on pull request #2612: YARN-10566. Elapsed time should be measured monotonicNow

hadoop-yetus commented on pull request #2612: URL: https://github.com/apache/hadoop/pull/2612#issuecomment-758331151 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[jira] [Commented] (HADOOP-16947) Stale record should be remove when MutableRollingAverages generating aggregate data.

[ https://issues.apache.org/jira/browse/HADOOP-16947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17263031#comment-17263031 ] Haibin Huang commented on HADOOP-16947: --- Thanks [~coconut_icecream] for comment,

[jira] [Comment Edited] (HADOOP-16947) Stale record should be remove when MutableRollingAverages generating aggregate data.

[ https://issues.apache.org/jira/browse/HADOOP-16947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17263031#comment-17263031 ] Haibin Huang edited comment on HADOOP-16947 at 1/12/21, 2:35 AM:

[GitHub] [hadoop] hadoop-yetus commented on pull request #2612: YARN-10566. Elapsed time should be measured monotonicNow

hadoop-yetus commented on pull request #2612: URL: https://github.com/apache/hadoop/pull/2612#issuecomment-758388464 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[GitHub] [hadoop] sumangala-patki commented on a change in pull request #2549: Hadoop 17428. ABFS: Implementation for getContentSummary

sumangala-patki commented on a change in pull request #2549: URL: https://github.com/apache/hadoop/pull/2549#discussion_r27292 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystem.java ## @@ -363,6 +367,22 @@ public boolea

[GitHub] [hadoop] sumangala-patki commented on a change in pull request #2549: Hadoop 17428. ABFS: Implementation for getContentSummary

sumangala-patki commented on a change in pull request #2549: URL: https://github.com/apache/hadoop/pull/2549#discussion_r27399 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/ContentSummaryProcessor.java ## @@ -0,0 +1,119 @@ +/*

[GitHub] [hadoop] sumangala-patki commented on a change in pull request #2549: Hadoop 17428. ABFS: Implementation for getContentSummary

sumangala-patki commented on a change in pull request #2549: URL: https://github.com/apache/hadoop/pull/2549#discussion_r27632 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/ContentSummaryProcessor.java ## @@ -0,0 +1,119 @@ +/*

[GitHub] [hadoop] sumangala-patki commented on a change in pull request #2549: Hadoop 17428. ABFS: Implementation for getContentSummary

sumangala-patki commented on a change in pull request #2549: URL: https://github.com/apache/hadoop/pull/2549#discussion_r27694 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/ContentSummaryProcessor.java ## @@ -0,0 +1,119 @@ +/*

[GitHub] [hadoop] sumangala-patki commented on a change in pull request #2549: Hadoop 17428. ABFS: Implementation for getContentSummary

sumangala-patki commented on a change in pull request #2549: URL: https://github.com/apache/hadoop/pull/2549#discussion_r27894 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/ContentSummaryProcessor.java ## @@ -0,0 +1,119 @@ +/*

[GitHub] [hadoop] sumangala-patki commented on a change in pull request #2549: Hadoop 17428. ABFS: Implementation for getContentSummary

sumangala-patki commented on a change in pull request #2549: URL: https://github.com/apache/hadoop/pull/2549#discussion_r28100 ## File path: hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestGetContentSummary.java ## @@ -0,0 +1,204 @@ +/**

[GitHub] [hadoop] sumangala-patki commented on a change in pull request #2549: Hadoop 17428. ABFS: Implementation for getContentSummary

sumangala-patki commented on a change in pull request #2549: URL: https://github.com/apache/hadoop/pull/2549#discussion_r28200 ## File path: hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestGetContentSummary.java ## @@ -0,0 +1,204 @@ +/**

[GitHub] [hadoop] aajisaka commented on pull request #2560: HADOOP-17438. Increase docker memory limit in Jenkins.

aajisaka commented on pull request #2560: URL: https://github.com/apache/hadoop/pull/2560#issuecomment-758437944 > So I guess go for it. Though I still think this is addressing a symptom, not the root cause I have the same opinion. --

[jira] [Work logged] (HADOOP-17438) Increase docker memory limit in Jenkins

[ https://issues.apache.org/jira/browse/HADOOP-17438?focusedWorklogId=534682&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534682 ] ASF GitHub Bot logged work on HADOOP-17438: --- Author: ASF GitHub B

[GitHub] [hadoop] aajisaka commented on pull request #2560: HADOOP-17438. Increase docker memory limit in Jenkins.

aajisaka commented on pull request #2560: URL: https://github.com/apache/hadoop/pull/2560#issuecomment-758439351 Merged. Thank you @amahussein and @eric-badger This is an automated message from the Apache Git Service. To res

[jira] [Work logged] (HADOOP-17438) Increase docker memory limit in Jenkins

[ https://issues.apache.org/jira/browse/HADOOP-17438?focusedWorklogId=534684&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534684 ] ASF GitHub Bot logged work on HADOOP-17438: --- Author: ASF GitHub B

[jira] [Work logged] (HADOOP-17438) Increase docker memory limit in Jenkins

[ https://issues.apache.org/jira/browse/HADOOP-17438?focusedWorklogId=534687&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534687 ] ASF GitHub Bot logged work on HADOOP-17438: --- Author: ASF GitHub B

[jira] [Resolved] (HADOOP-17438) Increase docker memory limit in Jenkins

[ https://issues.apache.org/jira/browse/HADOOP-17438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka resolved HADOOP-17438. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Increase dock

[jira] [Updated] (HADOOP-17438) Increase docker memory limit in Jenkins

[ https://issues.apache.org/jira/browse/HADOOP-17438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-17438: --- Fix Version/s: 3.2.3 2.10.2 3.1.5 3

[jira] [Commented] (HADOOP-17438) Increase docker memory limit in Jenkins

[ https://issues.apache.org/jira/browse/HADOOP-17438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17263111#comment-17263111 ] Akira Ajisaka commented on HADOOP-17438: Merged the PR into all the active bran

[GitHub] [hadoop] goiri commented on a change in pull request #2567: HDFS-15740. Add x-platform utilities

goiri commented on a change in pull request #2567: URL: https://github.com/apache/hadoop/pull/2567#discussion_r44085 ## File path: dev-support/docker/Dockerfile ## @@ -92,6 +96,21 @@ ENV MAVEN_HOME /usr ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64 ENV FINDBUGS_HOME /us

  1   2   >