[GitHub] [hadoop] liuml07 commented on pull request #2799: HDFS-15911 : Provide blocks moved count in Balancer iteration result

2021-03-23 Thread GitBox
liuml07 commented on pull request #2799: URL: https://github.com/apache/hadoop/pull/2799#issuecomment-805522423 Jenkins fails with unrelated problems. Maybe you can rebase and trigger the Jenkins again. Thanks, -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [hadoop] liuml07 commented on pull request #2797: HDFS-15911 : Provide blocks moved count in Balancer iteration result

2021-03-23 Thread GitBox
liuml07 commented on pull request #2797: URL: https://github.com/apache/hadoop/pull/2797#issuecomment-805521844 @virajith Could you run local tests for the failing hadoop.hdfs.server.balancer.TestBalancerWithHANameNodes and confirm that is not related to this? Thanks, -- This is an

[jira] [Work logged] (HADOOP-17133) Implement HttpServer2 metrics

2021-03-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17133?focusedWorklogId=570951=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570951 ] ASF GitHub Bot logged work on HADOOP-17133: --- Author: ASF GitHub Bot

[GitHub] [hadoop] aajisaka commented on a change in pull request #2145: HADOOP-17133. Implement HttpServer2 metrics

2021-03-23 Thread GitBox
aajisaka commented on a change in pull request #2145: URL: https://github.com/apache/hadoop/pull/2145#discussion_r600192098 ## File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/http/HttpServer2.java ## @@ -669,6 +674,16 @@ private void

[GitHub] [hadoop] liuml07 merged pull request #2796: HDFS-15911 : Provide blocks moved count in Balancer iteration result

2021-03-23 Thread GitBox
liuml07 merged pull request #2796: URL: https://github.com/apache/hadoop/pull/2796 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [hadoop] liuml07 merged pull request #2794: HDFS-15911 : Provide blocks moved count in Balancer iteration result

2021-03-23 Thread GitBox
liuml07 merged pull request #2794: URL: https://github.com/apache/hadoop/pull/2794 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[jira] [Work logged] (HADOOP-17133) Implement HttpServer2 metrics

2021-03-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17133?focusedWorklogId=570929=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570929 ] ASF GitHub Bot logged work on HADOOP-17133: --- Author: ASF GitHub Bot

[jira] [Work logged] (HADOOP-17133) Implement HttpServer2 metrics

2021-03-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17133?focusedWorklogId=570928=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570928 ] ASF GitHub Bot logged work on HADOOP-17133: --- Author: ASF GitHub Bot

[GitHub] [hadoop] aajisaka commented on a change in pull request #2145: HADOOP-17133. Implement HttpServer2 metrics

2021-03-23 Thread GitBox
aajisaka commented on a change in pull request #2145: URL: https://github.com/apache/hadoop/pull/2145#discussion_r600160475 ## File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/http/HttpServer2.java ## @@ -669,6 +674,16 @@ private void

[GitHub] [hadoop] aajisaka commented on a change in pull request #2145: HADOOP-17133. Implement HttpServer2 metrics

2021-03-23 Thread GitBox
aajisaka commented on a change in pull request #2145: URL: https://github.com/apache/hadoop/pull/2145#discussion_r600160149 ## File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/http/HttpServer2.java ## @@ -1227,6 +1242,16 @@ public void start()

[jira] [Commented] (HADOOP-17600) Time.monotonicNow() should be used to compute time interval, but not Time.now()

2021-03-23 Thread Jiajun Jiang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17307531#comment-17307531 ] Jiajun Jiang commented on HADOOP-17600: --- Please review the attached patch. > Time.monotonicNow()

[jira] [Updated] (HADOOP-17600) Time.monotonicNow() should be used to compute time interval, but not Time.now()

2021-03-23 Thread Jiajun Jiang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiajun Jiang updated HADOOP-17600: -- Attachment: HADOOP-17600.patch > Time.monotonicNow() should be used to compute time

[jira] [Work logged] (HADOOP-17600) Time.monotonicNow() should be used to compute time interval, but not Time.now()

2021-03-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17600?focusedWorklogId=570912=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570912 ] ASF GitHub Bot logged work on HADOOP-17600: --- Author: ASF GitHub Bot

[jira] [Updated] (HADOOP-17600) Time.monotonicNow() should be used to compute time interval, but not Time.now()

2021-03-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-17600: Labels: pull-request-available (was: ) > Time.monotonicNow() should be used to compute

[GitHub] [hadoop] TJJack opened a new pull request #2810: HADOOP-17600. Replace Time.now() with Time.monotonicNow() for measuring time durations

2021-03-23 Thread GitBox
TJJack opened a new pull request #2810: URL: https://github.com/apache/hadoop/pull/2810 JIRA ISSUE: [https://issues.apache.org/jira/browse/HADOOP-17600](https://issues.apache.org/jira/browse/HADOOP-17600) -- This is an automated message from the Apache Git Service. To respond to the

[jira] [Created] (HADOOP-17600) Time.monotonicNow() should be used to compute time interval, but not Time.now()

2021-03-23 Thread Jiajun Jiang (Jira)
Jiajun Jiang created HADOOP-17600: - Summary: Time.monotonicNow() should be used to compute time interval, but not Time.now() Key: HADOOP-17600 URL: https://issues.apache.org/jira/browse/HADOOP-17600

[GitHub] [hadoop] hadoop-yetus commented on pull request #2585: HDFS-15759. EC: Verify EC reconstruction correctness on DataNode

2021-03-23 Thread GitBox
hadoop-yetus commented on pull request #2585: URL: https://github.com/apache/hadoop/pull/2585#issuecomment-805441289 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] hadoop-yetus commented on pull request #2809: HDFS-15914. Possible Resource Leak in org.apache.hadoop.hdfs.server.common.blockaliasmap.impl.TextFileRegionAliasMap

2021-03-23 Thread GitBox
hadoop-yetus commented on pull request #2809: URL: https://github.com/apache/hadoop/pull/2809#issuecomment-805438031 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Work logged] (HADOOP-17531) DistCp: Reduce memory usage on copying huge directories

2021-03-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17531?focusedWorklogId=570870=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570870 ] ASF GitHub Bot logged work on HADOOP-17531: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #2808: HADOOP-17531. DistCp: Reduce memory usage on copying huge directories. (#2732).

2021-03-23 Thread GitBox
hadoop-yetus commented on pull request #2808: URL: https://github.com/apache/hadoop/pull/2808#issuecomment-805383059 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Work logged] (HADOOP-16948) ABFS: Support single writer dirs

2021-03-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16948?focusedWorklogId=570830=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570830 ] ASF GitHub Bot logged work on HADOOP-16948: --- Author: ASF GitHub Bot

[GitHub] [hadoop] billierinaldi commented on a change in pull request #1925: HADOOP-16948. Support single writer dirs.

2021-03-23 Thread GitBox
billierinaldi commented on a change in pull request #1925: URL: https://github.com/apache/hadoop/pull/1925#discussion_r57567 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsLease.java ## @@ -88,6 +88,12 @@ public

[jira] [Work logged] (HADOOP-17597) Add option to downgrade S3A rejection of Syncable to warning

2021-03-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17597?focusedWorklogId=570826=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570826 ] ASF GitHub Bot logged work on HADOOP-17597: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #2801: HADOOP-17597. Optionally downgrade on S3A Syncable calls

2021-03-23 Thread GitBox
hadoop-yetus commented on pull request #2801: URL: https://github.com/apache/hadoop/pull/2801#issuecomment-805304298 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] Nargeshdb opened a new pull request #2809: HDFS-15914. Possible Resource Leak in org.apache.hadoop.hdfs.server.common.blockaliasmap.impl.TextFileRegionAliasMap

2021-03-23 Thread GitBox
Nargeshdb opened a new pull request #2809: URL: https://github.com/apache/hadoop/pull/2809 This PR fixes the issue mentioned [here](https://issues.apache.org/jira/browse/HDFS-15914). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[jira] [Work logged] (HADOOP-17531) DistCp: Reduce memory usage on copying huge directories

2021-03-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17531?focusedWorklogId=570796=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570796 ] ASF GitHub Bot logged work on HADOOP-17531: --- Author: ASF GitHub Bot

[GitHub] [hadoop] ayushtkn opened a new pull request #2808: HADOOP-17531. DistCp: Reduce memory usage on copying huge directories. (#2732).

2021-03-23 Thread GitBox
ayushtkn opened a new pull request #2808: URL: https://github.com/apache/hadoop/pull/2808 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[jira] [Work logged] (HADOOP-17531) DistCp: Reduce memory usage on copying huge directories

2021-03-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17531?focusedWorklogId=570782=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570782 ] ASF GitHub Bot logged work on HADOOP-17531: --- Author: ASF GitHub Bot

[GitHub] [hadoop] ayushtkn merged pull request #2732: HADOOP-17531. DistCp: Reduce memory usage on copying huge directories.

2021-03-23 Thread GitBox
ayushtkn merged pull request #2732: URL: https://github.com/apache/hadoop/pull/2732 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [hadoop] hadoop-yetus commented on pull request #2748: HDFS-15879. Exclude slow nodes when choose targets for blocks

2021-03-23 Thread GitBox
hadoop-yetus commented on pull request #2748: URL: https://github.com/apache/hadoop/pull/2748#issuecomment-805173707 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Work logged] (HADOOP-17511) Add an Audit plugin point for S3A auditing/context

2021-03-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17511?focusedWorklogId=570727=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570727 ] ASF GitHub Bot logged work on HADOOP-17511: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #2807: HADOOP-17511. Add audit/telemetry logging to S3A connector

2021-03-23 Thread GitBox
hadoop-yetus commented on pull request #2807: URL: https://github.com/apache/hadoop/pull/2807#issuecomment-805169524 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Work logged] (HADOOP-17511) Add an Audit plugin point for S3A auditing/context

2021-03-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17511?focusedWorklogId=570715=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570715 ] ASF GitHub Bot logged work on HADOOP-17511: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on pull request #2807: HADOOP-17511. Add audit/telemetry logging to S3A connector

2021-03-23 Thread GitBox
steveloughran commented on pull request #2807: URL: https://github.com/apache/hadoop/pull/2807#issuecomment-805151587 its not merging and I've over-squashed things into the AWS metrics patch. Will need to unroll it -- This is an automated message from the Apache Git Service. To respond

[jira] [Work logged] (HADOOP-17531) DistCp: Reduce memory usage on copying huge directories

2021-03-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17531?focusedWorklogId=570706=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570706 ] ASF GitHub Bot logged work on HADOOP-17531: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on pull request #2732: HADOOP-17531. DistCp: Reduce memory usage on copying huge directories.

2021-03-23 Thread GitBox
steveloughran commented on pull request #2732: URL: https://github.com/apache/hadoop/pull/2732#issuecomment-805144616 OK, +1 from me. Merge to trunk and after a test run to branch-3.3; lets wait and see what surprises surface there -- This is an automated message from the Apache Git

[jira] [Work logged] (HADOOP-16948) ABFS: Support single writer dirs

2021-03-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16948?focusedWorklogId=570705=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570705 ] ASF GitHub Bot logged work on HADOOP-16948: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on a change in pull request #1925: HADOOP-16948. Support single writer dirs.

2021-03-23 Thread GitBox
steveloughran commented on a change in pull request #1925: URL: https://github.com/apache/hadoop/pull/1925#discussion_r599840802 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsLease.java ## @@ -88,6 +88,12 @@ public

[jira] [Work logged] (HADOOP-13551) Collect AwsSdkMetrics in S3A FileSystem IOStatistics

2021-03-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13551?focusedWorklogId=570703=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570703 ] ASF GitHub Bot logged work on HADOOP-13551: --- Author: ASF GitHub Bot

[jira] [Work logged] (HADOOP-13551) Collect AwsSdkMetrics in S3A FileSystem IOStatistics

2021-03-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13551?focusedWorklogId=570702=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570702 ] ASF GitHub Bot logged work on HADOOP-13551: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on pull request #2778: HADOOP-13551. AWS metrics wire-up

2021-03-23 Thread GitBox
steveloughran commented on pull request #2778: URL: https://github.com/apache/hadoop/pull/2778#issuecomment-805139914 > Have you added the doc changes because I don't see them here. There's a lot of stuff in the docs about endpoints, including the exception we've been seeing here.

[jira] [Work logged] (HADOOP-13551) Collect AwsSdkMetrics in S3A FileSystem IOStatistics

2021-03-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13551?focusedWorklogId=570701=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570701 ] ASF GitHub Bot logged work on HADOOP-13551: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #2778: HADOOP-13551. AWS metrics wire-up

2021-03-23 Thread GitBox
hadoop-yetus removed a comment on pull request #2778: URL: https://github.com/apache/hadoop/pull/2778#issuecomment-802097951 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:|

[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #2778: HADOOP-13551. AWS metrics wire-up

2021-03-23 Thread GitBox
hadoop-yetus removed a comment on pull request #2778: URL: https://github.com/apache/hadoop/pull/2778#issuecomment-802100977 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:|

[jira] [Work logged] (HADOOP-17476) ITestAssumeRole.testAssumeRoleBadInnerAuth failure

2021-03-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17476?focusedWorklogId=570700=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570700 ] ASF GitHub Bot logged work on HADOOP-17476: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on a change in pull request #2777: HADOOP-17476. ITestAssumeRole.testAssumeRoleBadInnerAuth failure.

2021-03-23 Thread GitBox
steveloughran commented on a change in pull request #2777: URL: https://github.com/apache/hadoop/pull/2777#discussion_r599835557 ## File path: hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/auth/ITestAssumeRole.java ## @@ -255,8 +255,7 @@ public void

[jira] [Work logged] (HADOOP-17138) Fix spotbugs warnings surfaced after upgrade to 4.0.6

2021-03-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17138?focusedWorklogId=570696=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570696 ] ASF GitHub Bot logged work on HADOOP-17138: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #2806: HADOOP-17138. Fix spotbugs warnings surfaced after upgrade to 4.0.6.

2021-03-23 Thread GitBox
hadoop-yetus commented on pull request #2806: URL: https://github.com/apache/hadoop/pull/2806#issuecomment-805132561 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] hadoop-yetus commented on pull request #2792: HDFS-15909. Make fnmatch cross platform

2021-03-23 Thread GitBox
hadoop-yetus commented on pull request #2792: URL: https://github.com/apache/hadoop/pull/2792#issuecomment-805102608 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Work logged] (HADOOP-17138) Fix spotbugs warnings surfaced after upgrade to 4.0.6

2021-03-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17138?focusedWorklogId=570649=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570649 ] ASF GitHub Bot logged work on HADOOP-17138: --- Author: ASF GitHub Bot

[GitHub] [hadoop] aajisaka commented on a change in pull request #2806: HADOOP-17138. Fix spotbugs warnings surfaced after upgrade to 4.0.6.

2021-03-23 Thread GitBox
aajisaka commented on a change in pull request #2806: URL: https://github.com/apache/hadoop/pull/2806#discussion_r599765175 ## File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java ## @@ -3716,7 +3716,7 @@ void

[GitHub] [hadoop] goiri commented on a change in pull request #2792: HDFS-15909. Make fnmatch cross platform

2021-03-23 Thread GitBox
goiri commented on a change in pull request #2792: URL: https://github.com/apache/hadoop/pull/2792#discussion_r599761580 ## File path: hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/fs/filesystem.cc ## @@ -722,16 +722,16 @@ void

[GitHub] [hadoop] goiri merged pull request #2793: HDFS-15910. Improve security with explicit_bzero

2021-03-23 Thread GitBox
goiri merged pull request #2793: URL: https://github.com/apache/hadoop/pull/2793 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[jira] [Work logged] (HADOOP-17511) Add an Audit plugin point for S3A auditing/context

2021-03-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17511?focusedWorklogId=570618=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570618 ] ASF GitHub Bot logged work on HADOOP-17511: --- Author: ASF GitHub Bot

[jira] [Work logged] (HADOOP-17511) Add an Audit plugin point for S3A auditing/context

2021-03-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17511?focusedWorklogId=570616=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570616 ] ASF GitHub Bot logged work on HADOOP-17511: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on pull request #2675: HADOOP-17511. Add audit/telemetry logging to S3A connector

2021-03-23 Thread GitBox
steveloughran commented on pull request #2675: URL: https://github.com/apache/hadoop/pull/2675#issuecomment-805044233 See #2807 for successor PR -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hadoop] hadoop-yetus commented on pull request #2807: HADOOP-17511. Add audit/telemetry logging to S3A connector

2021-03-23 Thread GitBox
hadoop-yetus commented on pull request #2807: URL: https://github.com/apache/hadoop/pull/2807#issuecomment-805044041 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] steveloughran opened a new pull request #2807: S3/hadoop 17511 auditing

2021-03-23 Thread GitBox
steveloughran opened a new pull request #2807: URL: https://github.com/apache/hadoop/pull/2807 Notion of AuditSpan which is created for a given operation; goal is to pass it along everywhere. It's thread local per FS-instance; store operations pick this up in their

[jira] [Work logged] (HADOOP-17511) Add an Audit plugin point for S3A auditing/context

2021-03-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17511?focusedWorklogId=570611=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570611 ] ASF GitHub Bot logged work on HADOOP-17511: --- Author: ASF GitHub Bot

[jira] [Work logged] (HADOOP-17511) Add an Audit plugin point for S3A auditing/context

2021-03-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17511?focusedWorklogId=570610=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570610 ] ASF GitHub Bot logged work on HADOOP-17511: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran closed pull request #2675: HADOOP-17511. Add audit/telemetry logging to S3A connector

2021-03-23 Thread GitBox
steveloughran closed pull request #2675: URL: https://github.com/apache/hadoop/pull/2675 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [hadoop] steveloughran commented on pull request #2675: HADOOP-17511. Add audit/telemetry logging to S3A connector

2021-03-23 Thread GitBox
steveloughran commented on pull request #2675: URL: https://github.com/apache/hadoop/pull/2675#issuecomment-805035373 PR no longer applies, even with rebase. Will have to close and resubmit -- This is an automated message from the Apache Git Service. To respond to the message, please log

[jira] [Work logged] (HADOOP-17511) Add an Audit plugin point for S3A auditing/context

2021-03-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17511?focusedWorklogId=570609=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570609 ] ASF GitHub Bot logged work on HADOOP-17511: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #2675: HADOOP-17511. Add audit/telemetry logging to S3A connector

2021-03-23 Thread GitBox
hadoop-yetus removed a comment on pull request #2675: URL: https://github.com/apache/hadoop/pull/2675#issuecomment-802429718 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [hadoop] goiri commented on a change in pull request #2787: HDFS-15900. RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode.

2021-03-23 Thread GitBox
goiri commented on a change in pull request #2787: URL: https://github.com/apache/hadoop/pull/2787#discussion_r599702122 ## File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/store/TestStateStoreMembershipState.java ## @@

[GitHub] [hadoop] hadoop-yetus commented on pull request #2805: HDFS-15913. Remove useless NULL checks before instanceof

2021-03-23 Thread GitBox
hadoop-yetus commented on pull request #2805: URL: https://github.com/apache/hadoop/pull/2805#issuecomment-805017373 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] goiri merged pull request #2805: HDFS-15913. Remove useless NULL checks before instanceof

2021-03-23 Thread GitBox
goiri merged pull request #2805: URL: https://github.com/apache/hadoop/pull/2805 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[jira] [Work logged] (HADOOP-17138) Fix spotbugs warnings surfaced after upgrade to 4.0.6

2021-03-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17138?focusedWorklogId=570573=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570573 ] ASF GitHub Bot logged work on HADOOP-17138: --- Author: ASF GitHub Bot

[GitHub] [hadoop] goiri commented on a change in pull request #2806: HADOOP-17138. Fix spotbugs warnings surfaced after upgrade to 4.0.6.

2021-03-23 Thread GitBox
goiri commented on a change in pull request #2806: URL: https://github.com/apache/hadoop/pull/2806#discussion_r599698654 ## File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java ## @@ -3716,7 +3716,7 @@ void incrUserConnections(String

[jira] [Commented] (HADOOP-17599) Remove NULL checks before instanceof

2021-03-23 Thread Jira
[ https://issues.apache.org/jira/browse/HADOOP-17599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17307180#comment-17307180 ] Íñigo Goiri commented on HADOOP-17599: -- Thanks [~Jack01] for the contribution. Merged the PR. >

[jira] [Assigned] (HADOOP-17599) Remove NULL checks before instanceof

2021-03-23 Thread Jira
[ https://issues.apache.org/jira/browse/HADOOP-17599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri reassigned HADOOP-17599: Assignee: Jiajun Jiang > Remove NULL checks before instanceof >

[jira] [Resolved] (HADOOP-17599) Remove NULL checks before instanceof

2021-03-23 Thread Jira
[ https://issues.apache.org/jira/browse/HADOOP-17599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri resolved HADOOP-17599. -- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Remove NULL

[jira] [Work logged] (HADOOP-17599) Remove NULL checks before instanceof

2021-03-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17599?focusedWorklogId=570567=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570567 ] ASF GitHub Bot logged work on HADOOP-17599: --- Author: ASF GitHub Bot

[GitHub] [hadoop] goiri merged pull request #2804: HADOOP-17599. Remove NULL checks before instanceof

2021-03-23 Thread GitBox
goiri merged pull request #2804: URL: https://github.com/apache/hadoop/pull/2804 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [hadoop] xiaoyuyao commented on a change in pull request #2784: HDFS-15850. Superuser actions should be reported to external enforcers

2021-03-23 Thread GitBox
xiaoyuyao commented on a change in pull request #2784: URL: https://github.com/apache/hadoop/pull/2784#discussion_r599677401 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeAttributeProvider.java ## @@ -405,6 +405,30 @@

[GitHub] [hadoop] xiaoyuyao commented on a change in pull request #2784: HDFS-15850. Superuser actions should be reported to external enforcers

2021-03-23 Thread GitBox
xiaoyuyao commented on a change in pull request #2784: URL: https://github.com/apache/hadoop/pull/2784#discussion_r599661862 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java ## @@ -7664,8 +7669,9 @@ void

[jira] [Work logged] (HADOOP-17511) Add an Audit plugin point for S3A auditing/context

2021-03-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17511?focusedWorklogId=570494=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570494 ] ASF GitHub Bot logged work on HADOOP-17511: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #2675: HADOOP-17511. Add audit/telemetry logging to S3A connector

2021-03-23 Thread GitBox
hadoop-yetus commented on pull request #2675: URL: https://github.com/apache/hadoop/pull/2675#issuecomment-804944797 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] hadoop-yetus commented on pull request #2794: HDFS-15911 : Provide blocks moved count in Balancer iteration result

2021-03-23 Thread GitBox
hadoop-yetus commented on pull request #2794: URL: https://github.com/apache/hadoop/pull/2794#issuecomment-804939036 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Work logged] (HADOOP-17476) ITestAssumeRole.testAssumeRoleBadInnerAuth failure

2021-03-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17476?focusedWorklogId=570457=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570457 ] ASF GitHub Bot logged work on HADOOP-17476: --- Author: ASF GitHub Bot

[GitHub] [hadoop] mukund-thakur commented on a change in pull request #2777: HADOOP-17476. ITestAssumeRole.testAssumeRoleBadInnerAuth failure.

2021-03-23 Thread GitBox
mukund-thakur commented on a change in pull request #2777: URL: https://github.com/apache/hadoop/pull/2777#discussion_r599582337 ## File path: hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/auth/ITestAssumeRole.java ## @@ -255,8 +255,7 @@ public void

[jira] [Work logged] (HADOOP-17476) ITestAssumeRole.testAssumeRoleBadInnerAuth failure

2021-03-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17476?focusedWorklogId=570447=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570447 ] ASF GitHub Bot logged work on HADOOP-17476: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on pull request #2777: HADOOP-17476. ITestAssumeRole.testAssumeRoleBadInnerAuth failure.

2021-03-23 Thread GitBox
steveloughran commented on pull request #2777: URL: https://github.com/apache/hadoop/pull/2777#issuecomment-804912248 @mukund-thakur any further comments? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hadoop] touchida commented on a change in pull request #2585: HDFS-15759. EC: Verify EC reconstruction correctness on DataNode

2021-03-23 Thread GitBox
touchida commented on a change in pull request #2585: URL: https://github.com/apache/hadoop/pull/2585#discussion_r599574539 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReconstructStripedFileWithValidator.java ## @@ -0,0 +1,115 @@

[GitHub] [hadoop] touchida commented on a change in pull request #2585: HDFS-15759. EC: Verify EC reconstruction correctness on DataNode

2021-03-23 Thread GitBox
touchida commented on a change in pull request #2585: URL: https://github.com/apache/hadoop/pull/2585#discussion_r599569744 ## File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/DecodingValidator.java ## @@ -0,0 +1,187 @@

[jira] [Work logged] (HADOOP-13551) Collect AwsSdkMetrics in S3A FileSystem IOStatistics

2021-03-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13551?focusedWorklogId=570412=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570412 ] ASF GitHub Bot logged work on HADOOP-13551: --- Author: ASF GitHub Bot

[GitHub] [hadoop] mukund-thakur commented on a change in pull request #2778: HADOOP-13551. AWS metrics wire-up

2021-03-23 Thread GitBox
mukund-thakur commented on a change in pull request #2778: URL: https://github.com/apache/hadoop/pull/2778#discussion_r599415813 ## File path: hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/DefaultS3ClientFactory.java ## @@ -226,31 +170,6 @@ protected static

[jira] [Work logged] (HADOOP-17599) Remove NULL checks before instanceof

2021-03-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17599?focusedWorklogId=570390=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570390 ] ASF GitHub Bot logged work on HADOOP-17599: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #2804: HADOOP-17599. Remove NULL checks before instanceof

2021-03-23 Thread GitBox
hadoop-yetus commented on pull request #2804: URL: https://github.com/apache/hadoop/pull/2804#issuecomment-804859776 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Commented] (HADOOP-16828) Zookeeper Delegation Token Manager fetch sequence number by batch

2021-03-23 Thread Stephen O'Donnell (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17306995#comment-17306995 ] Stephen O'Donnell commented on HADOOP-16828: [~fengnanli] These seems like a great change

[jira] [Work logged] (HADOOP-17599) Remove NULL checks before instanceof

2021-03-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17599?focusedWorklogId=570381=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570381 ] ASF GitHub Bot logged work on HADOOP-17599: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #2804: HADOOP-17599. Remove NULL checks before instanceof

2021-03-23 Thread GitBox
hadoop-yetus commented on pull request #2804: URL: https://github.com/apache/hadoop/pull/2804#issuecomment-804845800 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] ayushtkn commented on pull request #2790: HDFS-15908. Possible Resource Leak in org.apache.hadoop.hdfs.qjournal.server.Journal

2021-03-23 Thread GitBox
ayushtkn commented on pull request #2790: URL: https://github.com/apache/hadoop/pull/2790#issuecomment-804839355 Thanx @Nargeshdb for the contribution and @virajjasani for the review!!! -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [hadoop] ayushtkn merged pull request #2790: HDFS-15908. Possible Resource Leak in org.apache.hadoop.hdfs.qjournal.server.Journal

2021-03-23 Thread GitBox
ayushtkn merged pull request #2790: URL: https://github.com/apache/hadoop/pull/2790 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[jira] [Work logged] (HADOOP-17138) Fix spotbugs warnings surfaced after upgrade to 4.0.6

2021-03-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17138?focusedWorklogId=570347=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570347 ] ASF GitHub Bot logged work on HADOOP-17138: --- Author: ASF GitHub Bot

[GitHub] [hadoop] tasanuma commented on pull request #2806: HADOOP-17138. Fix spotbugs warnings surfaced after upgrade to 4.0.6.

2021-03-23 Thread GitBox
tasanuma commented on pull request #2806: URL: https://github.com/apache/hadoop/pull/2806#issuecomment-804803600 There are some spotbugs warnings in #2800. As branch-3.2 uses spotbugs after HADOOP-16870, we need to backport HADOOP-17138 to branch-3.2. -- This is an automated

[jira] [Work logged] (HADOOP-17138) Fix spotbugs warnings surfaced after upgrade to 4.0.6

2021-03-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17138?focusedWorklogId=570346=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570346 ] ASF GitHub Bot logged work on HADOOP-17138: --- Author: ASF GitHub Bot

[GitHub] [hadoop] tasanuma opened a new pull request #2806: HADOOP-17138. Fix spotbugs warnings surfaced after upgrade to 4.0.6.

2021-03-23 Thread GitBox
tasanuma opened a new pull request #2806: URL: https://github.com/apache/hadoop/pull/2806 …(#2155) (cherry picked from commit 1b29c9bfeee0035dd042357038b963843169d44c) ## NOTICE Please create an issue in ASF JIRA before opening a pull request, and you need to set the

[GitHub] [hadoop] hadoop-yetus commented on pull request #2796: HDFS-15911 : Provide blocks moved count in Balancer iteration result

2021-03-23 Thread GitBox
hadoop-yetus commented on pull request #2796: URL: https://github.com/apache/hadoop/pull/2796#issuecomment-804798602 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] tasanuma commented on pull request #2800: HDFS-15906. Close FSImage and FSNamesystem after formatting is complete for branch-3.2

2021-03-23 Thread GitBox
tasanuma commented on pull request #2800: URL: https://github.com/apache/hadoop/pull/2800#issuecomment-804764410 Merged to branch-3.2 and cherry-picked to branch-3.1. Thanks for your contribution, @tomscut! -- This is an automated message from the Apache Git Service. To respond to the

  1   2   >