[GitHub] [hadoop] hadoop-yetus commented on pull request #3842: HDFS-16403. Improve FUSE IO performance by supporting FUSE parameter max_background

2022-01-24 Thread GitBox
hadoop-yetus commented on pull request #3842: URL: https://github.com/apache/hadoop/pull/3842#issuecomment-1020904562 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[GitHub] [hadoop] hadoop-yetus commented on pull request #3842: HDFS-16403. Improve FUSE IO performance by supporting FUSE parameter max_background

2022-01-24 Thread GitBox
hadoop-yetus commented on pull request #3842: URL: https://github.com/apache/hadoop/pull/3842#issuecomment-1020903689 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :

[GitHub] [hadoop] Hexiaoqiao merged pull request #3898: HDFS-16428. Source path with storagePolicy cause wrong typeConsumed while rename

2022-01-24 Thread GitBox
Hexiaoqiao merged pull request #3898: URL: https://github.com/apache/hadoop/pull/3898 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issue

[jira] [Commented] (HADOOP-18090) Exclude com/jcraft/jsch classes from being shaded/relocated

2022-01-24 Thread mkv (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17481598#comment-17481598 ] mkv commented on HADOOP-18090: -- [~ste...@apache.org] [~busbey] Can this be accepted as a b

[GitHub] [hadoop] liubingxing edited a comment on pull request #3907: HDFS-16432. Namenode block report add yield to avoid holding write lo…

2022-01-24 Thread GitBox
liubingxing edited a comment on pull request #3907: URL: https://github.com/apache/hadoop/pull/3907#issuecomment-1019266612 @tasanuma @Hexiaoqiao Please take a look. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[jira] [Resolved] (HADOOP-18076) S3Object must be closed/ref set to null in S3AInputStream.closeStream()

2022-01-24 Thread Mukund Thakur (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukund Thakur resolved HADOOP-18076. Assignee: Mukund Thakur Resolution: Invalid After looking at the s3object one close

[jira] [Updated] (HADOOP-16555) Update commons-compress to 1.19

2022-01-24 Thread Akira Ajisaka (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-16555: --- Fix Version/s: 3.2.2 (was: 3.2.1) > Update commons-compress to 1.19

[GitHub] [hadoop] jojochuang merged pull request #3842: HDFS-16403. Improve FUSE IO performance by supporting FUSE parameter max_background

2022-01-24 Thread GitBox
jojochuang merged pull request #3842: URL: https://github.com/apache/hadoop/pull/3842 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issue

[GitHub] [hadoop] cndaimin commented on pull request #3842: HDFS-16403. Improve FUSE IO performance by supporting FUSE parameter max_background

2022-01-24 Thread GitBox
cndaimin commented on pull request #3842: URL: https://github.com/apache/hadoop/pull/3842#issuecomment-1020798721 The indentation difference seems to be gap between tab and space, I just fixed it. Thanks a lot. @jojochuang -- This is an automated message from the Apache Git Service. To

[GitHub] [hadoop] jojochuang opened a new pull request #3924: HDFS-16423. Balancer should not get blocks on stale storages (#3883)

2022-01-24 Thread GitBox
jojochuang opened a new pull request #3924: URL: https://github.com/apache/hadoop/pull/3924 Reviewed-by: litao Signed-off-by: Takanobu Asanuma (cherry picked from commit db2c3200e63a4377331643eddedcab79d4a51468) Conflicts: hadoop-hdfs-project/hadoop-hdfs/src/test/jav

[GitHub] [hadoop] tasanuma commented on a change in pull request #3831: HDFS-16398. Reconfig block report parameters for datanode

2022-01-24 Thread GitBox
tasanuma commented on a change in pull request #3831: URL: https://github.com/apache/hadoop/pull/3831#discussion_r791320927 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DNConf.java ## @@ -311,6 +299,19 @@ public DNConf(fina

[GitHub] [hadoop] tasanuma commented on pull request #3831: HDFS-16398. Reconfig block report parameters for datanode

2022-01-24 Thread GitBox
tasanuma commented on pull request #3831: URL: https://github.com/apache/hadoop/pull/3831#issuecomment-1020761891 @tomscut Thanks for the PR. It mostly looks good to me. - Could you fix the conflict? - `dfs.blockreport.initialDelay` is only used when initializing DataNode. Do we n

[GitHub] [hadoop] cndaimin commented on a change in pull request #3881: HDFS-16422. Fix thread safety of EC decoding during concurrent preads

2022-01-24 Thread GitBox
cndaimin commented on a change in pull request #3881: URL: https://github.com/apache/hadoop/pull/3881#discussion_r791312508 ## File path: hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/StripeReader.java ## @@ -432,7 +432,9 @@ void decodeAndFillBuff

[GitHub] [hadoop] jojochuang edited a comment on pull request #3923: HDFS-16303. Fix edge case where DatanodeAdminDefaultMonitor doubly en…

2022-01-24 Thread GitBox
jojochuang edited a comment on pull request #3923: URL: https://github.com/apache/hadoop/pull/3923#issuecomment-1020721954 @KevinWikant while this is a small patch, please file a new jira to make issue tracking easier. Thank you We'd like to keep one commit per issue. -- This is an au

[GitHub] [hadoop] jojochuang commented on pull request #3923: HDFS-16303. Fix edge case where DatanodeAdminDefaultMonitor doubly en…

2022-01-24 Thread GitBox
jojochuang commented on pull request #3923: URL: https://github.com/apache/hadoop/pull/3923#issuecomment-1020721954 @KevinWikant while this is a small patch, please file a new jira to make issue tracking easier. Thank you -- This is an automated message from the Apache Git Service. To re

[GitHub] [hadoop] jojochuang commented on a change in pull request #3881: HDFS-16422. Fix thread safety of EC decoding during concurrent preads

2022-01-24 Thread GitBox
jojochuang commented on a change in pull request #3881: URL: https://github.com/apache/hadoop/pull/3881#discussion_r791286115 ## File path: hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/StripeReader.java ## @@ -432,7 +432,9 @@ void decodeAndFillBu

[GitHub] [hadoop] jojochuang commented on a change in pull request #3881: HDFS-16422. Fix thread safety of EC decoding during concurrent preads

2022-01-24 Thread GitBox
jojochuang commented on a change in pull request #3881: URL: https://github.com/apache/hadoop/pull/3881#discussion_r791286115 ## File path: hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/StripeReader.java ## @@ -432,7 +432,9 @@ void decodeAndFillBu

[GitHub] [hadoop] hadoop-yetus commented on pull request #3923: HDFS-16303. Fix edge case where DatanodeAdminDefaultMonitor doubly en…

2022-01-24 Thread GitBox
hadoop-yetus commented on pull request #3923: URL: https://github.com/apache/hadoop/pull/3923#issuecomment-1020714294 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[GitHub] [hadoop] hadoop-yetus commented on pull request #3921: HDFS-16303. Improve handling of datanode lost while decommissioning

2022-01-24 Thread GitBox
hadoop-yetus commented on pull request #3921: URL: https://github.com/apache/hadoop/pull/3921#issuecomment-1020704708 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[GitHub] [hadoop] hadoop-yetus commented on pull request #3921: HDFS-16303. Improve handling of datanode lost while decommissioning

2022-01-24 Thread GitBox
hadoop-yetus commented on pull request #3921: URL: https://github.com/apache/hadoop/pull/3921#issuecomment-1020691857 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[GitHub] [hadoop] fapifta commented on pull request #3842: HDFS-16403. Improve FUSE IO performance by supporting FUSE parameter max_background

2022-01-24 Thread GitBox
fapifta commented on pull request #3842: URL: https://github.com/apache/hadoop/pull/3842#issuecomment-1020661356 Hi @cndaimin sorry for the long reply time, I could not get to this one last week, and still I was not able to review the linked study but I am pretty much convinced by the numb

[GitHub] [hadoop] hadoop-yetus commented on pull request #3920: HDFS-16303. Improve handling of datanode lost while decommissioning

2022-01-24 Thread GitBox
hadoop-yetus commented on pull request #3920: URL: https://github.com/apache/hadoop/pull/3920#issuecomment-1020622158 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[GitHub] [hadoop] goiri commented on pull request #3878: HDFS-16411 RBF: RouterId is NULL when disable RourterRpcServer

2022-01-24 Thread GitBox
goiri commented on pull request #3878: URL: https://github.com/apache/hadoop/pull/3878#issuecomment-1020620894 TestRouterDistCpProcedure seems to be failing consistently. @Hexiaoqiao @wojiaodoubao @ayushtkn @xiaoxiaopan118 can you take a look? -- This is an automated message from the A

[jira] [Work logged] (HADOOP-18087) fix bug when getting CNAME chain records and no record from remote dns server

2022-01-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18087?focusedWorklogId=714038&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-714038 ] ASF GitHub Bot logged work on HADOOP-18087: --- Author: ASF GitHub B

[GitHub] [hadoop] eubnara commented on pull request #3911: HADOOP-18087. fix bug when getting CNAME chain records and no record from remote dns server

2022-01-24 Thread GitBox
eubnara commented on pull request #3911: URL: https://github.com/apache/hadoop/pull/3911#issuecomment-1020563876 @steveloughran Why close this issue? YARN Registry DNS is not maintained anymore? -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [hadoop] KevinWikant closed pull request #3922: Feature/hdfs 16303

2022-01-24 Thread GitBox
KevinWikant closed pull request #3922: URL: https://github.com/apache/hadoop/pull/3922 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issu

[GitHub] [hadoop] KevinWikant opened a new pull request #3923: HDFS-16303. Fix edge case where DatanodeAdminDefaultMonitor doubly en…

2022-01-24 Thread GitBox
KevinWikant opened a new pull request #3923: URL: https://github.com/apache/hadoop/pull/3923 ### Description of PR A rare edge case was noticed in DatanodeAdminDefaultMonitor which causes a DatanodeDescriptor to be added twice to the pendingNodes queue. - a [datanode is unhea

[GitHub] [hadoop] hadoop-yetus commented on pull request #3922: Feature/hdfs 16303

2022-01-24 Thread GitBox
hadoop-yetus commented on pull request #3922: URL: https://github.com/apache/hadoop/pull/3922#issuecomment-1020478081 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[GitHub] [hadoop] KevinWikant opened a new pull request #3922: Feature/hdfs 16303

2022-01-24 Thread GitBox
KevinWikant opened a new pull request #3922: URL: https://github.com/apache/hadoop/pull/3922 ### Description of PR A rare edge case was noticed in DatanodeAdminDefaultMonitor which causes a DatanodeDescriptor to be added twice to the pendingNodes queue. - a [datanode is unhea

[GitHub] [hadoop] KevinWikant opened a new pull request #3921: HDFS-16303. Improve handling of datanode lost while decommissioning

2022-01-24 Thread GitBox
KevinWikant opened a new pull request #3921: URL: https://github.com/apache/hadoop/pull/3921 ### Description of PR Backport of this [trunk commit](https://github.com/apache/hadoop/commit/d20b598f97e76c67d6103a950ea9e89644be2c41) to Hadoop-3.x ### How was this patch tested?

[GitHub] [hadoop] hadoop-yetus commented on pull request #3888: HDFS-16427. Add debug log for BlockManager#chooseExcessRedundancyStriped

2022-01-24 Thread GitBox
hadoop-yetus commented on pull request #3888: URL: https://github.com/apache/hadoop/pull/3888#issuecomment-1020457945 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[jira] [Work logged] (HADOOP-18087) fix bug when getting CNAME chain records and no record from remote dns server

2022-01-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18087?focusedWorklogId=713942&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-713942 ] ASF GitHub Bot logged work on HADOOP-18087: --- Author: ASF GitHub B

[jira] [Work logged] (HADOOP-18087) fix bug when getting CNAME chain records and no record from remote dns server

2022-01-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18087?focusedWorklogId=713943&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-713943 ] ASF GitHub Bot logged work on HADOOP-18087: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on pull request #3911: HADOOP-18087. fix bug when getting CNAME chain records and no record from remote dns server

2022-01-24 Thread GitBox
steveloughran commented on pull request #3911: URL: https://github.com/apache/hadoop/pull/3911#issuecomment-1020397892 do we have someone who understands DNS well enough to review this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

[GitHub] [hadoop] steveloughran closed pull request #3911: HADOOP-18087. fix bug when getting CNAME chain records and no record from remote dns server

2022-01-24 Thread GitBox
steveloughran closed pull request #3911: URL: https://github.com/apache/hadoop/pull/3911 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-is

[jira] [Work logged] (HADOOP-18093) Better exception handling for testFileStatusOnMountLink() in ViewFsBaseTest.java

2022-01-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18093?focusedWorklogId=713926&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-713926 ] ASF GitHub Bot logged work on HADOOP-18093: --- Author: ASF GitHub B

[GitHub] [hadoop] xinglin commented on pull request #3918: HADOOP-18093: Better exception handling for testFileStatusOnMountLink

2022-01-24 Thread GitBox
xinglin commented on pull request #3918: URL: https://github.com/apache/hadoop/pull/3918#issuecomment-1020374349 Awesome! thanks @ayushtkn! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[jira] [Work logged] (HADOOP-18093) Better exception handling for testFileStatusOnMountLink() in ViewFsBaseTest.java

2022-01-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18093?focusedWorklogId=713925&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-713925 ] ASF GitHub Bot logged work on HADOOP-18093: --- Author: ASF GitHub B

[GitHub] [hadoop] ayushtkn commented on pull request #3918: HADOOP-18093: Better exception handling for testFileStatusOnMountLink

2022-01-24 Thread GitBox
ayushtkn commented on pull request #3918: URL: https://github.com/apache/hadoop/pull/3918#issuecomment-1020372992 Yeps, we are. TestIPC is flaky and is being tracked seperately. Will merge this by tomorrow if no further comments.. -- This is an automated message from the Apache Git Ser

[GitHub] [hadoop] KevinWikant opened a new pull request #3920: HDFS-16303. Improve handling of datanode lost while decommissioning

2022-01-24 Thread GitBox
KevinWikant opened a new pull request #3920: URL: https://github.com/apache/hadoop/pull/3920 ### Description of PR Backport of this [trunk commit](https://github.com/apache/hadoop/commit/d20b598f97e76c67d6103a950ea9e89644be2c41) to Hadoop-2.x ### How was this patch tested?

[jira] [Work logged] (HADOOP-18093) Better exception handling for testFileStatusOnMountLink() in ViewFsBaseTest.java

2022-01-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18093?focusedWorklogId=713918&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-713918 ] ASF GitHub Bot logged work on HADOOP-18093: --- Author: ASF GitHub B

[GitHub] [hadoop] xinglin commented on pull request #3918: HADOOP-18093: Better exception handling for testFileStatusOnMountLink

2022-01-24 Thread GitBox
xinglin commented on pull request #3918: URL: https://github.com/apache/hadoop/pull/3918#issuecomment-1020355511 this test failure seems to happen on trunk without my change as well. Are we good to merge this pull request? "TestIPC.testIOEOnListenerAccept:652 Expected an EOFException

[jira] [Created] (HADOOP-18095) s3a connector to fully support AWS partitions,

2022-01-24 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18095: --- Summary: s3a connector to fully support AWS partitions, Key: HADOOP-18095 URL: https://issues.apache.org/jira/browse/HADOOP-18095 Project: Hadoop Common

[GitHub] [hadoop] goiri merged pull request #3779: YARN-11015. Decouple queue capacity with ability to run OPPORTUNISTIC container

2022-01-24 Thread GitBox
goiri merged pull request #3779: URL: https://github.com/apache/hadoop/pull/3779 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-uns

[GitHub] [hadoop] goiri commented on a change in pull request #3779: YARN-11015. Decouple queue capacity with ability to run OPPORTUNISTIC container

2022-01-24 Thread GitBox
goiri commented on a change in pull request #3779: URL: https://github.com/apache/hadoop/pull/3779#discussion_r790889048 ## File path: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containerman

[jira] [Resolved] (HADOOP-18094) Disable S3A auditing by default.

2022-01-24 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18094. - Fix Version/s: 3.3.2 Resolution: Fixed > Disable S3A auditing by default. >

[jira] [Work logged] (HADOOP-18091) S3A auditing leaks memory through ThreadLocal references

2022-01-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18091?focusedWorklogId=713737&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-713737 ] ASF GitHub Bot logged work on HADOOP-18091: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran merged pull request #3916: HADOOP-18091. S3A Auditor Memory Leakage

2022-01-24 Thread GitBox
steveloughran merged pull request #3916: URL: https://github.com/apache/hadoop/pull/3916 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-is

[jira] [Work logged] (HADOOP-18091) S3A auditing leaks memory through ThreadLocal references

2022-01-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18091?focusedWorklogId=713735&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-713735 ] ASF GitHub Bot logged work on HADOOP-18091: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #3916: HADOOP-18091. S3A Auditor Memory Leakage

2022-01-24 Thread GitBox
hadoop-yetus removed a comment on pull request #3916: URL: https://github.com/apache/hadoop/pull/3916#issuecomment-1019563401 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:|

[jira] [Commented] (HADOOP-18091) S3A auditing leaks memory through ThreadLocal references

2022-01-24 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17481089#comment-17481089 ] Steve Loughran commented on HADOOP-18091: - First fix is to disable auditing by

[jira] [Updated] (HADOOP-18091) S3A auditing leaks memory through ThreadLocal references

2022-01-24 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18091: Target Version/s: 3.3.3 Priority: Major (was: Blocker) > S3A auditing leaks

[jira] [Created] (HADOOP-18094) Disable S3A auditing by default.

2022-01-24 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18094: --- Summary: Disable S3A auditing by default. Key: HADOOP-18094 URL: https://issues.apache.org/jira/browse/HADOOP-18094 Project: Hadoop Common Issue Type:

[jira] [Work logged] (HADOOP-18091) S3A auditing leaks memory through ThreadLocal references

2022-01-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18091?focusedWorklogId=713720&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-713720 ] ASF GitHub Bot logged work on HADOOP-18091: --- Author: ASF GitHub B

[jira] [Work logged] (HADOOP-18091) S3A auditing leaks memory through ThreadLocal references

2022-01-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18091?focusedWorklogId=713719&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-713719 ] ASF GitHub Bot logged work on HADOOP-18091: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #3916: HADOOP-18091. S3A Auditor Memory Leakage

2022-01-24 Thread GitBox
hadoop-yetus removed a comment on pull request #3916: URL: https://github.com/apache/hadoop/pull/3916#issuecomment-1019582799 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:|

[GitHub] [hadoop] steveloughran commented on pull request #3916: HADOOP-18091. S3A Auditor Memory Leakage

2022-01-24 Thread GitBox
steveloughran commented on pull request #3916: URL: https://github.com/apache/hadoop/pull/3916#issuecomment-1020088470 thx. I'm going to creata a new JIRA for this "add option to disable s3a auditing" and commit this under that, wiith the original one open. -- This is an automated

[jira] [Work logged] (HADOOP-18091) S3A auditing leaks memory through ThreadLocal references

2022-01-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18091?focusedWorklogId=713718&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-713718 ] ASF GitHub Bot logged work on HADOOP-18091: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on a change in pull request #3916: HADOOP-18091. S3A Auditor Memory Leakage

2022-01-24 Thread GitBox
steveloughran commented on a change in pull request #3916: URL: https://github.com/apache/hadoop/pull/3916#discussion_r790729182 ## File path: hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/auditing_architecture.md ## @@ -141,8 +152,19 @@ thread. ### class `Noop

[jira] [Work logged] (HADOOP-18091) S3A auditing leaks memory through ThreadLocal references

2022-01-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18091?focusedWorklogId=713707&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-713707 ] ASF GitHub Bot logged work on HADOOP-18091: --- Author: ASF GitHub B

[GitHub] [hadoop] mukund-thakur commented on a change in pull request #3916: HADOOP-18091. S3A Auditor Memory Leakage

2022-01-24 Thread GitBox
mukund-thakur commented on a change in pull request #3916: URL: https://github.com/apache/hadoop/pull/3916#discussion_r790707192 ## File path: hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/auditing_architecture.md ## @@ -141,8 +152,19 @@ thread. ### class `Noop

[GitHub] [hadoop] hadoop-yetus commented on pull request #3919: YARN-11067. Resource overcommitment due to incorrect resource normali…

2022-01-24 Thread GitBox
hadoop-yetus commented on pull request #3919: URL: https://github.com/apache/hadoop/pull/3919#issuecomment-1020031713 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :

[GitHub] [hadoop] hadoop-yetus commented on pull request #3900: HDFS-16429.Add DataSetLockManager to maintain block pool level lock a…

2022-01-24 Thread GitBox
hadoop-yetus commented on pull request #3900: URL: https://github.com/apache/hadoop/pull/3900#issuecomment-1020030757 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :

[jira] [Updated] (HADOOP-17593) hadoop-huaweicloud and hadoop-cloud-storage to remove log4j as transitive dependency

2022-01-24 Thread Akira Ajisaka (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-17593: --- Fix Version/s: 3.4.0 Resolution: Fixed Status: Resolved (was: Patch Avail

[jira] [Commented] (HADOOP-17593) hadoop-huaweicloud and hadoop-cloud-storage to remove log4j as transitive dependency

2022-01-24 Thread Akira Ajisaka (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17481041#comment-17481041 ] Akira Ajisaka commented on HADOOP-17593: Thank you [~Rigenyi] for your contribu

[jira] [Work logged] (HADOOP-18091) S3A auditing leaks memory through ThreadLocal references

2022-01-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18091?focusedWorklogId=713692&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-713692 ] ASF GitHub Bot logged work on HADOOP-18091: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus commented on pull request #3916: HADOOP-18091. S3A Auditor Memory Leakage

2022-01-24 Thread GitBox
hadoop-yetus commented on pull request #3916: URL: https://github.com/apache/hadoop/pull/3916#issuecomment-1020024652 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :

[jira] [Commented] (HADOOP-17593) hadoop-huaweicloud and hadoop-cloud-storage to remove log4j as transitive dependency

2022-01-24 Thread Akira Ajisaka (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17481030#comment-17481030 ] Akira Ajisaka commented on HADOOP-17593: I'll commit this after fixing tabs. >

[jira] [Work logged] (HADOOP-18091) S3A auditing leaks memory through ThreadLocal references

2022-01-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18091?focusedWorklogId=713681&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-713681 ] ASF GitHub Bot logged work on HADOOP-18091: --- Author: ASF GitHub B

[GitHub] [hadoop] mukund-thakur commented on a change in pull request #3916: HADOOP-18091. S3A Auditor Memory Leakage

2022-01-24 Thread GitBox
mukund-thakur commented on a change in pull request #3916: URL: https://github.com/apache/hadoop/pull/3916#discussion_r790658237 ## File path: hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/auditing_architecture.md ## @@ -141,8 +152,19 @@ thread. ### class `Noop

[jira] [Work logged] (HADOOP-18091) S3A auditing leaks memory through ThreadLocal references

2022-01-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18091?focusedWorklogId=713640&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-713640 ] ASF GitHub Bot logged work on HADOOP-18091: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on a change in pull request #3916: HADOOP-18091. S3A Auditor Memory Leakage

2022-01-24 Thread GitBox
steveloughran commented on a change in pull request #3916: URL: https://github.com/apache/hadoop/pull/3916#discussion_r790605728 ## File path: hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/auditing.md ## @@ -63,27 +75,43 @@ ideally even identifying the process/job

[jira] [Work logged] (HADOOP-18091) S3A auditing leaks memory through ThreadLocal references

2022-01-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18091?focusedWorklogId=713639&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-713639 ] ASF GitHub Bot logged work on HADOOP-18091: --- Author: ASF GitHub B

[GitHub] [hadoop] mehakmeet commented on a change in pull request #3916: HADOOP-18091. S3A Auditor Memory Leakage

2022-01-24 Thread GitBox
mehakmeet commented on a change in pull request #3916: URL: https://github.com/apache/hadoop/pull/3916#discussion_r790601482 ## File path: hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/auditing.md ## @@ -63,27 +75,43 @@ ideally even identifying the process/job gen

[jira] [Work logged] (HADOOP-18093) Better exception handling for testFileStatusOnMountLink() in ViewFsBaseTest.java

2022-01-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18093?focusedWorklogId=713633&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-713633 ] ASF GitHub Bot logged work on HADOOP-18093: --- Author: ASF GitHub B

[jira] [Work logged] (HADOOP-18091) S3A auditing leaks memory through ThreadLocal references

2022-01-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18091?focusedWorklogId=713632&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-713632 ] ASF GitHub Bot logged work on HADOOP-18091: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus commented on pull request #3918: HADOOP-18093: Better exception handling for testFileStatusOnMountLink

2022-01-24 Thread GitBox
hadoop-yetus commented on pull request #3918: URL: https://github.com/apache/hadoop/pull/3918#issuecomment-1019937362 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[GitHub] [hadoop] steveloughran commented on a change in pull request #3916: HADOOP-18091. S3A Auditor Memory Leakage

2022-01-24 Thread GitBox
steveloughran commented on a change in pull request #3916: URL: https://github.com/apache/hadoop/pull/3916#discussion_r790595047 ## File path: hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/auditing.md ## @@ -63,27 +75,43 @@ ideally even identifying the process/job

[GitHub] [hadoop] tomscut commented on pull request #3888: HDFS-16427. Add debug log for BlockManager#chooseExcessRedundancyStriped

2022-01-24 Thread GitBox
tomscut commented on pull request #3888: URL: https://github.com/apache/hadoop/pull/3888#issuecomment-1019934918 At that time, we found that the `candidates` and `replicasToDelete` were inconsistent, so we determined that the `placementPolicy` had a problem and resolved it. -- This is a

[GitHub] [hadoop] tomscut commented on pull request #3888: HDFS-16427. Add debug log for BlockManager#chooseExcessRedundancyStriped

2022-01-24 Thread GitBox
tomscut commented on pull request #3888: URL: https://github.com/apache/hadoop/pull/3888#issuecomment-1019931846 > @tomscut Sorry for being late. I agree with adding more EC logs, but I feel that the log information is too detailed, and some variables duplicate with other variables. Could

[jira] [Work logged] (HADOOP-11867) FS API: Add a high-performance vectored Read to FSDataInputStream API

2022-01-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-11867?focusedWorklogId=713622&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-713622 ] ASF GitHub Bot logged work on HADOOP-11867: --- Author: ASF GitHub B

[GitHub] [hadoop] mukund-thakur commented on a change in pull request #3904: HADOOP-11867. Add a high performance vectored read API to file system.

2022-01-24 Thread GitBox
mukund-thakur commented on a change in pull request #3904: URL: https://github.com/apache/hadoop/pull/3904#discussion_r790581863 ## File path: hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractContractVectoredReadTest.java ## @@ -120,6 +11

[jira] [Work logged] (HADOOP-11867) FS API: Add a high-performance vectored Read to FSDataInputStream API

2022-01-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-11867?focusedWorklogId=713621&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-713621 ] ASF GitHub Bot logged work on HADOOP-11867: --- Author: ASF GitHub B

[GitHub] [hadoop] mukund-thakur commented on a change in pull request #3904: HADOOP-11867. Add a high performance vectored read API to file system.

2022-01-24 Thread GitBox
mukund-thakur commented on a change in pull request #3904: URL: https://github.com/apache/hadoop/pull/3904#discussion_r790579683 ## File path: hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractContractVectoredReadTest.java ## @@ -120,6 +11

[GitHub] [hadoop] tasanuma commented on pull request #3888: HDFS-16427. Add debug log for BlockManager#chooseExcessRedundancyStriped

2022-01-24 Thread GitBox
tasanuma commented on pull request #3888: URL: https://github.com/apache/hadoop/pull/3888#issuecomment-1019917288 @tomscut Sorry for being late. I agree with adding more EC logs, but I feel that the log information is too detailed, and some variables duplicate with other variables. Could y

[jira] [Work logged] (HADOOP-18082) Add debug log when RPC#Reader gets a Call

2022-01-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18082?focusedWorklogId=713615&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-713615 ] ASF GitHub Bot logged work on HADOOP-18082: --- Author: ASF GitHub B

[GitHub] [hadoop] jianghuazhu commented on pull request #3891: HADOOP-18082.Add debug log when RPC#Reader gets a Call.

2022-01-24 Thread GitBox
jianghuazhu commented on pull request #3891: URL: https://github.com/apache/hadoop/pull/3891#issuecomment-1019910934 Thanks @virajjasani . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

[jira] [Work logged] (HADOOP-18091) S3A auditing leaks memory through ThreadLocal references

2022-01-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18091?focusedWorklogId=713607&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-713607 ] ASF GitHub Bot logged work on HADOOP-18091: --- Author: ASF GitHub B

[GitHub] [hadoop] mehakmeet commented on a change in pull request #3916: HADOOP-18091. S3A Auditor Memory Leakage

2022-01-24 Thread GitBox
mehakmeet commented on a change in pull request #3916: URL: https://github.com/apache/hadoop/pull/3916#discussion_r790434928 ## File path: hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/auditing.md ## @@ -92,13 +120,18 @@ The No-op auditor does not perform any logg

[GitHub] [hadoop] 9uapaw opened a new pull request #3919: YARN-11067. Resource overcommitment due to incorrect resource normali…

2022-01-24 Thread GitBox
9uapaw opened a new pull request #3919: URL: https://github.com/apache/hadoop/pull/3919 …sation logical order ### Description of PR ### How was this patch tested? ### For code changes: - [ ] Does the title or this PR starts with the corresponding J

[jira] [Commented] (HADOOP-17593) hadoop-huaweicloud and hadoop-cloud-storage to remove log4j as transitive dependency

2022-01-24 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480888#comment-17480888 ] Hadoop QA commented on HADOOP-17593: | (x) *{color:red}-1 overall{color}* | \\ \\ |