[GitHub] [hadoop] tomscut opened a new pull request #4009: HDFS-16477. [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS

2022-02-22 Thread GitBox
tomscut opened a new pull request #4009: URL: https://github.com/apache/hadoop/pull/4009 JIRA: [HDFS-16477](https://issues.apache.org/jira/browse/HDFS-16477). Currently we have no idea how many paths are waiting to be processed when using the SPS feature. We should add metric `Pendin

[GitHub] [hadoop] hadoop-yetus commented on pull request #3966: [WIP] HADOOP-15980 : Enable TLS in RPC client/server

2022-02-22 Thread GitBox
hadoop-yetus commented on pull request #3966: URL: https://github.com/apache/hadoop/pull/3966#issuecomment-1047582157 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[jira] [Work logged] (HADOOP-15980) Enable TLS in RPC client/server

2022-02-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-15980?focusedWorklogId=730709&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-730709 ] ASF GitHub Bot logged work on HADOOP-15980: --- Author: ASF GitHub B

[GitHub] [hadoop] jojochuang commented on a change in pull request #3941: HDFS-15382. Split one FsDatasetImpl lock to block pool grain locks.

2022-02-22 Thread GitBox
jojochuang commented on a change in pull request #3941: URL: https://github.com/apache/hadoop/pull/3941#discussion_r811729617 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java ## @@ -3230,7 +3235,9 @@ void transfer

[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #3964: HADOOP-18104: S3A: Add configs to configure minSeekForVectorReads and maxReadSizeForVectorReads

2022-02-22 Thread GitBox
hadoop-yetus removed a comment on pull request #3964: URL: https://github.com/apache/hadoop/pull/3964#issuecomment-1032554919 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:|

[jira] [Work logged] (HADOOP-18104) Add configs to configure minSeekForVectorReads and maxReadSizeForVectorReads

2022-02-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18104?focusedWorklogId=730746&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-730746 ] ASF GitHub Bot logged work on HADOOP-18104: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus commented on pull request #4000: HADOOP-18131. Upgrade maven enforcer plugin and relevant dependencies

2022-02-22 Thread GitBox
hadoop-yetus commented on pull request #4000: URL: https://github.com/apache/hadoop/pull/4000#issuecomment-1047629486 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[jira] [Work logged] (HADOOP-18131) Upgrade maven enforcer plugin and relevant dependencies

2022-02-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18131?focusedWorklogId=730753&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-730753 ] ASF GitHub Bot logged work on HADOOP-18131: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #3964: HADOOP-18104: S3A: Add configs to configure minSeekForVectorReads and maxReadSizeForVectorReads

2022-02-22 Thread GitBox
hadoop-yetus removed a comment on pull request #3964: URL: https://github.com/apache/hadoop/pull/3964#issuecomment-1033592807 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:|

[jira] [Work logged] (HADOOP-18104) Add configs to configure minSeekForVectorReads and maxReadSizeForVectorReads

2022-02-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18104?focusedWorklogId=730754&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-730754 ] ASF GitHub Bot logged work on HADOOP-18104: --- Author: ASF GitHub B

[GitHub] [hadoop] virajjasani commented on pull request #4000: HADOOP-18131. Upgrade maven enforcer plugin and relevant dependencies

2022-02-22 Thread GitBox
virajjasani commented on pull request #4000: URL: https://github.com/apache/hadoop/pull/4000#issuecomment-1047655837 We have full QA results from Jenkins for this PR. @aajisaka @jojochuang @tasanuma could you please take a look? -- This is an automated message from the Apache Git Servi

[jira] [Work logged] (HADOOP-18131) Upgrade maven enforcer plugin and relevant dependencies

2022-02-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18131?focusedWorklogId=730762&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-730762 ] ASF GitHub Bot logged work on HADOOP-18131: --- Author: ASF GitHub B

[jira] [Work logged] (HADOOP-13294) Test hadoop fs shell against s3a; fix problems

2022-02-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13294?focusedWorklogId=730770&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-730770 ] ASF GitHub Bot logged work on HADOOP-13294: --- Author: ASF GitHub B

[GitHub] [hadoop] monthonk commented on a change in pull request #4006: HADOOP-13294. Test hadoop fs shell against s3a

2022-02-22 Thread GitBox
monthonk commented on a change in pull request #4006: URL: https://github.com/apache/hadoop/pull/4006#discussion_r811810465 ## File path: hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AFsShell.java ## @@ -0,0 +1,590 @@ +/* + * Licensed to the Apache Sof

[GitHub] [hadoop] monthonk commented on a change in pull request #4006: HADOOP-13294. Test hadoop fs shell against s3a

2022-02-22 Thread GitBox
monthonk commented on a change in pull request #4006: URL: https://github.com/apache/hadoop/pull/4006#discussion_r811812364 ## File path: hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AFsShell.java ## @@ -0,0 +1,590 @@ +/* + * Licensed to the Apache Sof

[jira] [Work logged] (HADOOP-13294) Test hadoop fs shell against s3a; fix problems

2022-02-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13294?focusedWorklogId=730772&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-730772 ] ASF GitHub Bot logged work on HADOOP-13294: --- Author: ASF GitHub B

[GitHub] [hadoop] virajjasani edited a comment on pull request #4000: HADOOP-18131. Upgrade maven enforcer plugin and relevant dependencies

2022-02-22 Thread GitBox
virajjasani edited a comment on pull request #4000: URL: https://github.com/apache/hadoop/pull/4000#issuecomment-1047655837 We have full QA results from Jenkins for this PR. @aajisaka @jojochuang @tasanuma could you please take a look? (ASF license warning is not relevant) -- This is

[jira] [Work logged] (HADOOP-18131) Upgrade maven enforcer plugin and relevant dependencies

2022-02-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18131?focusedWorklogId=730777&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-730777 ] ASF GitHub Bot logged work on HADOOP-18131: --- Author: ASF GitHub B

[GitHub] [hadoop] szilard-nemeth commented on pull request #3869: YARN-10944. AbstractCSQueue: Eliminate code duplication in overloaded…

2022-02-22 Thread GitBox
szilard-nemeth commented on pull request #3869: URL: https://github.com/apache/hadoop/pull/3869#issuecomment-1047673176 Thanks @9uapaw for working on this. Latest patch LGTM, committed to trunk. -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [hadoop] szilard-nemeth closed pull request #3869: YARN-10944. AbstractCSQueue: Eliminate code duplication in overloaded…

2022-02-22 Thread GitBox
szilard-nemeth closed pull request #3869: URL: https://github.com/apache/hadoop/pull/3869 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-i

[GitHub] [hadoop] szilard-nemeth commented on pull request #3868: YARN-10590. Fix legacy auto queue creation absolute resource calculat…

2022-02-22 Thread GitBox
szilard-nemeth commented on pull request #3868: URL: https://github.com/apache/hadoop/pull/3868#issuecomment-1047699788 Thanks @9uapaw for working on this. This is a very nice fix. Thanks @brumi1024 for the review. -- This is an automated message from the Apache Git Service. To respond

[GitHub] [hadoop] szilard-nemeth closed pull request #3868: YARN-10590. Fix legacy auto queue creation absolute resource calculat…

2022-02-22 Thread GitBox
szilard-nemeth closed pull request #3868: URL: https://github.com/apache/hadoop/pull/3868 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-i

[GitHub] [hadoop] szilard-nemeth commented on pull request #3858: YARN-10918. Simplify method: CapacitySchedulerQueueManager#parseQueue

2022-02-22 Thread GitBox
szilard-nemeth commented on pull request #3858: URL: https://github.com/apache/hadoop/pull/3858#issuecomment-1047700406 Thanks @9uapaw for working on this. Latest patch LGTM, waiting for Jenkins. -- This is an automated message from the Apache Git Service. To respond to the message, pl

[GitHub] [hadoop] ferhui commented on pull request #3998: [SPS]: Fix bug for unit test of reconfiguring SPS mode

2022-02-22 Thread GitBox
ferhui commented on pull request #3998: URL: https://github.com/apache/hadoop/pull/3998#issuecomment-1047705494 @tomscut BTW, it is better to add the issue num within commit message next time.:) I will add the issue num when merge this PR. -- This is an automated message from the Apac

[jira] [Commented] (HADOOP-14837) Handle S3A "glacier" data

2022-02-22 Thread Ahmar Suhail (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-14837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17496047#comment-17496047 ] Ahmar Suhail commented on HADOOP-14837: --- [~ste...@apache.org] I've been looking a

[GitHub] [hadoop] MingXiangLi commented on a change in pull request #3941: HDFS-15382. Split one FsDatasetImpl lock to block pool grain locks.

2022-02-22 Thread GitBox
MingXiangLi commented on a change in pull request #3941: URL: https://github.com/apache/hadoop/pull/3941#discussion_r811871088 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java ## @@ -3230,7 +3235,9 @@ void transfe

[GitHub] [hadoop] MingXiangLi commented on a change in pull request #3941: HDFS-15382. Split one FsDatasetImpl lock to block pool grain locks.

2022-02-22 Thread GitBox
MingXiangLi commented on a change in pull request #3941: URL: https://github.com/apache/hadoop/pull/3941#discussion_r811872874 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/ReplicaMap.java ## @@ -117,7 +121,7

[GitHub] [hadoop] tomscut commented on a change in pull request #3941: HDFS-15382. Split one FsDatasetImpl lock to block pool grain locks.

2022-02-22 Thread GitBox
tomscut commented on a change in pull request #3941: URL: https://github.com/apache/hadoop/pull/3941#discussion_r811864144 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java ## @@ -1850,7 +1828,8

[GitHub] [hadoop] hadoop-yetus commented on pull request #3785: YARN-11042. Fix testQueueSubmitWithACLsEnabledWithQueueMapping in Tes…

2022-02-22 Thread GitBox
hadoop-yetus commented on pull request #3785: URL: https://github.com/apache/hadoop/pull/3785#issuecomment-1047727497 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[GitHub] [hadoop] MingXiangLi commented on a change in pull request #3941: HDFS-15382. Split one FsDatasetImpl lock to block pool grain locks.

2022-02-22 Thread GitBox
MingXiangLi commented on a change in pull request #3941: URL: https://github.com/apache/hadoop/pull/3941#discussion_r811873922 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DiskBalancer.java ## @@ -504,15 +503,13 @@ private

[GitHub] [hadoop] MingXiangLi commented on a change in pull request #3941: HDFS-15382. Split one FsDatasetImpl lock to block pool grain locks.

2022-02-22 Thread GitBox
MingXiangLi commented on a change in pull request #3941: URL: https://github.com/apache/hadoop/pull/3941#discussion_r811874920 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetImpl.java ## @@ -232,11

[GitHub] [hadoop] tomscut commented on pull request #3998: HDFS-16458. [SPS]: Fix bug for unit test of reconfiguring SPS mode

2022-02-22 Thread GitBox
tomscut commented on pull request #3998: URL: https://github.com/apache/hadoop/pull/3998#issuecomment-1047729381 > @tomscut BTW, it is better to add the issue num within commit message next time.:) I will add the issue num when merge this PR. My bad. Thanks @ferhui for your review. I

[GitHub] [hadoop] MingXiangLi commented on a change in pull request #3941: HDFS-15382. Split one FsDatasetImpl lock to block pool grain locks.

2022-02-22 Thread GitBox
MingXiangLi commented on a change in pull request #3941: URL: https://github.com/apache/hadoop/pull/3941#discussion_r811881764 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/ReplicaMap.java ## @@ -184,7 +203,7

[GitHub] [hadoop] MingXiangLi commented on a change in pull request #3941: HDFS-15382. Split one FsDatasetImpl lock to block pool grain locks.

2022-02-22 Thread GitBox
MingXiangLi commented on a change in pull request #3941: URL: https://github.com/apache/hadoop/pull/3941#discussion_r811881764 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/ReplicaMap.java ## @@ -184,7 +203,7

[GitHub] [hadoop] hadoop-yetus commented on pull request #4006: HADOOP-13294. Test hadoop fs shell against s3a

2022-02-22 Thread GitBox
hadoop-yetus commented on pull request #4006: URL: https://github.com/apache/hadoop/pull/4006#issuecomment-1047741683 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :

[jira] [Work logged] (HADOOP-13294) Test hadoop fs shell against s3a; fix problems

2022-02-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13294?focusedWorklogId=730853&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-730853 ] ASF GitHub Bot logged work on HADOOP-13294: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus commented on pull request #4008: HDFS-16475. Cleanup code in the write path

2022-02-22 Thread GitBox
hadoop-yetus commented on pull request #4008: URL: https://github.com/apache/hadoop/pull/4008#issuecomment-1047779034 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[GitHub] [hadoop] bbeaudreault commented on pull request #3271: HDFS-16155: Allow configurable exponential backoff in DFSInputStream refetchLocations

2022-02-22 Thread GitBox
bbeaudreault commented on pull request #3271: URL: https://github.com/apache/hadoop/pull/3271#issuecomment-1047810182 @Hexiaoqiao can this be merged? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [hadoop] jianghuazhu opened a new pull request #4010: HDFS-16476.Increase the number of metrics used to record PendingRecoveryBlocks.

2022-02-22 Thread GitBox
jianghuazhu opened a new pull request #4010: URL: https://github.com/apache/hadoop/pull/4010 ### Description of PR Now we don't know how many blocks are happening or are about to recover, the purpose of this pr is to record them through metrics. Details: HDFS-16476 ### How was

[GitHub] [hadoop] hadoop-yetus commented on pull request #3964: HADOOP-18104: S3A: Add configs to configure minSeekForVectorReads and maxReadSizeForVectorReads

2022-02-22 Thread GitBox
hadoop-yetus commented on pull request #3964: URL: https://github.com/apache/hadoop/pull/3964#issuecomment-1047822929 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[jira] [Work logged] (HADOOP-18104) Add configs to configure minSeekForVectorReads and maxReadSizeForVectorReads

2022-02-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18104?focusedWorklogId=730924&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-730924 ] ASF GitHub Bot logged work on HADOOP-18104: --- Author: ASF GitHub B

[GitHub] [hadoop] 9uapaw closed pull request #3874: YARN-11017. Unify node label access in queues

2022-02-22 Thread GitBox
9uapaw closed pull request #3874: URL: https://github.com/apache/hadoop/pull/3874 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-un

[GitHub] [hadoop] hadoop-yetus commented on pull request #3938: YARN-11069. Dynamic Queue ACL handling in Legacy and Flexible Auto Created Queues

2022-02-22 Thread GitBox
hadoop-yetus commented on pull request #3938: URL: https://github.com/apache/hadoop/pull/3938#issuecomment-1047844590 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[GitHub] [hadoop] busbey commented on pull request #4007: HADOOP-18138. UserGroupInformation shouldn't log exception unnecessarily

2022-02-22 Thread GitBox
busbey commented on pull request #4007: URL: https://github.com/apache/hadoop/pull/4007#issuecomment-1047924692 I'm wary of things that require us to reimplement the logging system handling the stack trace details. Maybe we could convey the purpose in either the exception message or the ex

[GitHub] [hadoop] busbey edited a comment on pull request #4007: HADOOP-18138. UserGroupInformation shouldn't log exception unnecessarily

2022-02-22 Thread GitBox
busbey edited a comment on pull request #4007: URL: https://github.com/apache/hadoop/pull/4007#issuecomment-1047924692 I'm wary of things that require us to reimplement the logging system handling the stack trace details. Maybe we could convey the purpose in either the exception message or

[jira] [Work logged] (HADOOP-18138) UserGroupInformation shouldn't log exception

2022-02-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18138?focusedWorklogId=731028&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-731028 ] ASF GitHub Bot logged work on HADOOP-18138: --- Author: ASF GitHub B

[jira] [Work logged] (HADOOP-18138) UserGroupInformation shouldn't log exception

2022-02-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18138?focusedWorklogId=731029&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-731029 ] ASF GitHub Bot logged work on HADOOP-18138: --- Author: ASF GitHub B

[GitHub] [hadoop] 9uapaw opened a new pull request #4011: YARN-10945. Add javadoc to all methods of AbstractCSQueue

2022-02-22 Thread GitBox
9uapaw opened a new pull request #4011: URL: https://github.com/apache/hadoop/pull/4011 ### Description of PR ### How was this patch tested? ### For code changes: - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-1

[GitHub] [hadoop] brumi1024 commented on a change in pull request #3858: YARN-10918. Simplify method: CapacitySchedulerQueueManager#parseQueue

2022-02-22 Thread GitBox
brumi1024 commented on a change in pull request #3858: URL: https://github.com/apache/hadoop/pull/3858#discussion_r812097838 ## File path: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/

[GitHub] [hadoop] hadoop-yetus commented on pull request #4009: HDFS-16477. [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS

2022-02-22 Thread GitBox
hadoop-yetus commented on pull request #4009: URL: https://github.com/apache/hadoop/pull/4009#issuecomment-1047953347 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[GitHub] [hadoop] brumi1024 opened a new pull request #4012: YARN-11075. Explicitly declare serialVersionUID in LogMutation class.…

2022-02-22 Thread GitBox
brumi1024 opened a new pull request #4012: URL: https://github.com/apache/hadoop/pull/4012 … Contributed by Benjamin Teke ### Description of PR ### How was this patch tested? ### For code changes: - [ ] Does the title or this PR starts with the cor

[GitHub] [hadoop] brumi1024 opened a new pull request #4013: YARN-11075. Explicitly declare serialVersionUID in LogMutation class.…

2022-02-22 Thread GitBox
brumi1024 opened a new pull request #4013: URL: https://github.com/apache/hadoop/pull/4013 … Contributed by Benjamin Teke ### Description of PR ### How was this patch tested? ### For code changes: - [ ] Does the title or this PR starts with the cor

[GitHub] [hadoop] brumi1024 opened a new pull request #4014: YARN-11075. Explicitly declare serialVersionUID in LogMutation class.…

2022-02-22 Thread GitBox
brumi1024 opened a new pull request #4014: URL: https://github.com/apache/hadoop/pull/4014 … Contributed by Benjamin Teke ### Description of PR ### How was this patch tested? ### For code changes: - [ ] Does the title or this PR starts with the cor

[GitHub] [hadoop] shvachko opened a new pull request #4015: HADOOP-18127. Backport HADOOP-13055 into branch-2.10

2022-02-22 Thread GitBox
shvachko opened a new pull request #4015: URL: https://github.com/apache/hadoop/pull/4015 ### Description of PR Back porting HADOOP-13055 needed two more commits. So this PR includes three commits: - HADOOP-13722. Code cleanup -- ViewFileSystem and InodeTree. - HADOOP-12077. P

[jira] [Updated] (HADOOP-18127) Backport HADOOP-13055 into branch-2.10

2022-02-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-18127: Labels: pull-request-available (was: ) > Backport HADOOP-13055 into branch-2.10 > --

[jira] [Work logged] (HADOOP-18127) Backport HADOOP-13055 into branch-2.10

2022-02-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18127?focusedWorklogId=731103&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-731103 ] ASF GitHub Bot logged work on HADOOP-18127: --- Author: ASF GitHub B

[jira] [Commented] (HADOOP-18127) Backport HADOOP-13055 into branch-2.10

2022-02-22 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17496262#comment-17496262 ] Konstantin Shvachko commented on HADOOP-18127: -- Back porting HADOOP-13055

[GitHub] [hadoop] hadoop-yetus commented on pull request #3988: [HADOOP-15983] use jersey-json that is built to use jackson2

2022-02-22 Thread GitBox
hadoop-yetus commented on pull request #3988: URL: https://github.com/apache/hadoop/pull/3988#issuecomment-1048123764 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[jira] [Work logged] (HADOOP-15983) Remove the usage of jersey-json to remove jackson 1.x dependency.

2022-02-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-15983?focusedWorklogId=731123&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-731123 ] ASF GitHub Bot logged work on HADOOP-15983: --- Author: ASF GitHub B

[GitHub] [hadoop] madrob opened a new pull request #4016: HDFS-16478 Add thread id to command processor name

2022-02-22 Thread GitBox
madrob opened a new pull request #4016: URL: https://github.com/apache/hadoop/pull/4016 ### Description of PR Provides additional clarity in logs when debugging MiniDFSCluster with multiple in-process DataNodes. ### How was this patch tested? Visual inspection of log

[GitHub] [hadoop] hadoop-yetus commented on pull request #3804: YARN-11049. MutableConfScheduler is referred as plain String instead of class name

2022-02-22 Thread GitBox
hadoop-yetus commented on pull request #3804: URL: https://github.com/apache/hadoop/pull/3804#issuecomment-1048154759 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[jira] [Updated] (HADOOP-8299) ViewFileSystem link slash mount point crashes with IndexOutOfBoundsException

2022-02-22 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-8299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HADOOP-8299: Fix Version/s: 2.10.1 > ViewFileSystem link slash mount point crashes with IndexOu

[GitHub] [hadoop] hadoop-yetus commented on pull request #4011: YARN-10945. Add javadoc to all methods of AbstractCSQueue

2022-02-22 Thread GitBox
hadoop-yetus commented on pull request #4011: URL: https://github.com/apache/hadoop/pull/4011#issuecomment-1048168103 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[GitHub] [hadoop] hadoop-yetus commented on pull request #4014: YARN-11075. Explicitly declare serialVersionUID in LogMutation class.…

2022-02-22 Thread GitBox
hadoop-yetus commented on pull request #4014: URL: https://github.com/apache/hadoop/pull/4014#issuecomment-1048182155 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[GitHub] [hadoop] hadoop-yetus commented on pull request #4013: YARN-11075. Explicitly declare serialVersionUID in LogMutation class.…

2022-02-22 Thread GitBox
hadoop-yetus commented on pull request #4013: URL: https://github.com/apache/hadoop/pull/4013#issuecomment-1048184448 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[GitHub] [hadoop] hadoop-yetus commented on pull request #3858: YARN-10918. Simplify method: CapacitySchedulerQueueManager#parseQueue

2022-02-22 Thread GitBox
hadoop-yetus commented on pull request #3858: URL: https://github.com/apache/hadoop/pull/3858#issuecomment-1048190431 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[GitHub] [hadoop] hadoop-yetus commented on pull request #3785: YARN-11042. Fix testQueueSubmitWithACLsEnabledWithQueueMapping in Tes…

2022-02-22 Thread GitBox
hadoop-yetus commented on pull request #3785: URL: https://github.com/apache/hadoop/pull/3785#issuecomment-1048217083 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[GitHub] [hadoop] hadoop-yetus commented on pull request #4012: YARN-11075. Explicitly declare serialVersionUID in LogMutation class.…

2022-02-22 Thread GitBox
hadoop-yetus commented on pull request #4012: URL: https://github.com/apache/hadoop/pull/4012#issuecomment-1048224125 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[GitHub] [hadoop] hadoop-yetus commented on pull request #4015: HADOOP-18127. Backport HADOOP-13055 into branch-2.10

2022-02-22 Thread GitBox
hadoop-yetus commented on pull request #4015: URL: https://github.com/apache/hadoop/pull/4015#issuecomment-1048232451 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[jira] [Work logged] (HADOOP-18127) Backport HADOOP-13055 into branch-2.10

2022-02-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18127?focusedWorklogId=731187&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-731187 ] ASF GitHub Bot logged work on HADOOP-18127: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus commented on pull request #3995: YARN-10049. FIFOOrderingPolicy Improvements

2022-02-22 Thread GitBox
hadoop-yetus commented on pull request #3995: URL: https://github.com/apache/hadoop/pull/3995#issuecomment-1048234685 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[GitHub] [hadoop] goiri commented on a change in pull request #4016: HDFS-16478 Add thread id to command processor name

2022-02-22 Thread GitBox
goiri commented on a change in pull request #4016: URL: https://github.com/apache/hadoop/pull/4016#discussion_r812388320 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPServiceActor.java ## @@ -1378,7 +1378,7 @@ public long

[GitHub] [hadoop] madrob commented on a change in pull request #4016: HDFS-16478 Add thread id to command processor name

2022-02-22 Thread GitBox
madrob commented on a change in pull request #4016: URL: https://github.com/apache/hadoop/pull/4016#discussion_r812391433 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPServiceActor.java ## @@ -1378,7 +1378,7 @@ public long

[GitHub] [hadoop] hadoop-yetus commented on pull request #4010: HDFS-16476.Increase the number of metrics used to record PendingRecoveryBlocks.

2022-02-22 Thread GitBox
hadoop-yetus commented on pull request #4010: URL: https://github.com/apache/hadoop/pull/4010#issuecomment-1048273369 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[GitHub] [hadoop] hadoop-yetus commented on pull request #3938: YARN-11069. Dynamic Queue ACL handling in Legacy and Flexible Auto Created Queues

2022-02-22 Thread GitBox
hadoop-yetus commented on pull request #3938: URL: https://github.com/apache/hadoop/pull/3938#issuecomment-1048282470 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[GitHub] [hadoop] smarthanwang commented on pull request #3909: HDFS-16431. Truncate CallerContext in client side.

2022-02-22 Thread GitBox
smarthanwang commented on pull request #3909: URL: https://github.com/apache/hadoop/pull/3909#issuecomment-1048392253 Hi @jojochuang @sodonnel @Hexiaoqiao @ferhui , could you please help take a look? -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [hadoop] jianghuazhu commented on pull request #4010: HDFS-16476.Increase the number of metrics used to record PendingRecoveryBlocks.

2022-02-22 Thread GitBox
jianghuazhu commented on pull request #4010: URL: https://github.com/apache/hadoop/pull/4010#issuecomment-1048400030 There are some ci/cd related failures here that don't seem to be related. Here is an example with some online clusters, obtained by getting the NameNode's jmx: ![image

[GitHub] [hadoop] jojochuang commented on a change in pull request #3966: [WIP] HADOOP-15980 : Enable TLS in RPC client/server

2022-02-22 Thread GitBox
jojochuang commented on a change in pull request #3966: URL: https://github.com/apache/hadoop/pull/3966#discussion_r812515540 ## File path: hadoop-common-project/hadoop-common/pom.xml ## @@ -1014,6 +1050,36 @@ + +o

[jira] [Work logged] (HADOOP-15980) Enable TLS in RPC client/server

2022-02-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-15980?focusedWorklogId=731331&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-731331 ] ASF GitHub Bot logged work on HADOOP-15980: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus commented on pull request #4016: HDFS-16478 Add thread id to command processor name

2022-02-22 Thread GitBox
hadoop-yetus commented on pull request #4016: URL: https://github.com/apache/hadoop/pull/4016#issuecomment-1048418622 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[GitHub] [hadoop] tomscut commented on pull request #4001: HDFS-16460. [SPS]: Handle failure retries for moving tasks

2022-02-22 Thread GitBox
tomscut commented on pull request #4001: URL: https://github.com/apache/hadoop/pull/4001#issuecomment-1048427499 Hi @jojochuang @tasanuma @Hexiaoqiao @ferhui , PTAL. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [hadoop] Hexiaoqiao commented on a change in pull request #4016: HDFS-16478 Add thread id to command processor name

2022-02-22 Thread GitBox
Hexiaoqiao commented on a change in pull request #4016: URL: https://github.com/apache/hadoop/pull/4016#discussion_r812548069 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPServiceActor.java ## @@ -1378,7 +1378,7 @@ public

[GitHub] [hadoop] Hexiaoqiao commented on pull request #3271: HDFS-16155: Allow configurable exponential backoff in DFSInputStream refetchLocations

2022-02-22 Thread GitBox
Hexiaoqiao commented on pull request #3271: URL: https://github.com/apache/hadoop/pull/3271#issuecomment-1048438966 Thanks @bbeaudreault for your great works here and sorry for the late response. It looks good to me in general. +2 from my side. I would like to wait if any other guys are in

[GitHub] [hadoop] Hexiaoqiao edited a comment on pull request #3271: HDFS-16155: Allow configurable exponential backoff in DFSInputStream refetchLocations

2022-02-22 Thread GitBox
Hexiaoqiao edited a comment on pull request #3271: URL: https://github.com/apache/hadoop/pull/3271#issuecomment-1048438966 Thanks @bbeaudreault for your great works here and sorry for the late response. It looks good to me in general. +1 from my side. I would like to wait if any other guys

[GitHub] [hadoop] Hexiaoqiao commented on pull request #3909: HDFS-16431. Truncate CallerContext in client side.

2022-02-22 Thread GitBox
Hexiaoqiao commented on pull request #3909: URL: https://github.com/apache/hadoop/pull/3909#issuecomment-1048440490 Thanks @smarthanwang involve me here. I am not very familiar with CallerContext. I think @jojochuang should be the good candidate to review it. -- This is an automated mess

[GitHub] [hadoop] madrob commented on a change in pull request #4016: HDFS-16478 Add thread id to command processor name

2022-02-22 Thread GitBox
madrob commented on a change in pull request #4016: URL: https://github.com/apache/hadoop/pull/4016#discussion_r812552531 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPServiceActor.java ## @@ -1378,7 +1378,7 @@ public long

[GitHub] [hadoop] Hexiaoqiao commented on pull request #3941: HDFS-15382. Split one FsDatasetImpl lock to block pool grain locks.

2022-02-22 Thread GitBox
Hexiaoqiao commented on pull request #3941: URL: https://github.com/apache/hadoop/pull/3941#issuecomment-1048442517 @jojochuang @tomscut The previous PR has merge refer to https://github.com/apache/hadoop/pull/3900. At this PR it improve LightWeightResizableGSet to thread-safe and create L

[GitHub] [hadoop] Hexiaoqiao commented on a change in pull request #4016: HDFS-16478 Add thread id to command processor name

2022-02-22 Thread GitBox
Hexiaoqiao commented on a change in pull request #4016: URL: https://github.com/apache/hadoop/pull/4016#discussion_r812559159 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPServiceActor.java ## @@ -1378,7 +1378,7 @@ public

[GitHub] [hadoop] ayushtkn commented on a change in pull request #4016: HDFS-16478 Add thread id to command processor name

2022-02-22 Thread GitBox
ayushtkn commented on a change in pull request #4016: URL: https://github.com/apache/hadoop/pull/4016#discussion_r812562187 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPServiceActor.java ## @@ -1378,7 +1378,7 @@ public lo

[GitHub] [hadoop] ayushtkn commented on a change in pull request #3987: HADOOP-18122. ViewFileSystem fails on determining owning group when primary group doesn't exist for user

2022-02-22 Thread GitBox
ayushtkn commented on a change in pull request #3987: URL: https://github.com/apache/hadoop/pull/3987#discussion_r812563504 ## File path: hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/ViewFileSystemBaseTest.java ## @@ -1671,4 +1675,62 @@ public

[jira] [Work logged] (HADOOP-18122) ViewFileSystem fails on determining owning group when primary group doesn't exist for user

2022-02-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18122?focusedWorklogId=731355&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-731355 ] ASF GitHub Bot logged work on HADOOP-18122: --- Author: ASF GitHub B

[GitHub] [hadoop] ayushtkn commented on pull request #4010: HDFS-16476.Increase the number of metrics used to record PendingRecoveryBlocks.

2022-02-22 Thread GitBox
ayushtkn commented on pull request #4010: URL: https://github.com/apache/hadoop/pull/4010#issuecomment-1048454886 > I think the block recovery work is something between NameNode and DataNode, so in the RBF module For all such cases we have got the values from all namenodes at Router and

[GitHub] [hadoop] ayushtkn edited a comment on pull request #4010: HDFS-16476.Increase the number of metrics used to record PendingRecoveryBlocks.

2022-02-22 Thread GitBox
ayushtkn edited a comment on pull request #4010: URL: https://github.com/apache/hadoop/pull/4010#issuecomment-1048454886 > I think the block recovery work is something between NameNode and DataNode, so in the RBF module For all such cases we have got the values from all namenodes at

[GitHub] [hadoop] jianghuazhu commented on pull request #4010: HDFS-16476.Increase the number of metrics used to record PendingRecoveryBlocks.

2022-02-22 Thread GitBox
jianghuazhu commented on pull request #4010: URL: https://github.com/apache/hadoop/pull/4010#issuecomment-1048463738 OK, I'll try to perfect it as much as possible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [hadoop] ayushtkn commented on pull request #3862: HDFS-11107.TestStartup#testStorageBlockContentsStaleAfterNNRestart fails intermittently.

2022-02-22 Thread GitBox
ayushtkn commented on pull request #3862: URL: https://github.com/apache/hadoop/pull/3862#issuecomment-1048485017 The error message in the description is an Assertion Error. Not a timeout error. ``` Error Message expected:<0> but was:<2> Stacktrace java.lang.Assertio

[GitHub] [hadoop] ChaosJu opened a new pull request #4017: YARN-11080. Improve debug log in ContainerTokenIdentifier when writin…

2022-02-22 Thread GitBox
ChaosJu opened a new pull request #4017: URL: https://github.com/apache/hadoop/pull/4017 …g ContainerTokenIdentifier to RPC layer ### Description of PR ### How was this patch tested? ### For code changes: - [ ] Does the title or this PR starts with

[GitHub] [hadoop] ChaosJu commented on pull request #4017: YARN-11080. Improve debug log in ContainerTokenIdentifier when writin…

2022-02-22 Thread GitBox
ChaosJu commented on pull request #4017: URL: https://github.com/apache/hadoop/pull/4017#issuecomment-1048492932 @zhuqi-lucas @szilard-nemeth ,would you please take a look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [hadoop] jojochuang commented on a change in pull request #3941: HDFS-15382. Split one FsDatasetImpl lock to block pool grain locks.

2022-02-22 Thread GitBox
jojochuang commented on a change in pull request #3941: URL: https://github.com/apache/hadoop/pull/3941#discussion_r812604507 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/ReplicaMap.java ## @@ -6,58 +6,62 @@

[GitHub] [hadoop] jojochuang commented on a change in pull request #3941: HDFS-15382. Split one FsDatasetImpl lock to block pool grain locks.

2022-02-22 Thread GitBox
jojochuang commented on a change in pull request #3941: URL: https://github.com/apache/hadoop/pull/3941#discussion_r812604507 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/ReplicaMap.java ## @@ -6,58 +6,62 @@

[GitHub] [hadoop] jojochuang commented on a change in pull request #3966: [WIP] HADOOP-15980 : Enable TLS in RPC client/server

2022-02-22 Thread GitBox
jojochuang commented on a change in pull request #3966: URL: https://github.com/apache/hadoop/pull/3966#discussion_r812609428 ## File path: hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestNettyRPC.java ## @@ -0,0 +1,1742 @@ +/** Review comment:

  1   2   >