[GitHub] [hadoop] TJJack opened a new pull request #2803: YARN-10708. Remove NULL checks before instanceof

2021-03-22 Thread GitBox
TJJack opened a new pull request #2803: URL: https://github.com/apache/hadoop/pull/2803 JIRA ISSUE: [https://issues.apache.org/jira/browse/YARN-10708](https://issues.apache.org/jira/browse/YARN-10708) -- This is an automated message from the Apache Git Service. To respond to the message,

[jira] [Work logged] (HADOOP-17596) ABFS: Change default Readahead Queue Depth from num(processors) to const

2021-03-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17596?focusedWorklogId=570266&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570266 ] ASF GitHub Bot logged work on HADOOP-17596: --- Author: ASF GitHub B

[GitHub] [hadoop] sumangala-patki commented on pull request #2795: HADOOP-17596. ABFS: Change default Readahead Queue Depth from num(processors) to const

2021-03-22 Thread GitBox
sumangala-patki commented on pull request #2795: URL: https://github.com/apache/hadoop/pull/2795#issuecomment-804657103 TEST RESULTS HNS Account Location: East US 2 NonHNS Account Location: East US 2, Central US ``` HNS OAuth [INFO] Tests run: 94, Failures: 0, Erro

[GitHub] [hadoop] liuml07 commented on pull request #1942: MAPREDUCE-7270. TestHistoryViewerPrinter could be failed when the locale isn't English.

2021-03-22 Thread GitBox
liuml07 commented on pull request #1942: URL: https://github.com/apache/hadoop/pull/1942#issuecomment-804651653 Thank you very much @ayushtkn The report was not posted here, but the new run was fine. Since there is a checkstyle warning, could you rebase this PR from `trunk` branch,

[GitHub] [hadoop] virajjasani commented on a change in pull request #2794: HDFS-15911 : Provide blocks moved count in Balancer iteration result

2021-03-22 Thread GitBox
virajjasani commented on a change in pull request #2794: URL: https://github.com/apache/hadoop/pull/2794#discussion_r599298222 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java ## @@ -2309,8 +2310,11 @@ public

[GitHub] [hadoop] liuml07 commented on a change in pull request #2794: HDFS-15911 : Provide blocks moved count in Balancer iteration result

2021-03-22 Thread GitBox
liuml07 commented on a change in pull request #2794: URL: https://github.com/apache/hadoop/pull/2794#discussion_r599297317 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java ## @@ -2309,8 +2310,11 @@ public void

[GitHub] [hadoop] virajjasani commented on pull request #2799: HDFS-15911 : Provide blocks moved count in Balancer iteration result

2021-03-22 Thread GitBox
virajjasani commented on pull request #2799: URL: https://github.com/apache/hadoop/pull/2799#issuecomment-804647792 Build failure doesn't seem related to PR changes for branch-3.1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[jira] [Work logged] (HADOOP-17596) ABFS: Change default Readahead Queue Depth from num(processors) to const

2021-03-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17596?focusedWorklogId=570249&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570249 ] ASF GitHub Bot logged work on HADOOP-17596: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus commented on pull request #2795: HADOOP-17596. ABFS: Change default Readahead Queue Depth from num(processors) to const

2021-03-22 Thread GitBox
hadoop-yetus commented on pull request #2795: URL: https://github.com/apache/hadoop/pull/2795#issuecomment-804643431 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[GitHub] [hadoop] hadoop-yetus commented on pull request #2784: HDFS-15850. Superuser actions should be reported to external enforcers

2021-03-22 Thread GitBox
hadoop-yetus commented on pull request #2784: URL: https://github.com/apache/hadoop/pull/2784#issuecomment-804640926 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] runitao commented on a change in pull request #2585: HDFS-15759. EC: Verify EC reconstruction correctness on DataNode

2021-03-22 Thread GitBox
runitao commented on a change in pull request #2585: URL: https://github.com/apache/hadoop/pull/2585#discussion_r599281959 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReconstructStripedFileWithValidator.java ## @@ -0,0 +1,115 @@ +/**

[jira] [Resolved] (HADOOP-17598) Fix java doc issue introduced by HADOOP-17578

2021-03-22 Thread Ayush Saxena (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved HADOOP-17598. --- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed Merged to trunk

[jira] [Work logged] (HADOOP-17598) Fix java doc issue introduced by HADOOP-17578

2021-03-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17598?focusedWorklogId=570230&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570230 ] ASF GitHub Bot logged work on HADOOP-17598: --- Author: ASF GitHub B

[GitHub] [hadoop] ayushtkn merged pull request #2802: HADOOP-17598. Fix java doc issue introduced by HADOOP-17578.

2021-03-22 Thread GitBox
ayushtkn merged pull request #2802: URL: https://github.com/apache/hadoop/pull/2802 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [hadoop] virajjasani commented on pull request #2794: HDFS-15911 : Provide blocks moved count in Balancer iteration result

2021-03-22 Thread GitBox
virajjasani commented on pull request #2794: URL: https://github.com/apache/hadoop/pull/2794#issuecomment-804607379 > Thanx @virajjasani, Can you take care of the checkstyle complains. Done, Thanks. @ayushtkn @liuml07 would you like to take a look? I have raised backport PRs too.

[jira] [Work logged] (HADOOP-16948) ABFS: Support single writer dirs

2021-03-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16948?focusedWorklogId=570209&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570209 ] ASF GitHub Bot logged work on HADOOP-16948: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus commented on pull request #1925: HADOOP-16948. Support single writer dirs.

2021-03-22 Thread GitBox
hadoop-yetus commented on pull request #1925: URL: https://github.com/apache/hadoop/pull/1925#issuecomment-804560110 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[jira] [Work logged] (HADOOP-17598) Fix java doc issue introduced by HADOOP-17578

2021-03-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17598?focusedWorklogId=570203&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570203 ] ASF GitHub Bot logged work on HADOOP-17598: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus commented on pull request #2802: HADOOP-17598. Fix java doc issue introduced by HADOOP-17578.

2021-03-22 Thread GitBox
hadoop-yetus commented on pull request #2802: URL: https://github.com/apache/hadoop/pull/2802#issuecomment-804548577 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] tomscut commented on pull request #2800: HDFS-15906. Close FSImage and FSNamesystem after formatting is complete for branch-3.2

2021-03-22 Thread GitBox
tomscut commented on pull request #2800: URL: https://github.com/apache/hadoop/pull/2800#issuecomment-804547668 Thanks @tasanuma for your test. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [hadoop] tasanuma commented on pull request #2787: HDFS-15900. RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode.

2021-03-22 Thread GitBox
tasanuma commented on pull request #2787: URL: https://github.com/apache/hadoop/pull/2787#issuecomment-804537584 The failed tests succeeded in my local environment and seem not to related. -- This is an automated message from the Apache Git Service. To respond to the message, please log o

[GitHub] [hadoop] tasanuma commented on pull request #2800: HDFS-15906. Close FSImage and FSNamesystem after formatting is complete for branch-3.2

2021-03-22 Thread GitBox
tasanuma commented on pull request #2800: URL: https://github.com/apache/hadoop/pull/2800#issuecomment-804533980 The failed tests succeeded in my local environment. I think the spotbugs warnings are not related. I will file them later. -- This is an automated message from the Apache Gi

[GitHub] [hadoop] tasanuma commented on pull request #2787: HDFS-15900. RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode.

2021-03-22 Thread GitBox
tasanuma commented on pull request #2787: URL: https://github.com/apache/hadoop/pull/2787#issuecomment-804532574 @hdaikoku Could you fix the checkstyle warnings in the CI report? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [hadoop] hadoop-yetus commented on pull request #2787: HDFS-15900. RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode.

2021-03-22 Thread GitBox
hadoop-yetus commented on pull request #2787: URL: https://github.com/apache/hadoop/pull/2787#issuecomment-804529045 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[jira] [Work logged] (HADOOP-17531) DistCp: Reduce memory usage on copying huge directories

2021-03-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17531?focusedWorklogId=570185&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570185 ] ASF GitHub Bot logged work on HADOOP-17531: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus commented on pull request #2732: HADOOP-17531. DistCp: Reduce memory usage on copying huge directories.

2021-03-22 Thread GitBox
hadoop-yetus commented on pull request #2732: URL: https://github.com/apache/hadoop/pull/2732#issuecomment-804525566 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] aajisaka commented on a change in pull request #2787: HDFS-15900. RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode.

2021-03-22 Thread GitBox
aajisaka commented on a change in pull request #2787: URL: https://github.com/apache/hadoop/pull/2787#discussion_r599190892 ## File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/store/TestStateStoreMembershipState.java ## @@ -

[GitHub] [hadoop] tasanuma commented on pull request #2787: HDFS-15900. RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode.

2021-03-22 Thread GitBox
tasanuma commented on pull request #2787: URL: https://github.com/apache/hadoop/pull/2787#issuecomment-804510471 +1, pending jenkins. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hadoop] hdaikoku commented on a change in pull request #2787: HDFS-15900. RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode.

2021-03-22 Thread GitBox
hdaikoku commented on a change in pull request #2787: URL: https://github.com/apache/hadoop/pull/2787#discussion_r599170545 ## File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/store/TestStateStoreMembershipState.java ## @@ -

[GitHub] [hadoop] hdaikoku commented on a change in pull request #2787: HDFS-15900. RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode.

2021-03-22 Thread GitBox
hdaikoku commented on a change in pull request #2787: URL: https://github.com/apache/hadoop/pull/2787#discussion_r599170545 ## File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/store/TestStateStoreMembershipState.java ## @@ -

[GitHub] [hadoop] hdaikoku commented on a change in pull request #2787: HDFS-15900. RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode.

2021-03-22 Thread GitBox
hdaikoku commented on a change in pull request #2787: URL: https://github.com/apache/hadoop/pull/2787#discussion_r599167124 ## File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/store/TestStateStoreMembershipState.java ## @@ -

[GitHub] [hadoop] hadoop-yetus commented on pull request #2794: HDFS-15911 : Provide blocks moved count in Balancer iteration result

2021-03-22 Thread GitBox
hadoop-yetus commented on pull request #2794: URL: https://github.com/apache/hadoop/pull/2794#issuecomment-804491621 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] hadoop-yetus commented on pull request #2748: HDFS-15879. Exclude slow nodes when choose targets for blocks

2021-03-22 Thread GitBox
hadoop-yetus commented on pull request #2748: URL: https://github.com/apache/hadoop/pull/2748#issuecomment-804487762 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[jira] [Work logged] (HADOOP-17578) Improve UGI debug log to help troubleshooting TokenCache related issues

2021-03-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17578?focusedWorklogId=570152&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570152 ] ASF GitHub Bot logged work on HADOOP-17578: --- Author: ASF GitHub B

[GitHub] [hadoop] xiaoyuyao commented on pull request #2762: HADOOP-17578. Improve UGI debug log to help troubleshooting TokenCach…

2021-03-22 Thread GitBox
xiaoyuyao commented on pull request #2762: URL: https://github.com/apache/hadoop/pull/2762#issuecomment-804480648 https://github.com/apache/hadoop/pull/2802 is opened to fix the javadoc issue. -- This is an automated message from the Apache Git Service. To respond to the message, please

[jira] [Updated] (HADOOP-17598) Fix java doc issue introduced by HADOOP-17578

2021-03-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-17598: Labels: pull-request-available (was: ) > Fix java doc issue introduced by HADOOP-17578 >

[jira] [Work logged] (HADOOP-17598) Fix java doc issue introduced by HADOOP-17578

2021-03-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17598?focusedWorklogId=570151&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570151 ] ASF GitHub Bot logged work on HADOOP-17598: --- Author: ASF GitHub B

[GitHub] [hadoop] xiaoyuyao opened a new pull request #2802: HADOOP-17598. Fix java doc issue introduced by HADOOP-17578.

2021-03-22 Thread GitBox
xiaoyuyao opened a new pull request #2802: URL: https://github.com/apache/hadoop/pull/2802 https://issues.apache.org/jira/browse/HADOOP-17598 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[jira] [Updated] (HADOOP-17598) Fix java doc issue introduced by HADOOP-17578

2021-03-22 Thread Xiaoyu Yao (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HADOOP-17598: Priority: Minor (was: Major) > Fix java doc issue introduced by HADOOP-17578 > -

[jira] [Created] (HADOOP-17598) Fix java doc issue introduced by HADOOP-17578

2021-03-22 Thread Xiaoyu Yao (Jira)
Xiaoyu Yao created HADOOP-17598: --- Summary: Fix java doc issue introduced by HADOOP-17578 Key: HADOOP-17598 URL: https://issues.apache.org/jira/browse/HADOOP-17598 Project: Hadoop Common Issue T

[jira] [Work logged] (HADOOP-17578) Improve UGI debug log to help troubleshooting TokenCache related issues

2021-03-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17578?focusedWorklogId=570147&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570147 ] ASF GitHub Bot logged work on HADOOP-17578: --- Author: ASF GitHub B

[GitHub] [hadoop] xiaoyuyao commented on pull request #2762: HADOOP-17578. Improve UGI debug log to help troubleshooting TokenCach…

2021-03-22 Thread GitBox
xiaoyuyao commented on pull request #2762: URL: https://github.com/apache/hadoop/pull/2762#issuecomment-804474635 Thanks @steveloughran and @aajisaka. I will open a new ticket for the javadoc fix. -- This is an automated message from the Apache Git Service. To respond to the message, p

[GitHub] [hadoop] vivekratnavel commented on pull request #1528: HDDS-2181. Ozone Manager should send correct ACL type in ACL requests…

2021-03-22 Thread GitBox
vivekratnavel commented on pull request #1528: URL: https://github.com/apache/hadoop/pull/1528#issuecomment-804471306 Merged via https://github.com/apache/hadoop-ozone/pull/43 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

[GitHub] [hadoop] vivekratnavel closed pull request #1528: HDDS-2181. Ozone Manager should send correct ACL type in ACL requests…

2021-03-22 Thread GitBox
vivekratnavel closed pull request #1528: URL: https://github.com/apache/hadoop/pull/1528 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, p

[jira] [Work logged] (HADOOP-17471) ABFS to collect IOStatistics

2021-03-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17471?focusedWorklogId=570094&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570094 ] ASF GitHub Bot logged work on HADOOP-17471: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on pull request #2731: HADOOP-17471. ABFS to collect IOStatistics

2021-03-22 Thread GitBox
steveloughran commented on pull request #2731: URL: https://github.com/apache/hadoop/pull/2731#issuecomment-804430014 Looks great! My main production suggestion is to make the counters a DurationTrackerFactory, giving us the option of feeding it in to anything we add in Hadoop common &c

[GitHub] [hadoop] hadoop-yetus commented on pull request #2796: HDFS-15911 : Provide blocks moved count in Balancer iteration result

2021-03-22 Thread GitBox
hadoop-yetus commented on pull request #2796: URL: https://github.com/apache/hadoop/pull/2796#issuecomment-804425039 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[jira] [Work logged] (HADOOP-17531) DistCp: Reduce memory usage on copying huge directories

2021-03-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17531?focusedWorklogId=570075&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570075 ] ASF GitHub Bot logged work on HADOOP-17531: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on pull request #2732: HADOOP-17531. DistCp: Reduce memory usage on copying huge directories.

2021-03-22 Thread GitBox
steveloughran commented on pull request #2732: URL: https://github.com/apache/hadoop/pull/2732#issuecomment-804420292 LGTM. I'm going to make one final change, but +1 this PR pending that change anyway. Can you put the CommonCallableSupplier into `org.apache.hadoop.util.func

[jira] [Commented] (HADOOP-17222) Create socket address leveraging URI cache

2021-03-22 Thread Mingliang Liu (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17306606#comment-17306606 ] Mingliang Liu commented on HADOOP-17222: Thanks [~sodonnell].  I guess I do no

[jira] [Work logged] (HADOOP-17578) Improve UGI debug log to help troubleshooting TokenCache related issues

2021-03-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17578?focusedWorklogId=570073&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570073 ] ASF GitHub Bot logged work on HADOOP-17578: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on pull request #2762: HADOOP-17578. Improve UGI debug log to help troubleshooting TokenCach…

2021-03-22 Thread GitBox
steveloughran commented on pull request #2762: URL: https://github.com/apache/hadoop/pull/2762#issuecomment-804418330 yeah, we do need a fix for this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[GitHub] [hadoop] hadoop-yetus commented on pull request #2797: HDFS-15911 : Provide blocks moved count in Balancer iteration result

2021-03-22 Thread GitBox
hadoop-yetus commented on pull request #2797: URL: https://github.com/apache/hadoop/pull/2797#issuecomment-804410150 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[jira] [Work logged] (HADOOP-17597) Add option to downgrade S3A rejection of Syncable to warning

2021-03-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17597?focusedWorklogId=570045&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570045 ] ASF GitHub Bot logged work on HADOOP-17597: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on pull request #2801: HADOOP-17597. Optionally downgrade on S3A Syncable calls

2021-03-22 Thread GitBox
steveloughran commented on pull request #2801: URL: https://github.com/apache/hadoop/pull/2801#issuecomment-804401718 Test s3 london with test options ` -Dparallel-tests -DtestsThreadCount=8 -Dmarkers=keep -Dscale` Failures ``` [ERROR] Errors: [ERROR] ITestDelegatedMRJob.

[GitHub] [hadoop] hadoop-yetus commented on pull request #2800: HDFS-15906. Close FSImage and FSNamesystem after formatting is complete for branch-3.2

2021-03-22 Thread GitBox
hadoop-yetus commented on pull request #2800: URL: https://github.com/apache/hadoop/pull/2800#issuecomment-804379198 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] xiaoyuyao commented on a change in pull request #2784: HDFS-15850. Superuser actions should be reported to external enforcers

2021-03-22 Thread GitBox
xiaoyuyao commented on a change in pull request #2784: URL: https://github.com/apache/hadoop/pull/2784#discussion_r599041257 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java ## @@ -7665,7 +7670,7 @@ void addCa

[GitHub] [hadoop] hadoop-yetus commented on pull request #2796: HDFS-15911 : Provide blocks moved count in Balancer iteration result

2021-03-22 Thread GitBox
hadoop-yetus commented on pull request #2796: URL: https://github.com/apache/hadoop/pull/2796#issuecomment-804359684 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] premal closed pull request #2617: Patch for ATS

2021-03-22 Thread GitBox
premal closed pull request #2617: URL: https://github.com/apache/hadoop/pull/2617 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please c

[jira] [Work logged] (HADOOP-17597) Add option to downgrade S3A rejection of Syncable to warning

2021-03-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17597?focusedWorklogId=569970&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-569970 ] ASF GitHub Bot logged work on HADOOP-17597: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus commented on pull request #2801: HADOOP-17597. Optionally downgrade on S3A Syncable calls

2021-03-22 Thread GitBox
hadoop-yetus commented on pull request #2801: URL: https://github.com/apache/hadoop/pull/2801#issuecomment-804336566 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] hadoop-yetus commented on pull request #2796: HDFS-15911 : Provide blocks moved count in Balancer iteration result

2021-03-22 Thread GitBox
hadoop-yetus commented on pull request #2796: URL: https://github.com/apache/hadoop/pull/2796#issuecomment-804300239 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] hadoop-yetus commented on pull request #2797: HDFS-15911 : Provide blocks moved count in Balancer iteration result

2021-03-22 Thread GitBox
hadoop-yetus commented on pull request #2797: URL: https://github.com/apache/hadoop/pull/2797#issuecomment-804295328 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] hadoop-yetus commented on pull request #2799: HDFS-15911 : Provide blocks moved count in Balancer iteration result

2021-03-22 Thread GitBox
hadoop-yetus commented on pull request #2799: URL: https://github.com/apache/hadoop/pull/2799#issuecomment-804271262 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 22m 8

[GitHub] [hadoop] saintstack commented on pull request #2693: Hadoop 16524 - resubmission following some unit test fixes

2021-03-22 Thread GitBox
saintstack commented on pull request #2693: URL: https://github.com/apache/hadoop/pull/2693#issuecomment-804268943 I see: ``` [ERROR] Tests run: 21, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 450.303 s <<< FAILURE! - in org.apache.hadoop.yarn.applications.distributedsh

[GitHub] [hadoop] hadoop-yetus commented on pull request #2799: HDFS-15911 : Provide blocks moved count in Balancer iteration result

2021-03-22 Thread GitBox
hadoop-yetus commented on pull request #2799: URL: https://github.com/apache/hadoop/pull/2799#issuecomment-804257123 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 18m 29

[GitHub] [hadoop] tasanuma commented on pull request #2800: HDFS-15906. Close FSImage and FSNamesystem after formatting is complete for branch-3.2

2021-03-22 Thread GitBox
tasanuma commented on pull request #2800: URL: https://github.com/apache/hadoop/pull/2800#issuecomment-804254973 +1, pending Jenkins. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hadoop] hadoop-yetus commented on pull request #2790: HDFS-15908. Possible Resource Leak in org.apache.hadoop.hdfs.qjournal.server.Journal

2021-03-22 Thread GitBox
hadoop-yetus commented on pull request #2790: URL: https://github.com/apache/hadoop/pull/2790#issuecomment-804254342 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] tasanuma commented on a change in pull request #2585: HDFS-15759. EC: Verify EC reconstruction correctness on DataNode

2021-03-22 Thread GitBox
tasanuma commented on a change in pull request #2585: URL: https://github.com/apache/hadoop/pull/2585#discussion_r598925928 ## File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/DecodingValidator.java ## @@ -0,0 +1,187 @@ +/*

[GitHub] [hadoop] hadoop-yetus commented on pull request #2748: HDFS-15879. Exclude slow nodes when choose targets for blocks

2021-03-22 Thread GitBox
hadoop-yetus commented on pull request #2748: URL: https://github.com/apache/hadoop/pull/2748#issuecomment-804220194 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] ayushtkn commented on pull request #2794: HDFS-15911 : Provide blocks moved count in Balancer iteration result

2021-03-22 Thread GitBox
ayushtkn commented on pull request #2794: URL: https://github.com/apache/hadoop/pull/2794#issuecomment-804213263 Thanx @virajjasani, Can you take care of the checkstyle complains. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [hadoop] hadoop-yetus commented on pull request #2794: HDFS-15911 : Provide blocks moved count in Balancer iteration result

2021-03-22 Thread GitBox
hadoop-yetus commented on pull request #2794: URL: https://github.com/apache/hadoop/pull/2794#issuecomment-804190199 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] Nargeshdb commented on a change in pull request #2790: HDFS-15908. Possible Resource Leak in org.apache.hadoop.hdfs.qjournal.server.Journal

2021-03-22 Thread GitBox
Nargeshdb commented on a change in pull request #2790: URL: https://github.com/apache/hadoop/pull/2790#discussion_r598850886 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/qjournal/server/Journal.java ## @@ -264,9 +264,9 @@ void format(Names

[GitHub] [hadoop] steveloughran commented on pull request #2801: HADOOP-17597. Optionally downgrade on S3A Syncable calls

2021-03-22 Thread GitBox
steveloughran commented on pull request #2801: URL: https://github.com/apache/hadoop/pull/2801#issuecomment-804178990 Testing: new tests all happy. Now regression testing with s3 london -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

[GitHub] [hadoop] steveloughran opened a new pull request #2801: HADOOP-17597. Optionally downgrade on S3A Syncable calls

2021-03-22 Thread GitBox
steveloughran opened a new pull request #2801: URL: https://github.com/apache/hadoop/pull/2801 Adds a new option fs.s3a.downgrade.syncable.exceptions Which converts calls to Syncable hsync/hflush on S3A output streams to * log once at warn (for entire process life, not just the

[jira] [Work logged] (HADOOP-17597) Add option to downgrade S3A rejection of Syncable to warning

2021-03-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17597?focusedWorklogId=569813&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-569813 ] ASF GitHub Bot logged work on HADOOP-17597: --- Author: ASF GitHub B

[jira] [Updated] (HADOOP-17597) Add option to downgrade S3A rejection of Syncable to warning

2021-03-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-17597: Labels: pull-request-available (was: ) > Add option to downgrade S3A rejection of Syncab

[jira] [Work logged] (HADOOP-17597) Add option to downgrade S3A rejection of Syncable to warning

2021-03-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17597?focusedWorklogId=569812&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-569812 ] ASF GitHub Bot logged work on HADOOP-17597: --- Author: ASF GitHub B

[GitHub] [hadoop] goiri commented on a change in pull request #2787: HDFS-15900. RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode.

2021-03-22 Thread GitBox
goiri commented on a change in pull request #2787: URL: https://github.com/apache/hadoop/pull/2787#discussion_r598845045 ## File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/store/TestStateStoreMembershipState.java ## @@ -473

[GitHub] [hadoop] goiri commented on a change in pull request #2787: HDFS-15900. RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode.

2021-03-22 Thread GitBox
goiri commented on a change in pull request #2787: URL: https://github.com/apache/hadoop/pull/2787#discussion_r598843870 ## File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/store/TestStateStoreMembershipState.java ## @@ -473

[GitHub] [hadoop] hadoop-yetus commented on pull request #2799: HDFS-15911 : Provide blocks moved count in Balancer iteration result

2021-03-22 Thread GitBox
hadoop-yetus commented on pull request #2799: URL: https://github.com/apache/hadoop/pull/2799#issuecomment-804169499 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 13m 8

[jira] [Work logged] (HADOOP-16948) ABFS: Support single writer dirs

2021-03-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16948?focusedWorklogId=569804&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-569804 ] ASF GitHub Bot logged work on HADOOP-16948: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus commented on pull request #1925: HADOOP-16948. Support single writer dirs.

2021-03-22 Thread GitBox
hadoop-yetus commented on pull request #1925: URL: https://github.com/apache/hadoop/pull/1925#issuecomment-804164776 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[GitHub] [hadoop] goiri merged pull request #2783: HDFS-15903. Refactor X-Platform lib

2021-03-22 Thread GitBox
goiri merged pull request #2783: URL: https://github.com/apache/hadoop/pull/2783 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please co

[GitHub] [hadoop] tomscut commented on pull request #2788: HDFS-15906. Close FSImage and FSNamesystem after formatting is complete

2021-03-22 Thread GitBox
tomscut commented on pull request #2788: URL: https://github.com/apache/hadoop/pull/2788#issuecomment-804157399 Thanks @tasanuma for your merge. I submitted a [PR](https://github.com/apache/hadoop/pull/2800) for branch-3.2. -- This is an automated message from the Apache Git Service. To

[GitHub] [hadoop] tomscut opened a new pull request #2800: HDFS-15906. Close FSImage and FSNamesystem after formatting is complete for branch-3.2

2021-03-22 Thread GitBox
tomscut opened a new pull request #2800: URL: https://github.com/apache/hadoop/pull/2800 Close FSImage and FSNamesystem after formatting is complete. For branch-3.2. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [hadoop] hadoop-yetus commented on pull request #2798: MAPREDUCE-7330. Add access check for getJobAttempts

2021-03-22 Thread GitBox
hadoop-yetus commented on pull request #2798: URL: https://github.com/apache/hadoop/pull/2798#issuecomment-804154415 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] virajjasani opened a new pull request #2799: HDFS-15911 : Provide blocks moved count in Balancer iteration result

2021-03-22 Thread GitBox
virajjasani opened a new pull request #2799: URL: https://github.com/apache/hadoop/pull/2799 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this servic

[GitHub] [hadoop] hadoop-yetus commented on pull request #2741: HDFS-15855.Solve the problem of incorrect EC progress when loading FsImage.

2021-03-22 Thread GitBox
hadoop-yetus commented on pull request #2741: URL: https://github.com/apache/hadoop/pull/2741#issuecomment-804139203 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[jira] [Work logged] (HADOOP-16948) ABFS: Support single writer dirs

2021-03-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16948?focusedWorklogId=569750&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-569750 ] ASF GitHub Bot logged work on HADOOP-16948: --- Author: ASF GitHub B

[GitHub] [hadoop] billierinaldi commented on a change in pull request #1925: HADOOP-16948. Support single writer dirs.

2021-03-22 Thread GitBox
billierinaldi commented on a change in pull request #1925: URL: https://github.com/apache/hadoop/pull/1925#discussion_r598767866 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsLease.java ## @@ -0,0 +1,165 @@ +/** + * Licensed t

[jira] [Work logged] (HADOOP-16948) ABFS: Support single writer dirs

2021-03-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16948?focusedWorklogId=569745&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-569745 ] ASF GitHub Bot logged work on HADOOP-16948: --- Author: ASF GitHub B

[GitHub] [hadoop] billierinaldi commented on a change in pull request #1925: HADOOP-16948. Support single writer dirs.

2021-03-22 Thread GitBox
billierinaldi commented on a change in pull request #1925: URL: https://github.com/apache/hadoop/pull/1925#discussion_r598764212 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsLease.java ## @@ -0,0 +1,165 @@ +/** + * Licensed t

[GitHub] [hadoop] lujiefsi opened a new pull request #2798: MAPREDUCE-7330. Add access check for getJobAttempts

2021-03-22 Thread GitBox
lujiefsi opened a new pull request #2798: URL: https://github.com/apache/hadoop/pull/2798 Hi, i have found an inconsist, all the methods who get job infomation have access check, except getJobAttempts. -- This is an automated message from the Apache Git Service. To respond to the messag

[GitHub] [hadoop] hadoop-yetus commented on pull request #2794: HDFS-15911 : Provide blocks moved count in Balancer iteration result

2021-03-22 Thread GitBox
hadoop-yetus commented on pull request #2794: URL: https://github.com/apache/hadoop/pull/2794#issuecomment-804092518 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] virajjasani opened a new pull request #2797: HDFS-15911 : Provide blocks moved count in Balancer iteration result

2021-03-22 Thread GitBox
virajjasani opened a new pull request #2797: URL: https://github.com/apache/hadoop/pull/2797 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this servic

[jira] [Work logged] (HADOOP-17511) Add an Audit plugin point for S3A auditing/context

2021-03-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17511?focusedWorklogId=569695&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-569695 ] ASF GitHub Bot logged work on HADOOP-17511: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus commented on pull request #2675: HADOOP-17511. Add audit/telemetry logging to S3A connector

2021-03-22 Thread GitBox
hadoop-yetus commented on pull request #2675: URL: https://github.com/apache/hadoop/pull/2675#issuecomment-804053931 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[jira] [Created] (HADOOP-17597) Add option to downgrade S3A rejection of Syncable to warning + iostatistics

2021-03-22 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17597: --- Summary: Add option to downgrade S3A rejection of Syncable to warning + iostatistics Key: HADOOP-17597 URL: https://issues.apache.org/jira/browse/HADOOP-17597 P

[jira] [Updated] (HADOOP-17597) Add option to downgrade S3A rejection of Syncable to warning

2021-03-22 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17597: Summary: Add option to downgrade S3A rejection of Syncable to warning (was: Add option t

  1   2   >