[GitHub] [hadoop] LeonGao91 opened a new pull request #3284: HDFS-16157. Support configuring DNS record to get list of journal nodes.

2021-08-09 Thread GitBox
LeonGao91 opened a new pull request #3284: URL: https://github.com/apache/hadoop/pull/3284 ## NOTICE Please create an issue in ASF JIRA before opening a pull request, and you need to set the title of the pull request which starts with the corresponding JIRA issue number. (e.g. H

[jira] [Commented] (HADOOP-17785) mvn test failed about hadoop@3.2.1

2021-08-09 Thread shixijun (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17396394#comment-17396394 ] shixijun commented on HADOOP-17785: --- but running as not root can be suceed?Can you tr

[jira] [Work logged] (HADOOP-17799) Improve the GitHub pull request template

2021-08-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17799?focusedWorklogId=636241&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-636241 ] ASF GitHub Bot logged work on HADOOP-17799: --- Author: ASF GitHub B

[GitHub] [hadoop] jojochuang commented on a change in pull request #3277: HADOOP-17799. Improve the GitHub pull request template

2021-08-09 Thread GitBox
jojochuang commented on a change in pull request #3277: URL: https://github.com/apache/hadoop/pull/3277#discussion_r685647872 ## File path: .github/pull_request_template.md ## @@ -1,6 +1,18 @@ -## NOTICE + + +### Description of PR + + +### How was this patch tested? + + +### Fo

[jira] [Work logged] (HADOOP-17841) Remove ListenerHandle from Hadoop registry

2021-08-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17841?focusedWorklogId=636230&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-636230 ] ASF GitHub Bot logged work on HADOOP-17841: --- Author: ASF GitHub B

[GitHub] [hadoop] aajisaka commented on pull request #3278: HADOOP-17841. Remove ListenerHandle from Hadoop registry

2021-08-09 Thread GitBox
aajisaka commented on pull request #3278: URL: https://github.com/apache/hadoop/pull/3278#issuecomment-895677988 Thank you @virajjasani and @jojochuang -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [hadoop] shvachko commented on a change in pull request #3232: HDFS-16141. Address permission related issues with File / Directory

2021-08-09 Thread GitBox
shvachko commented on a change in pull request #3232: URL: https://github.com/apache/hadoop/pull/3232#discussion_r685606157 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirMkdirOp.java ## @@ -284,8 +294,8 @@ private stati

[GitHub] [hadoop] shuzirra merged pull request #3283: YARN-10814 3.2 backport

2021-08-09 Thread GitBox
shuzirra merged pull request #3283: URL: https://github.com/apache/hadoop/pull/3283 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-

[GitHub] [hadoop] shuzirra commented on pull request #3283: YARN-10814 3.2 backport

2021-08-09 Thread GitBox
shuzirra commented on pull request #3283: URL: https://github.com/apache/hadoop/pull/3283#issuecomment-895621081 @tomicooler thank you for backporting this, LGTM+1. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [hadoop] hadoop-yetus commented on pull request #3280: [Test PR] De-flake testDecommissionStatus

2021-08-09 Thread GitBox
hadoop-yetus commented on pull request #3280: URL: https://github.com/apache/hadoop/pull/3280#issuecomment-895482115 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] hadoop-yetus commented on pull request #3280: [Test PR] De-flake testDecommissionStatus

2021-08-09 Thread GitBox
hadoop-yetus commented on pull request #3280: URL: https://github.com/apache/hadoop/pull/3280#issuecomment-895479913 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] hadoop-yetus commented on pull request #3280: [Test PR] De-flake testDecommissionStatus

2021-08-09 Thread GitBox
hadoop-yetus commented on pull request #3280: URL: https://github.com/apache/hadoop/pull/3280#issuecomment-895411917 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[GitHub] [hadoop] hadoop-yetus commented on pull request #3280: [Test PR] De-flake testDecommissionStatus

2021-08-09 Thread GitBox
hadoop-yetus commented on pull request #3280: URL: https://github.com/apache/hadoop/pull/3280#issuecomment-895409304 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[jira] [Work logged] (HADOOP-17825) Add BuiltInGzipCompressor

2021-08-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17825?focusedWorklogId=635974&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-635974 ] ASF GitHub Bot logged work on HADOOP-17825: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #3250: HADOOP-17825. Add BuiltInGzipCompressor

2021-08-09 Thread GitBox
hadoop-yetus removed a comment on pull request #3250: URL: https://github.com/apache/hadoop/pull/3250#issuecomment-893970892 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [hadoop] hadoop-yetus commented on pull request #2908: HDFS-15976. Make mkdtemp cross platform

2021-08-09 Thread GitBox
hadoop-yetus commented on pull request #2908: URL: https://github.com/apache/hadoop/pull/2908#issuecomment-895398121 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[jira] [Work logged] (HADOOP-15327) Upgrade MR ShuffleHandler to use Netty4

2021-08-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-15327?focusedWorklogId=635956&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-635956 ] ASF GitHub Bot logged work on HADOOP-15327: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus commented on pull request #3259: HADOOP-15327. Upgrade MR ShuffleHandler to use Netty4

2021-08-09 Thread GitBox
hadoop-yetus commented on pull request #3259: URL: https://github.com/apache/hadoop/pull/3259#issuecomment-895393918 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] hadoop-yetus commented on pull request #3283: YARN-10814 3.2 backport

2021-08-09 Thread GitBox
hadoop-yetus commented on pull request #3283: URL: https://github.com/apache/hadoop/pull/3283#issuecomment-895366615 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[jira] [Work logged] (HADOOP-17825) Add BuiltInGzipCompressor

2021-08-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17825?focusedWorklogId=635936&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-635936 ] ASF GitHub Bot logged work on HADOOP-17825: --- Author: ASF GitHub B

[GitHub] [hadoop] viirya commented on pull request #3250: HADOOP-17825. Add BuiltInGzipCompressor

2021-08-09 Thread GitBox
viirya commented on pull request #3250: URL: https://github.com/apache/hadoop/pull/3250#issuecomment-895365571 Thanks @steveloughran. I also don't think there's risk here. The compressing here basically takes the buffers given by the caller, instead of allocating by itself. -- T

[jira] [Assigned] (HADOOP-17156) Clear abfs readahead requests on stream close

2021-08-09 Thread Mukund Thakur (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukund Thakur reassigned HADOOP-17156: -- Assignee: Mukund Thakur > Clear abfs readahead requests on stream close > --

[GitHub] [hadoop] hadoop-yetus commented on pull request #3281: HADOOP-17863. Improve logging on ABFS network errors

2021-08-09 Thread GitBox
hadoop-yetus commented on pull request #3281: URL: https://github.com/apache/hadoop/pull/3281#issuecomment-895272713 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] hadoop-yetus commented on pull request #3281: HADOOP-17863. Improve logging on ABFS network errors

2021-08-09 Thread GitBox
hadoop-yetus commented on pull request #3281: URL: https://github.com/apache/hadoop/pull/3281#issuecomment-895269345 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] tomicooler opened a new pull request #3283: YARN-10814 3.2 backport

2021-08-09 Thread GitBox
tomicooler opened a new pull request #3283: URL: https://github.com/apache/hadoop/pull/3283 ## NOTICE Please create an issue in ASF JIRA before opening a pull request, and you need to set the title of the pull request which starts with the corresponding JIRA issue number. (e.g.

[GitHub] [hadoop] hadoop-yetus commented on pull request #2908: HDFS-15976. Make mkdtemp cross platform

2021-08-09 Thread GitBox
hadoop-yetus commented on pull request #2908: URL: https://github.com/apache/hadoop/pull/2908#issuecomment-895262990 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[jira] [Commented] (HADOOP-16752) ABFS: test failure testLastModifiedTime()

2021-08-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17396049#comment-17396049 ] Steve Loughran commented on HADOOP-16752: - sorry, typo, meant 3.3.2. it's only

[GitHub] [hadoop] tomicooler opened a new pull request #3282: YARN-10814 3.3 backport

2021-08-09 Thread GitBox
tomicooler opened a new pull request #3282: URL: https://github.com/apache/hadoop/pull/3282 …pty. The rest endpoint would be unusable with an empty secret file (throwing IllegalArgumentExceptions). Any IO error would have resulted in the same fallback path. Change-Id:

[jira] [Updated] (HADOOP-16752) ABFS: test failure testLastModifiedTime()

2021-08-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-16752: Target Version/s: 3.3.2 (was: 3.2.2) > ABFS: test failure testLastModifiedTime() > -

[jira] [Updated] (HADOOP-17842) S3a parquet reads slow with Spark on Kubernetes (EKS)

2021-08-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17842: Summary: S3a parquet reads slow with Spark on Kubernetes (EKS) (was: HADOOP-AWS with Spa

[jira] [Resolved] (HADOOP-17842) HADOOP-AWS with Spark on Kubernetes (EKS)

2021-08-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17842. - Resolution: Works for Me > HADOOP-AWS with Spark on Kubernetes (EKS) >

[jira] [Moved] (HADOOP-17842) HADOOP-AWS with Spark on Kubernetes (EKS)

2021-08-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran moved HDFS-16156 to HADOOP-17842: Component/s: (was: fs/s3) fs/s3

[jira] [Work logged] (HADOOP-17825) Add BuiltInGzipCompressor

2021-08-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17825?focusedWorklogId=635864&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-635864 ] ASF GitHub Bot logged work on HADOOP-17825: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on pull request #3250: HADOOP-17825. Add BuiltInGzipCompressor

2021-08-09 Thread GitBox
steveloughran commented on pull request #3250: URL: https://github.com/apache/hadoop/pull/3250#issuecomment-895217443 One concern here: anything we have to worry about from a security perspective? That is, if someone sends in something with an invalid range, does that trigger allocation of

[jira] [Work logged] (HADOOP-17198) Support S3 Access Points

2021-08-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17198?focusedWorklogId=635858&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-635858 ] ASF GitHub Bot logged work on HADOOP-17198: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on pull request #3260: HADOOP-17198 Support S3 AccessPoint

2021-08-09 Thread GitBox
steveloughran commented on pull request #3260: URL: https://github.com/apache/hadoop/pull/3260#issuecomment-895211561 > Shouldn't we just throw an exception during initialization if AccessPoint is set and S3Guard is enabled to tell the users that AccessPoints are incompatible with S3Guard?

[jira] [Work logged] (HADOOP-17198) Support S3 Access Points

2021-08-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17198?focusedWorklogId=635857&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-635857 ] ASF GitHub Bot logged work on HADOOP-17198: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on a change in pull request #3260: HADOOP-17198 Support S3 AccessPoint

2021-08-09 Thread GitBox
steveloughran commented on a change in pull request #3260: URL: https://github.com/apache/hadoop/pull/3260#discussion_r685178675 ## File path: hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java ## @@ -2570,6 +2614,11 @@ protected S3ListResult con

[GitHub] [hadoop] steveloughran commented on pull request #3281: HADOOP-17863. Improve logging on ABFS network errors

2021-08-09 Thread GitBox
steveloughran commented on pull request #3281: URL: https://github.com/apache/hadoop/pull/3281#issuecomment-895208357 test rerun on its own is fine -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [hadoop] steveloughran commented on pull request #3281: HADOOP-17863. Improve logging on ABFS network errors

2021-08-09 Thread GitBox
steveloughran commented on pull request #3281: URL: https://github.com/apache/hadoop/pull/3281#issuecomment-895204265 ``` mvn verify -Dparallel-tests=abfs -DtestsThreadCount=8 ``` one failure ``` [ERROR] testDeleteNonExistentFile(org.apache.hadoop.fs.azurebfs.ITestAz

[GitHub] [hadoop] steveloughran opened a new pull request #3281: HADOOP-17863. Improve logging on ABFS network errors

2021-08-09 Thread GitBox
steveloughran opened a new pull request #3281: URL: https://github.com/apache/hadoop/pull/3281 * include masked URL and nested exceptions in wrapping IOEs * wrapping IOEs now path IOEs * avoid scaring people by printing full stack trace on a recoverable retry loop, logging su

[jira] [Assigned] (HADOOP-17836) ABFS connection reset reporting excessively noisy

2021-08-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-17836: --- Assignee: Steve Loughran > ABFS connection reset reporting excessively noisy > ---

[jira] [Updated] (HADOOP-17836) ABFS connection reset reporting excessively noisy

2021-08-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17836: Summary: ABFS connection reset reporting excessively noisy (was: ABFS connection reset o

[jira] [Updated] (HADOOP-17836) ABFS connection reset reporting excessively noisy

2021-08-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17836: Priority: Minor (was: Major) > ABFS connection reset reporting excessively noisy > -

[jira] [Commented] (HADOOP-17836) ABFS connection reset on large (10GB) download

2021-08-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17396027#comment-17396027 ] Steve Loughran commented on HADOOP-17836: - it't not error, just noisy. Will fix

[GitHub] [hadoop] hadoop-yetus commented on pull request #2908: HDFS-15976. Make mkdtemp cross platform

2021-08-09 Thread GitBox
hadoop-yetus commented on pull request #2908: URL: https://github.com/apache/hadoop/pull/2908#issuecomment-895139767 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[GitHub] [hadoop] hadoop-yetus commented on pull request #3279: YARN-10693. Add document for YARN-10623 auto refresh queue conf in CS…

2021-08-09 Thread GitBox
hadoop-yetus commented on pull request #3279: URL: https://github.com/apache/hadoop/pull/3279#issuecomment-895111981 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[GitHub] [hadoop] zhuqi-lucas opened a new pull request #3279: YARN-10693. Add document for YARN-10623 auto refresh queue conf in CS…

2021-08-09 Thread GitBox
zhuqi-lucas opened a new pull request #3279: URL: https://github.com/apache/hadoop/pull/3279 …. Contributed by Qi Zhu. ## NOTICE Please create an issue in ASF JIRA before opening a pull request, and you need to set the title of the pull request which starts with the corre

[jira] [Work logged] (HADOOP-17841) Remove ListenerHandle from Hadoop registry

2021-08-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17841?focusedWorklogId=635809&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-635809 ] ASF GitHub Bot logged work on HADOOP-17841: --- Author: ASF GitHub B

[jira] [Updated] (HADOOP-17841) Remove ListenerHandle from Hadoop registry

2021-08-09 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HADOOP-17841: - Fix Version/s: 3.4.0 Resolution: Fixed Status: Resolved (was: Patch A

[GitHub] [hadoop] jojochuang merged pull request #3278: HADOOP-17841. Remove ListenerHandle from Hadoop registry

2021-08-09 Thread GitBox
jojochuang merged pull request #3278: URL: https://github.com/apache/hadoop/pull/3278 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issue

[GitHub] [hadoop] hadoop-yetus commented on pull request #2908: HDFS-15976. Make mkdtemp cross platform

2021-08-09 Thread GitBox
hadoop-yetus commented on pull request #2908: URL: https://github.com/apache/hadoop/pull/2908#issuecomment-895033947 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o