[GitHub] [hadoop] hadoop-yetus commented on pull request #2639: HDFS-15785. Datanode to support using DNS to resolve nameservices to IP addresses to get list of namenodes.

2021-06-22 Thread GitBox
hadoop-yetus commented on pull request #2639: URL: https://github.com/apache/hadoop/pull/2639#issuecomment-866548387 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Updated] (HADOOP-17773) Avoid using zookeeper deprecated API and classes

2021-06-22 Thread Surendra Singh Lilhore (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Surendra Singh Lilhore updated HADOOP-17773: Summary: Avoid using zookeeper deprecated API and classes (was: Avoid

[jira] [Created] (HADOOP-17773) Avoid using zookeeper deprecated API in hadoop code

2021-06-22 Thread Surendra Singh Lilhore (Jira)
Surendra Singh Lilhore created HADOOP-17773: --- Summary: Avoid using zookeeper deprecated API in hadoop code Key: HADOOP-17773 URL: https://issues.apache.org/jira/browse/HADOOP-17773 Project:

[jira] [Updated] (HADOOP-17773) Avoid using zookeeper deprecated API

2021-06-22 Thread Surendra Singh Lilhore (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Surendra Singh Lilhore updated HADOOP-17773: Summary: Avoid using zookeeper deprecated API (was: Avoid using

[jira] [Commented] (HADOOP-15327) Upgrade MR ShuffleHandler to use Netty4

2021-06-22 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-15327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17367842#comment-17367842 ] Wei-Chiu Chuang commented on HADOOP-15327: -- This is an excerpt of the error: {noformat}

[GitHub] [hadoop] jojochuang merged pull request #3127: HDFS-16082. Atomic operations on exceptionsSinceLastBalance and failedTimesSinceLastSuccessfulBalance in Balancer

2021-06-22 Thread GitBox
jojochuang merged pull request #3127: URL: https://github.com/apache/hadoop/pull/3127 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[jira] [Work logged] (HADOOP-17749) Remove lock contention in SelectorPool of SocketIOWithTimeout

2021-06-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17749?focusedWorklogId=613786=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-613786 ] ASF GitHub Bot logged work on HADOOP-17749: --- Author: ASF GitHub Bot

[GitHub] [hadoop] liangxs commented on pull request #3080: HADOOP-17749. Remove lock contention in SelectorPool of SocketIOWithTimeout

2021-06-22 Thread GitBox
liangxs commented on pull request #3080: URL: https://github.com/apache/hadoop/pull/3080#issuecomment-866478672 @ferhui Thanks for your suggestion, I added unit test. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[jira] [Work logged] (HADOOP-17769) Upgrade JUnit to 4.13.2

2021-06-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17769?focusedWorklogId=613783=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-613783 ] ASF GitHub Bot logged work on HADOOP-17769: --- Author: ASF GitHub Bot

[GitHub] [hadoop] ayushtkn commented on pull request #3130: HADOOP-17769. Upgrade JUnit to 4.13.2. fixes TestBlockRecovery

2021-06-22 Thread GitBox
ayushtkn commented on pull request #3130: URL: https://github.com/apache/hadoop/pull/3130#issuecomment-866473863 @amahussein can you update this file, as suggested on 2.10 PR https://github.com/apache/hadoop/blob/10b79a26fe0677b266acf237e8458e93743424a6/LICENSE-binary#L505 -- This is

[GitHub] [hadoop] zhuxiangyi commented on pull request #3063: HDFS-16043. HDFS: Delete performance optimization

2021-06-22 Thread GitBox
zhuxiangyi commented on pull request #3063: URL: https://github.com/apache/hadoop/pull/3063#issuecomment-866461745 > Reason Tests > Failed junit tests hadoop.hdfs.server.namenode.TestDecommissioningStatus > hadoop.hdfs.server.namenode.TestDecommissioningStatusWithBackoffMonitor

[GitHub] [hadoop] tomscut opened a new pull request #3134: HDFS-16085. Move the getPermissionChecker out of the read lock

2021-06-22 Thread GitBox
tomscut opened a new pull request #3134: URL: https://github.com/apache/hadoop/pull/3134 JIRA: [HDFS-16085](https://issues.apache.org/jira/browse/HDFS-16085) Move the getPermissionChecker out of the read lock in NamenodeFsck#getBlockLocations() since the operation does not need to

[jira] [Updated] (HADOOP-16963) HADOOP-16582 changed mkdirs() behavior

2021-06-22 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HADOOP-16963: - Target Version/s: 3.4.0, 3.3.2 Add target version 3.3.2 so I don't forget it in the

[jira] [Commented] (HADOOP-16458) LocatedFileStatusFetcher scans failing intermittently against S3 store

2021-06-22 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17367798#comment-17367798 ] Wei-Chiu Chuang commented on HADOOP-16458: -- Looks like it caused a small regression in Hive.

[jira] [Work logged] (HADOOP-17769) Upgrade JUnit to 4.13.2

2021-06-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17769?focusedWorklogId=613774=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-613774 ] ASF GitHub Bot logged work on HADOOP-17769: --- Author: ASF GitHub Bot

[GitHub] [hadoop] jojochuang commented on pull request #3131: HADOOP-17769. Upgrade JUnit to 4.13.2. fixes TestBlockRecovery

2021-06-22 Thread GitBox
jojochuang commented on pull request #3131: URL: https://github.com/apache/hadoop/pull/3131#issuecomment-866449353 Oh yeah sorry I wasn't aware this is for branch-2.10. No issue now. +1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[jira] [Resolved] (HADOOP-17762) branch-2.10 daily build fails to pull latest changes

2021-06-22 Thread Akira Ajisaka (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka resolved HADOOP-17762. Resolution: Duplicate Probably this issue has been fixed by INFRA-22020. > branch-2.10

[GitHub] [hadoop] tomscut commented on a change in pull request #3117: HDFS-16076. Avoid using slow DataNodes for reading by sorting locations

2021-06-22 Thread GitBox
tomscut commented on a change in pull request #3117: URL: https://github.com/apache/hadoop/pull/3117#discussion_r656688215 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestSortLocatedBlock.java ## @@ -126,11

[jira] [Work logged] (HADOOP-17769) Upgrade JUnit to 4.13.2

2021-06-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17769?focusedWorklogId=613769=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-613769 ] ASF GitHub Bot logged work on HADOOP-17769: --- Author: ASF GitHub Bot

[GitHub] [hadoop] aajisaka commented on pull request #3131: HADOOP-17769. Upgrade JUnit to 4.13.2. fixes TestBlockRecovery

2021-06-22 Thread GitBox
aajisaka commented on pull request #3131: URL: https://github.com/apache/hadoop/pull/3131#issuecomment-866439534 I think the file exists in trunk but not in branch-2.10. https://github.com/apache/hadoop/blob/10b79a26fe0677b266acf237e8458e93743424a6/LICENSE-binary#L505 -- This is an

[jira] [Work logged] (HADOOP-17766) CI for Debian 10

2021-06-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17766?focusedWorklogId=613746=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-613746 ] ASF GitHub Bot logged work on HADOOP-17766: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #3129: HADOOP-17766. CI for Debian 10

2021-06-22 Thread GitBox
hadoop-yetus commented on pull request #3129: URL: https://github.com/apache/hadoop/pull/3129#issuecomment-866397240 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] hadoop-yetus commented on pull request #2919: HDFS-15979. Move within EZ fails and cannot remove nested EZs. Contriā€¦

2021-06-22 Thread GitBox
hadoop-yetus commented on pull request #2919: URL: https://github.com/apache/hadoop/pull/2919#issuecomment-866370482 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Created] (HADOOP-17772) ABFS: delete() should have timeout option

2021-06-22 Thread Zhuangyu Han (Jira)
Zhuangyu Han created HADOOP-17772: - Summary: ABFS: delete() should have timeout option Key: HADOOP-17772 URL: https://issues.apache.org/jira/browse/HADOOP-17772 Project: Hadoop Common Issue

[jira] [Work logged] (HADOOP-17766) CI for Debian 10

2021-06-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17766?focusedWorklogId=613688=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-613688 ] ASF GitHub Bot logged work on HADOOP-17766: --- Author: ASF GitHub Bot

[jira] [Work logged] (HADOOP-17766) CI for Debian 10

2021-06-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17766?focusedWorklogId=613687=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-613687 ] ASF GitHub Bot logged work on HADOOP-17766: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #3129: HADOOP-17766. CI for Debian 10

2021-06-22 Thread GitBox
hadoop-yetus commented on pull request #3129: URL: https://github.com/apache/hadoop/pull/3129#issuecomment-866312583 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at

[GitHub] [hadoop] hadoop-yetus commented on pull request #3129: HADOOP-17766. CI for Debian 10

2021-06-22 Thread GitBox
hadoop-yetus commented on pull request #3129: URL: https://github.com/apache/hadoop/pull/3129#issuecomment-866312403 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Work logged] (HADOOP-13887) Encrypt S3A data client-side with AWS SDK (S3-CSE)

2021-06-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13887?focusedWorklogId=613651=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-613651 ] ASF GitHub Bot logged work on HADOOP-13887: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #2706: HADOOP-13887. Support S3 client side encryption (S3-CSE) using AWS-SDK

2021-06-22 Thread GitBox
hadoop-yetus commented on pull request #2706: URL: https://github.com/apache/hadoop/pull/2706#issuecomment-866239357 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] tasanuma commented on a change in pull request #3117: HDFS-16076. Avoid using slow DataNodes for reading by sorting locations

2021-06-22 Thread GitBox
tasanuma commented on a change in pull request #3117: URL: https://github.com/apache/hadoop/pull/3117#discussion_r656413524 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestSortLocatedBlock.java ## @@ -126,11

[jira] [Work logged] (HADOOP-17771) S3AFS creation fails without region set in ~/.aws/config

2021-06-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17771?focusedWorklogId=613609=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-613609 ] ASF GitHub Bot logged work on HADOOP-17771: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on pull request #3133: HADOOP-17771. S3AFS creation fails without region set in ~/.aws/config.

2021-06-22 Thread GitBox
steveloughran commented on pull request #3133: URL: https://github.com/apache/hadoop/pull/3133#issuecomment-866183472 tested with `-Dparallel-tests -DtestsThreadCount=7 -Dmarkers=keep -Dscale -Ds3guard -Ddynamo` transient read buffer underflow failure; one extra S3Guard write than

[jira] [Work logged] (HADOOP-17766) CI for Debian 10

2021-06-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17766?focusedWorklogId=613607=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-613607 ] ASF GitHub Bot logged work on HADOOP-17766: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #3129: HADOOP-17766. CI for Debian 10

2021-06-22 Thread GitBox
hadoop-yetus commented on pull request #3129: URL: https://github.com/apache/hadoop/pull/3129#issuecomment-866177000 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at

[jira] [Work logged] (HADOOP-17766) CI for Debian 10

2021-06-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17766?focusedWorklogId=613605=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-613605 ] ASF GitHub Bot logged work on HADOOP-17766: --- Author: ASF GitHub Bot

[jira] [Work logged] (HADOOP-17766) CI for Debian 10

2021-06-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17766?focusedWorklogId=613606=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-613606 ] ASF GitHub Bot logged work on HADOOP-17766: --- Author: ASF GitHub Bot

[GitHub] [hadoop] GauthamBanasandra commented on a change in pull request #3129: HADOOP-17766. CI for Debian 10

2021-06-22 Thread GitBox
GauthamBanasandra commented on a change in pull request #3129: URL: https://github.com/apache/hadoop/pull/3129#discussion_r656424841 ## File path: dev-support/jenkins.sh ## @@ -0,0 +1,205 @@ +#!/usr/bin/env bash + +# Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [hadoop] GauthamBanasandra commented on a change in pull request #3129: HADOOP-17766. CI for Debian 10

2021-06-22 Thread GitBox
GauthamBanasandra commented on a change in pull request #3129: URL: https://github.com/apache/hadoop/pull/3129#discussion_r656424632 ## File path: dev-support/Jenkinsfile ## @@ -57,7 +56,12 @@ pipeline { } } -stage ('precommit-run') { +

[jira] [Work logged] (HADOOP-17766) CI for Debian 10

2021-06-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17766?focusedWorklogId=613597=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-613597 ] ASF GitHub Bot logged work on HADOOP-17766: --- Author: ASF GitHub Bot

[GitHub] [hadoop] goiri commented on a change in pull request #3129: HADOOP-17766. CI for Debian 10

2021-06-22 Thread GitBox
goiri commented on a change in pull request #3129: URL: https://github.com/apache/hadoop/pull/3129#discussion_r656407279 ## File path: dev-support/jenkins.sh ## @@ -0,0 +1,205 @@ +#!/usr/bin/env bash + +# Licensed to the Apache Software Foundation (ASF) under one +# or more

[jira] [Work logged] (HADOOP-17766) CI for Debian 10

2021-06-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17766?focusedWorklogId=613596=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-613596 ] ASF GitHub Bot logged work on HADOOP-17766: --- Author: ASF GitHub Bot

[GitHub] [hadoop] GauthamBanasandra commented on pull request #3129: HADOOP-17766. CI for Debian 10

2021-06-22 Thread GitBox
GauthamBanasandra commented on pull request #3129: URL: https://github.com/apache/hadoop/pull/3129#issuecomment-866154858 Jenkins has run for Debian 10 since this PR contains C++ change and platform changes -

[jira] [Work logged] (HADOOP-17749) Remove lock contention in SelectorPool of SocketIOWithTimeout

2021-06-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17749?focusedWorklogId=613563=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-613563 ] ASF GitHub Bot logged work on HADOOP-17749: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #3080: HADOOP-17749. Remove lock contention in SelectorPool of SocketIOWithTimeout

2021-06-22 Thread GitBox
hadoop-yetus commented on pull request #3080: URL: https://github.com/apache/hadoop/pull/3080#issuecomment-866101260 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Updated] (HADOOP-17584) s3a magic committer may commit more data

2021-06-22 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17584: Parent: HADOOP-17566 Issue Type: Sub-task (was: Bug) > s3a magic committer may

[jira] [Updated] (HADOOP-17500) S3A doesn't calculate Content-MD5 on uploads

2021-06-22 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17500: Parent: HADOOP-17566 Issue Type: Sub-task (was: Bug) > S3A doesn't calculate

[jira] [Commented] (HADOOP-17500) S3A doesn't calculate Content-MD5 on uploads

2021-06-22 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17367433#comment-17367433 ] Steve Loughran commented on HADOOP-17500: - have you made any progress here? Even if its not

[jira] [Commented] (HADOOP-17755) EOF reached error reading ORC file on S3A

2021-06-22 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17367430#comment-17367430 ] Steve Loughran commented on HADOOP-17755: - Still awaiting the requested information. If you

[jira] [Work logged] (HADOOP-13887) Encrypt S3A data client-side with AWS SDK (S3-CSE)

2021-06-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13887?focusedWorklogId=613521=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-613521 ] ASF GitHub Bot logged work on HADOOP-13887: --- Author: ASF GitHub Bot

[GitHub] [hadoop] mehakmeet commented on pull request #2706: HADOOP-13887. Support S3 client side encryption (S3-CSE) using AWS-SDK

2021-06-22 Thread GitBox
mehakmeet commented on pull request #2706: URL: https://github.com/apache/hadoop/pull/2706#issuecomment-866043189 Seems like git pull, didn't work for me, just before pushing, still have some conflicts, I'll rebase once more by fetching and applying. -- This is an automated message

[jira] [Work logged] (HADOOP-13887) Encrypt S3A data client-side with AWS SDK (S3-CSE)

2021-06-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13887?focusedWorklogId=613517=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-613517 ] ASF GitHub Bot logged work on HADOOP-13887: --- Author: ASF GitHub Bot

[GitHub] [hadoop] mehakmeet commented on pull request #2706: HADOOP-13887. Support S3 client side encryption (S3-CSE) using AWS-SDK

2021-06-22 Thread GitBox
mehakmeet commented on pull request #2706: URL: https://github.com/apache/hadoop/pull/2706#issuecomment-866032252 Force pushing due to merge conflicts. - **Multipart upload:** - Added the serial uploads by setting the "Active upload blocks" equal to 1, when CSE is

[jira] [Work logged] (HADOOP-17769) Upgrade JUnit to 4.13.2

2021-06-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17769?focusedWorklogId=613513=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-613513 ] ASF GitHub Bot logged work on HADOOP-17769: --- Author: ASF GitHub Bot

[GitHub] [hadoop] amahussein commented on pull request #3131: HADOOP-17769. Upgrade JUnit to 4.13.2. fixes TestBlockRecovery

2021-06-22 Thread GitBox
amahussein commented on pull request #3131: URL: https://github.com/apache/hadoop/pull/3131#issuecomment-866027772 > Tried the mvn dependency tree on trunk, it looks fine. > @jojochuang do you mean this file? >

[jira] [Work logged] (HADOOP-17771) S3AFS creation fails without region set in ~/.aws/config

2021-06-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17771?focusedWorklogId=613489=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-613489 ] ASF GitHub Bot logged work on HADOOP-17771: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #3133: HADOOP-17771. S3AFS creation fails without region set in ~/.aws/config.

2021-06-22 Thread GitBox
hadoop-yetus commented on pull request #3133: URL: https://github.com/apache/hadoop/pull/3133#issuecomment-865993418 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Work logged] (HADOOP-17749) Remove lock contention in SelectorPool of SocketIOWithTimeout

2021-06-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17749?focusedWorklogId=613486=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-613486 ] ASF GitHub Bot logged work on HADOOP-17749: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #3080: HADOOP-17749. Remove lock contention in SelectorPool of SocketIOWithTimeout

2021-06-22 Thread GitBox
hadoop-yetus commented on pull request #3080: URL: https://github.com/apache/hadoop/pull/3080#issuecomment-865990719 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Work logged] (HADOOP-17749) Remove lock contention in SelectorPool of SocketIOWithTimeout

2021-06-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17749?focusedWorklogId=613485=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-613485 ] ASF GitHub Bot logged work on HADOOP-17749: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #3080: HADOOP-17749. Remove lock contention in SelectorPool of SocketIOWithTimeout

2021-06-22 Thread GitBox
hadoop-yetus commented on pull request #3080: URL: https://github.com/apache/hadoop/pull/3080#issuecomment-865989933 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] hadoop-yetus commented on pull request #1867: HADOOP-16983. Update ADLS client credential creation docs.

2021-06-22 Thread GitBox
hadoop-yetus commented on pull request #1867: URL: https://github.com/apache/hadoop/pull/1867#issuecomment-865973792 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Work logged] (HADOOP-17771) S3AFS creation fails without region set in ~/.aws/config

2021-06-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17771?focusedWorklogId=613474=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-613474 ] ASF GitHub Bot logged work on HADOOP-17771: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on pull request #3133: HADOOP-17771. S3AFS creation fails without region set in ~/.aws/config.

2021-06-22 Thread GitBox
steveloughran commented on pull request #3133: URL: https://github.com/apache/hadoop/pull/3133#issuecomment-865965433 Test run with `-Dparallel-tests -DtestsThreadCount=7 -Dmarkers=keep` -no failures! rerunning with dynamo & scale -- This is an automated message from the Apache

[jira] [Work logged] (HADOOP-17764) S3AInputStream read does not re-open the input stream on the second read retry attempt

2021-06-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17764?focusedWorklogId=613473=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-613473 ] ASF GitHub Bot logged work on HADOOP-17764: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on pull request #3109: HADOOP-17764. S3AInputStream read does not re-open the input stream on the second read retry attempt

2021-06-22 Thread GitBox
steveloughran commented on pull request #3109: URL: https://github.com/apache/hadoop/pull/3109#issuecomment-865964446 thanks for running the tests. 1. Can you run the entire suite (I.e in the hadoop-aws module, run `mvn verify ...` and leave out the -Dtest and -Dit.test bits 2.

[jira] [Work logged] (HADOOP-17771) S3AFS creation fails without region set in ~/.aws/config

2021-06-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17771?focusedWorklogId=613460=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-613460 ] ASF GitHub Bot logged work on HADOOP-17771: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on pull request #3133: HADOOP-17771. S3AFS creation fails without region set in ~/.aws/config.

2021-06-22 Thread GitBox
steveloughran commented on pull request #3133: URL: https://github.com/apache/hadoop/pull/3133#issuecomment-865943337 + No new tests here. The only way to test would be to destroy ~/.aws/config and & skip if the env vars/syssprops resolved or running in EC2. If we could set the resolve

[jira] [Work logged] (HADOOP-17771) S3AFS creation fails without region set in ~/.aws/config

2021-06-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17771?focusedWorklogId=613456=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-613456 ] ASF GitHub Bot logged work on HADOOP-17771: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on pull request #3133: HADOOP-17771. S3AFS creation fails without region set in ~/.aws/config.

2021-06-22 Thread GitBox
steveloughran commented on pull request #3133: URL: https://github.com/apache/hadoop/pull/3133#issuecomment-865941502 ## `cloudstore regions` This verifies that the local system is set up to fail without the patch. ``` > bin/hadoop jar $CLOUDSTORE regions

[jira] [Work logged] (HADOOP-17771) S3AFS creation fails without region set in ~/.aws/config

2021-06-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17771?focusedWorklogId=613454=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-613454 ] ASF GitHub Bot logged work on HADOOP-17771: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on pull request #3133: HADOOP-17771. S3AFS creation fails without region set in ~/.aws/config.

2021-06-22 Thread GitBox
steveloughran commented on pull request #3133: URL: https://github.com/apache/hadoop/pull/3133#issuecomment-865940542 Testing in progress, in a setup where * the failure mode is replicated * fs.s3a.endpoint is not set globally or for the test bucket. -- This is an automated

[jira] [Updated] (HADOOP-17771) S3AFS creation fails without region set in ~/.aws/config

2021-06-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-17771: Labels: pull-request-available (was: ) > S3AFS creation fails without region set in

[jira] [Work logged] (HADOOP-17771) S3AFS creation fails without region set in ~/.aws/config

2021-06-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17771?focusedWorklogId=613452=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-613452 ] ASF GitHub Bot logged work on HADOOP-17771: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran opened a new pull request #3133: HADOOP-17771. S3AFS creation fails without region set in ~/.aws/config.

2021-06-22 Thread GitBox
steveloughran opened a new pull request #3133: URL: https://github.com/apache/hadoop/pull/3133 Contributed by Steve Loughran. Change-Id: I94284178d27a48947e7c0942a7c8565379de7e9b -- This is an automated message from the Apache Git Service. To respond to the message, please log

[jira] [Updated] (HADOOP-17771) S3AFS creation fails without region set in ~/.aws/config

2021-06-22 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17771: Summary: S3AFS creation fails without region set in ~/.aws/config (was: S3AFS creation

[jira] [Work logged] (HADOOP-17764) S3AInputStream read does not re-open the input stream on the second read retry attempt

2021-06-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17764?focusedWorklogId=613438=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-613438 ] ASF GitHub Bot logged work on HADOOP-17764: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #3132: HADOOP-17764. S3AInputStream read does not re-open the input stream on the second read retry attempt

2021-06-22 Thread GitBox
hadoop-yetus commented on pull request #3132: URL: https://github.com/apache/hadoop/pull/3132#issuecomment-865915522 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Commented] (HADOOP-17771) S3AFS creation fails without region set ~/.aws/config

2021-06-22 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17367229#comment-17367229 ] Steve Loughran commented on HADOOP-17771: - new cloudstore command "regions" to see how your

[jira] [Comment Edited] (HADOOP-17771) S3AFS creation fails without region set ~/.aws/config

2021-06-22 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17367222#comment-17367222 ] Steve Loughran edited comment on HADOOP-17771 at 6/22/21, 10:46 AM:

[jira] [Commented] (HADOOP-17771) S3AFS creation fails without region set ~/.aws/config

2021-06-22 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17367222#comment-17367222 ] Steve Loughran commented on HADOOP-17771: - +1 probably also the DynamoDB binding of S3Guard >

[jira] [Commented] (HADOOP-17771) S3AFS creation fails without region set ~/.aws/config

2021-06-22 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17367221#comment-17367221 ] Steve Loughran commented on HADOOP-17771: - Note: the same region binding issue surfaces for STS

[GitHub] [hadoop] zhuxiangyi commented on a change in pull request #3086: HDFS-16039. RBF: Some indicators of RBFMetrics count inaccurately

2021-06-22 Thread GitBox
zhuxiangyi commented on a change in pull request #3086: URL: https://github.com/apache/hadoop/pull/3086#discussion_r650121325 ## File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/metrics/RBFMetrics.java ## @@ -165,11 +172,46

[jira] [Work logged] (HADOOP-17764) S3AInputStream read does not re-open the input stream on the second read retry attempt

2021-06-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17764?focusedWorklogId=613404=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-613404 ] ASF GitHub Bot logged work on HADOOP-17764: --- Author: ASF GitHub Bot

[jira] [Work logged] (HADOOP-17764) S3AInputStream read does not re-open the input stream on the second read retry attempt

2021-06-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17764?focusedWorklogId=613403=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-613403 ] ASF GitHub Bot logged work on HADOOP-17764: --- Author: ASF GitHub Bot

[GitHub] [hadoop] majdyz edited a comment on pull request #3109: HADOOP-17764. S3AInputStream read does not re-open the input stream on the second read retry attempt

2021-06-22 Thread GitBox
majdyz edited a comment on pull request #3109: URL: https://github.com/apache/hadoop/pull/3109#issuecomment-865863147 I got an error that requires me to provide DynamoDB table to run the S3A test, is this expected? Is there any workaround to not use it? -- This is an automated

[jira] [Work logged] (HADOOP-17764) S3AInputStream read does not re-open the input stream on the second read retry attempt

2021-06-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17764?focusedWorklogId=613402=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-613402 ] ASF GitHub Bot logged work on HADOOP-17764: --- Author: ASF GitHub Bot

[jira] [Work logged] (HADOOP-17764) S3AInputStream read does not re-open the input stream on the second read retry attempt

2021-06-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17764?focusedWorklogId=613401=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-613401 ] ASF GitHub Bot logged work on HADOOP-17764: --- Author: ASF GitHub Bot

[GitHub] [hadoop] majdyz edited a comment on pull request #3109: HADOOP-17764. S3AInputStream read does not re-open the input stream on the second read retry attempt

2021-06-22 Thread GitBox
majdyz edited a comment on pull request #3109: URL: https://github.com/apache/hadoop/pull/3109#issuecomment-865863147 @steveloughran I got an error that requires me to provide DynamoDB table to run the S3A test, is this expected? Is there any workaround to not use it? -- This is an

[GitHub] [hadoop] majdyz commented on pull request #3109: HADOOP-17764. S3AInputStream read does not re-open the input stream on the second read retry attempt

2021-06-22 Thread GitBox
majdyz commented on pull request #3109: URL: https://github.com/apache/hadoop/pull/3109#issuecomment-865863147 @steveloughran I got an error that requires me to provide DynamoDB table to run the S3A test, is this expected? Is there a way to exclude this tests ? -- This is an

[jira] [Work logged] (HADOOP-17764) S3AInputStream read does not re-open the input stream on the second read retry attempt

2021-06-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17764?focusedWorklogId=613393=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-613393 ] ASF GitHub Bot logged work on HADOOP-17764: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #3109: HADOOP-17764. S3AInputStream read does not re-open the input stream on the second read retry attempt

2021-06-22 Thread GitBox
hadoop-yetus commented on pull request #3109: URL: https://github.com/apache/hadoop/pull/3109#issuecomment-865842982 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Commented] (HADOOP-17771) S3AFS creation fails without region set ~/.aws/config

2021-06-22 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17367173#comment-17367173 ] Steve Loughran commented on HADOOP-17771: - Example of a good configuration file {code} > cat

[jira] [Updated] (HADOOP-17771) S3AFS creation fails without region set ~/.aws/config

2021-06-22 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17771: Description: If you don't have {{fs.s3a.endpoint}} set and lack a region set in env var

[jira] [Commented] (HADOOP-17771) S3AFS creation fails without region set ~/.aws/config

2021-06-22 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17367170#comment-17367170 ] Steve Loughran commented on HADOOP-17771: - Stack {code} 2021-06-22 10:02:18,451 [main] INFO

[jira] [Created] (HADOOP-17771) S3AFS creation fails without region set ~/.aws/config

2021-06-22 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17771: --- Summary: S3AFS creation fails without region set ~/.aws/config Key: HADOOP-17771 URL: https://issues.apache.org/jira/browse/HADOOP-17771 Project: Hadoop Common

[jira] [Work logged] (HADOOP-17767) ABFS: Improve test scripts

2021-06-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17767?focusedWorklogId=613357=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-613357 ] ASF GitHub Bot logged work on HADOOP-17767: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #3124: HADOOP-17767. ABFS: Updates test scripts

2021-06-22 Thread GitBox
hadoop-yetus commented on pull request #3124: URL: https://github.com/apache/hadoop/pull/3124#issuecomment-865735062 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Work logged] (HADOOP-17764) S3AInputStream read does not re-open the input stream on the second read retry attempt

2021-06-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17764?focusedWorklogId=613356=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-613356 ] ASF GitHub Bot logged work on HADOOP-17764: --- Author: ASF GitHub Bot

[jira] [Work logged] (HADOOP-17764) S3AInputStream read does not re-open the input stream on the second read retry attempt

2021-06-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17764?focusedWorklogId=613354=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-613354 ] ASF GitHub Bot logged work on HADOOP-17764: --- Author: ASF GitHub Bot

  1   2   >