[GitHub] [hadoop] hemanthboyina commented on pull request #2390: YARN-10442. RM should make sure node label file highly available.

2020-10-29 Thread GitBox
hemanthboyina commented on pull request #2390: URL: https://github.com/apache/hadoop/pull/2390#issuecomment-719223720 thanks @surendralilhore for the PR by default we can have the replication as 0 , if the user need to change the file system's default replication let the user configure

[GitHub] [hadoop] LeonGao91 commented on a change in pull request #2288: HDFS-15548. Allow configuring DISK/ARCHIVE storage types on same device mount

2020-10-29 Thread GitBox
LeonGao91 commented on a change in pull request #2288: URL: https://github.com/apache/hadoop/pull/2288#discussion_r514877753 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/MountVolumeMap.java ## @@ -0,0 +1,143

[GitHub] [hadoop] LeonGao91 commented on a change in pull request #2288: HDFS-15548. Allow configuring DISK/ARCHIVE storage types on same device mount

2020-10-29 Thread GitBox
LeonGao91 commented on a change in pull request #2288: URL: https://github.com/apache/hadoop/pull/2288#discussion_r514875357 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java ## @@ -1503,6 +1503,20 @@ public static final

[GitHub] [hadoop] LeonGao91 commented on a change in pull request #2288: HDFS-15548. Allow configuring DISK/ARCHIVE storage types on same device mount

2020-10-29 Thread GitBox
LeonGao91 commented on a change in pull request #2288: URL: https://github.com/apache/hadoop/pull/2288#discussion_r514875357 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java ## @@ -1503,6 +1503,20 @@ public static final

[jira] [Commented] (HADOOP-17336) Backport HADOOP-16005-"NativeAzureFileSystem does not support setXAttr" and HADOOP-16785. "Improve wasb and abfs resilience on double close() calls. followup to abfs

2020-10-29 Thread Sally Zuo (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223396#comment-17223396 ] Sally Zuo commented on HADOOP-17336: The new patch removed the errors from checkstyle. > Backport

[GitHub] [hadoop] snvijaya commented on pull request #2368: Hadoop-17296. ABFS: Force reads to be always of buffer size

2020-10-29 Thread GitBox
snvijaya commented on pull request #2368: URL: https://github.com/apache/hadoop/pull/2368#issuecomment-719153256 HNS-OAuth [INFO] Results: [INFO] [INFO] Tests run: 89, Failures: 0, Errors: 0, Skipped: 0 [INFO] Results: [INFO] [WARNING]

[jira] [Commented] (HADOOP-17336) Backport HADOOP-16005-"NativeAzureFileSystem does not support setXAttr" and HADOOP-16785. "Improve wasb and abfs resilience on double close() calls. followup to abfs

2020-10-29 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223370#comment-17223370 ] Hadoop QA commented on HADOOP-17336: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (HADOOP-17165) Implement service-user feature in DecayRPCScheduler

2020-10-29 Thread Janus Chow (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223321#comment-17223321 ] Janus Chow commented on HADOOP-17165: - [~csun] Thanks for the reply. I think _refreshCallQueue_ is

[GitHub] [hadoop] huangtianhua commented on a change in pull request #2377: HDFS-15624. fix the function of setting quota by storage type

2020-10-29 Thread GitBox
huangtianhua commented on a change in pull request #2377: URL: https://github.com/apache/hadoop/pull/2377#discussion_r514680759 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeLayoutVersion.java ## @@ -89,7 +89,8 @@

[GitHub] [hadoop] huangtianhua commented on a change in pull request #2377: HDFS-15624. fix the function of setting quota by storage type

2020-10-29 Thread GitBox
huangtianhua commented on a change in pull request #2377: URL: https://github.com/apache/hadoop/pull/2377#discussion_r514679551 ## File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/StorageType.java ## @@ -33,13 +33,12 @@

[GitHub] [hadoop] shvachko commented on pull request #2403: HDFS-15623. Respect configured values of rpc.engine

2020-10-29 Thread GitBox
shvachko commented on pull request #2403: URL: https://github.com/apache/hadoop/pull/2403#issuecomment-719118282 This looks good to me This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [hadoop] Jing9 commented on a change in pull request #2288: HDFS-15548. Allow configuring DISK/ARCHIVE storage types on same device mount

2020-10-29 Thread GitBox
Jing9 commented on a change in pull request #2288: URL: https://github.com/apache/hadoop/pull/2288#discussion_r514666342 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/MountVolumeMap.java ## @@ -0,0 +1,143 @@

[jira] [Updated] (HADOOP-17336) Backport HADOOP-16005-"NativeAzureFileSystem does not support setXAttr" and HADOOP-16785. "Improve wasb and abfs resilience on double close() calls. followup to abfs cl

2020-10-29 Thread Sally Zuo (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sally Zuo updated HADOOP-17336: --- Status: Patch Available (was: Open) > Backport HADOOP-16005-"NativeAzureFileSystem does not

[jira] [Updated] (HADOOP-17336) Backport HADOOP-16005-"NativeAzureFileSystem does not support setXAttr" and HADOOP-16785. "Improve wasb and abfs resilience on double close() calls. followup to abfs cl

2020-10-29 Thread Sally Zuo (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sally Zuo updated HADOOP-17336: --- Attachment: (was: HADOOP-17336-branch-2.10.002.patch) > Backport

[jira] [Updated] (HADOOP-17336) Backport HADOOP-16005-"NativeAzureFileSystem does not support setXAttr" and HADOOP-16785. "Improve wasb and abfs resilience on double close() calls. followup to abfs cl

2020-10-29 Thread Sally Zuo (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sally Zuo updated HADOOP-17336: --- Attachment: Hadoop-17336-branch-2.10.002.patch > Backport HADOOP-16005-"NativeAzureFileSystem does

[jira] [Updated] (HADOOP-17336) Backport HADOOP-16005-"NativeAzureFileSystem does not support setXAttr" and HADOOP-16785. "Improve wasb and abfs resilience on double close() calls. followup to abfs cl

2020-10-29 Thread Sally Zuo (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sally Zuo updated HADOOP-17336: --- Attachment: Hadoop-17336-branch-2.10.001.patch > Backport HADOOP-16005-"NativeAzureFileSystem does

[jira] [Updated] (HADOOP-17336) Backport HADOOP-16005-"NativeAzureFileSystem does not support setXAttr" and HADOOP-16785. "Improve wasb and abfs resilience on double close() calls. followup to abfs cl

2020-10-29 Thread Sally Zuo (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sally Zuo updated HADOOP-17336: --- Attachment: (was: Hadoop-17336-branch-2.10.001.patch) > Backport

[jira] [Updated] (HADOOP-17336) Backport HADOOP-16005-"NativeAzureFileSystem does not support setXAttr" and HADOOP-16785. "Improve wasb and abfs resilience on double close() calls. followup to abfs cl

2020-10-29 Thread Sally Zuo (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sally Zuo updated HADOOP-17336: --- Status: Open (was: Patch Available) > Backport HADOOP-16005-"NativeAzureFileSystem does not

[jira] [Updated] (HADOOP-17336) Backport HADOOP-16005-"NativeAzureFileSystem does not support setXAttr" and HADOOP-16785. "Improve wasb and abfs resilience on double close() calls. followup to abfs cl

2020-10-29 Thread Sally Zuo (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sally Zuo updated HADOOP-17336: --- Attachment: HADOOP-17336-branch-2.10.002.patch > Backport HADOOP-16005-"NativeAzureFileSystem does

[GitHub] [hadoop] hadoop-yetus commented on pull request #2423: YARN-10472. [branch-3.2.2] Backport YARN-10314. YarnClient throws NoClassDefFoundError for WebSocketException with only shaded client ja

2020-10-29 Thread GitBox
hadoop-yetus commented on pull request #2423: URL: https://github.com/apache/hadoop/pull/2423#issuecomment-719107330 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 15m

[jira] [Comment Edited] (HADOOP-17201) Spark job with s3acommitter stuck at the last stage

2020-10-29 Thread James Yu (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223273#comment-17223273 ] James Yu edited comment on HADOOP-17201 at 10/29/20, 11:39 PM: ---

[jira] [Comment Edited] (HADOOP-17201) Spark job with s3acommitter stuck at the last stage

2020-10-29 Thread James Yu (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223273#comment-17223273 ] James Yu edited comment on HADOOP-17201 at 10/29/20, 11:35 PM: ---

[jira] [Commented] (HADOOP-17201) Spark job with s3acommitter stuck at the last stage

2020-10-29 Thread James Yu (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223273#comment-17223273 ] James Yu commented on HADOOP-17201: --- [~ste...@apache.org] I turned on both org.apache.hadoop.fs.s3a

[GitHub] [hadoop] smengcl opened a new pull request #2423: YARN-10472. Backport YARN-10314. YarnClient throws NoClassDefFoundErr…

2020-10-29 Thread GitBox
smengcl opened a new pull request #2423: URL: https://github.com/apache/hadoop/pull/2423 Cherry-picking https://github.com/apache/hadoop/pull/2412 from `branch-3.2` to `branch-3.2.2`. Submitting this PR to trigger CI.

[GitHub] [hadoop] smengcl merged pull request #2412: YARN-10472. [branch-3.2] Backport YARN-10314. YarnClient throws NoClassDefFoundError for WebSocketException with only shaded client jars

2020-10-29 Thread GitBox
smengcl merged pull request #2412: URL: https://github.com/apache/hadoop/pull/2412 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hadoop] smengcl commented on pull request #2412: YARN-10472. [branch-3.2] Backport YARN-10314. YarnClient throws NoClassDefFoundError for WebSocketException with only shaded client jars

2020-10-29 Thread GitBox
smengcl commented on pull request #2412: URL: https://github.com/apache/hadoop/pull/2412#issuecomment-719076242 Thanks @jojochuang for reviewing. Will merge this shortly. This is an automated message from the Apache Git

[jira] [Updated] (HADOOP-17336) Backport HADOOP-16005-"NativeAzureFileSystem does not support setXAttr" and HADOOP-16785. "Improve wasb and abfs resilience on double close() calls. followup to abfs cl

2020-10-29 Thread Sally Zuo (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sally Zuo updated HADOOP-17336: --- Description: Backport:  HADOOP-16005-"NativeAzureFileSystem does not support setXAttr" to

[jira] [Commented] (HADOOP-17336) Backport HADOOP-16005-"NativeAzureFileSystem does not support setXAttr" and HADOOP-16785. "Improve wasb and abfs resilience on double close() calls. followup to abfs

2020-10-29 Thread Sally Zuo (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223194#comment-17223194 ] Sally Zuo commented on HADOOP-17336: The two failing tests seem to be unrelated to this patch. All

[jira] [Work logged] (HADOOP-17313) FileSystem.get to support slow-to-instantiate FS clients

2020-10-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17313?focusedWorklogId=506385=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-506385 ] ASF GitHub Bot logged work on HADOOP-17313: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #2396: HADOOP-17313. FileSystem.get to support slow-to-instantiate FS clients.

2020-10-29 Thread GitBox
hadoop-yetus commented on pull request #2396: URL: https://github.com/apache/hadoop/pull/2396#issuecomment-719002193 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Work logged] (HADOOP-16948) ABFS: Support single writer dirs

2020-10-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16948?focusedWorklogId=506382=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-506382 ] ASF GitHub Bot logged work on HADOOP-16948: --- Author: ASF GitHub Bot

[GitHub] [hadoop] billierinaldi commented on pull request #1925: HADOOP-16948. Support single writer dirs.

2020-10-29 Thread GitBox
billierinaldi commented on pull request #1925: URL: https://github.com/apache/hadoop/pull/1925#issuecomment-718999512 Thanks for reviewing again, @steveloughran! I appreciate your helpful comments and will look into those.

[jira] [Commented] (HADOOP-17336) Backport HADOOP-16005-"NativeAzureFileSystem does not support setXAttr" and HADOOP-16785. "Improve wasb and abfs resilience on double close() calls. followup to abfs

2020-10-29 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223182#comment-17223182 ] Hadoop QA commented on HADOOP-17336: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[GitHub] [hadoop] hadoop-yetus commented on pull request #2421: WIP. HDFS-15643. cleanup the objects in teardown

2020-10-29 Thread GitBox
hadoop-yetus commented on pull request #2421: URL: https://github.com/apache/hadoop/pull/2421#issuecomment-718955616 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Updated] (HADOOP-17336) Backport HADOOP-16005-"NativeAzureFileSystem does not support setXAttr" and HADOOP-16785. "Improve wasb and abfs resilience on double close() calls. followup to abfs cl

2020-10-29 Thread Sally Zuo (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sally Zuo updated HADOOP-17336: --- Attachment: Hadoop-17336-branch-2.10.001.patch Status: Patch Available (was: In Progress)

[jira] [Updated] (HADOOP-17336) Backport HADOOP-16005-"NativeAzureFileSystem does not support setXAttr" and HADOOP-16785. "Improve wasb and abfs resilience on double close() calls. followup to abfs cl

2020-10-29 Thread Sally Zuo (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sally Zuo updated HADOOP-17336: --- Attachment: (was: Hadoop-17336-branch-2.10.001.patch) > Backport

[jira] [Commented] (HADOOP-17335) s3a listing operation will fail in async prefetch if fs closed

2020-10-29 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223088#comment-17223088 ] Steve Loughran commented on HADOOP-17335: - (this was in an app which was doing listFiles() on

[jira] [Work logged] (HADOOP-16948) ABFS: Support single writer dirs

2020-10-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16948?focusedWorklogId=506323=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-506323 ] ASF GitHub Bot logged work on HADOOP-16948: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on a change in pull request #1925: HADOOP-16948. Support single writer dirs.

2020-10-29 Thread GitBox
steveloughran commented on a change in pull request #1925: URL: https://github.com/apache/hadoop/pull/1925#discussion_r514450156 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystemStore.java ## @@ -243,6 +252,16 @@ public

[GitHub] [hadoop] akiyamaneko removed a comment on pull request #2401: YARN-10469. YARN-UI2 The accuracy of the percentage values in the same chart on the YARN 'Cluster OverView' page are inconsisten

2020-10-29 Thread GitBox
akiyamaneko removed a comment on pull request #2401: URL: https://github.com/apache/hadoop/pull/2401#issuecomment-714415749 cc @sunilgovind could you help to review it? thanks. This is an automated message from the

[jira] [Work logged] (HADOOP-17311) ABFS: Logs should redact SAS signature

2020-10-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17311?focusedWorklogId=506311=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-506311 ] ASF GitHub Bot logged work on HADOOP-17311: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on a change in pull request #2422: HADOOP-17311. ABFS: Masking SAS signatures from logs

2020-10-29 Thread GitBox
steveloughran commented on a change in pull request #2422: URL: https://github.com/apache/hadoop/pull/2422#discussion_r514443770 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsRestOperation.java ## @@ -256,7 +256,9 @@ private

[jira] [Commented] (HADOOP-17165) Implement service-user feature in DecayRPCScheduler

2020-10-29 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223073#comment-17223073 ] Chao Sun commented on HADOOP-17165: --- [~Symious], yes like discussed in the previous comments, we

[jira] [Work logged] (HADOOP-17313) FileSystem.get to support slow-to-instantiate FS clients

2020-10-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17313?focusedWorklogId=506293=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-506293 ] ASF GitHub Bot logged work on HADOOP-17313: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on pull request #2396: HADOOP-17313. FileSystem.get to support slow-to-instantiate FS clients.

2020-10-29 Thread GitBox
steveloughran commented on pull request #2396: URL: https://github.com/apache/hadoop/pull/2396#issuecomment-718893014 > Is there particular reasoning for "64" as the default value? No. I wondered whether to make it smaller or just leave large. Large: no visible impact of this

[GitHub] [hadoop] steveloughran commented on pull request #2396: HADOOP-17313. FileSystem.get to support slow-to-instantiate FS clients.

2020-10-29 Thread GitBox
steveloughran commented on pull request #2396: URL: https://github.com/apache/hadoop/pull/2396#issuecomment-718889722 ``` ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java:3512: checkArgument(permits > 0 , "Invalid value of %s: %s",:33: ','

[jira] [Work logged] (HADOOP-17313) FileSystem.get to support slow-to-instantiate FS clients

2020-10-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17313?focusedWorklogId=506291=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-506291 ] ASF GitHub Bot logged work on HADOOP-17313: --- Author: ASF GitHub Bot

[jira] [Work logged] (HADOOP-17313) FileSystem.get to support slow-to-instantiate FS clients

2020-10-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17313?focusedWorklogId=506282=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-506282 ] ASF GitHub Bot logged work on HADOOP-17313: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on a change in pull request #2396: HADOOP-17313. FileSystem.get to support slow-to-instantiate FS clients.

2020-10-29 Thread GitBox
steveloughran commented on a change in pull request #2396: URL: https://github.com/apache/hadoop/pull/2396#discussion_r514412041 ## File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java ## @@ -3517,33 +3546,86 @@ private FileSystem

[GitHub] [hadoop] steveloughran commented on a change in pull request #2396: HADOOP-17313. FileSystem.get to support slow-to-instantiate FS clients.

2020-10-29 Thread GitBox
steveloughran commented on a change in pull request #2396: URL: https://github.com/apache/hadoop/pull/2396#discussion_r514411189 ## File path: hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileSystemCaching.java ## @@ -336,4 +344,134 @@ public

[jira] [Work logged] (HADOOP-17313) FileSystem.get to support slow-to-instantiate FS clients

2020-10-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17313?focusedWorklogId=506281=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-506281 ] ASF GitHub Bot logged work on HADOOP-17313: --- Author: ASF GitHub Bot

[GitHub] [hadoop] vinayakumarb commented on a change in pull request #2377: HDFS-15624. fix the function of setting quota by storage type

2020-10-29 Thread GitBox
vinayakumarb commented on a change in pull request #2377: URL: https://github.com/apache/hadoop/pull/2377#discussion_r514403558 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeLayoutVersion.java ## @@ -89,7 +89,8 @@

[GitHub] [hadoop] vinayakumarb commented on a change in pull request #2377: HDFS-15624. fix the function of setting quota by storage type

2020-10-29 Thread GitBox
vinayakumarb commented on a change in pull request #2377: URL: https://github.com/apache/hadoop/pull/2377#discussion_r514401987 ## File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/StorageType.java ## @@ -33,13 +33,12 @@

[GitHub] [hadoop] vinayakumarb commented on pull request #2377: HDFS-15624. fix the function of setting quota by storage type

2020-10-29 Thread GitBox
vinayakumarb commented on pull request #2377: URL: https://github.com/apache/hadoop/pull/2377#issuecomment-718871538 > I think, you can hold on till HDFS-15660 addressed. I dont think need to hold this Jira, unless HDFS-15660 cant be solved in the same release as NVDIMM feature

[GitHub] [hadoop] vinayakumarb commented on a change in pull request #2377: HDFS-15624. fix the function of setting quota by storage type

2020-10-29 Thread GitBox
vinayakumarb commented on a change in pull request #2377: URL: https://github.com/apache/hadoop/pull/2377#discussion_r513143383 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java ## @@ -3569,6 +3569,9 @@ void

[jira] [Work logged] (HADOOP-17271) S3A statistics to support IOStatistics

2020-10-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17271?focusedWorklogId=506263=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-506263 ] ASF GitHub Bot logged work on HADOOP-17271: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran edited a comment on pull request #2324: HADOOP-17271. S3A statistics to support IOStatistics

2020-10-29 Thread GitBox
steveloughran edited a comment on pull request #2324: URL: https://github.com/apache/hadoop/pull/2324#issuecomment-718847026 test run failure ``` mvit -Dparallel-tests -DtestsThreadCount=4 -Dmarkers=keep -Ds3guard -Ddynamo -Dfs.s3a.directory.marker.audit=true -Dscale ```

[jira] [Commented] (HADOOP-16881) PseudoAuthenticator does not disconnect HttpURLConnection leading to CLOSE_WAIT cnxns

2020-10-29 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222989#comment-17222989 ] Steve Loughran commented on HADOOP-16881: - no idea > PseudoAuthenticator does not disconnect

[jira] [Work logged] (HADOOP-17271) S3A statistics to support IOStatistics

2020-10-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17271?focusedWorklogId=506256=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-506256 ] ASF GitHub Bot logged work on HADOOP-17271: --- Author: ASF GitHub Bot

[jira] [Work logged] (HADOOP-17271) S3A statistics to support IOStatistics

2020-10-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17271?focusedWorklogId=506257=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-506257 ] ASF GitHub Bot logged work on HADOOP-17271: --- Author: ASF GitHub Bot

[jira] [Work logged] (HADOOP-17271) S3A statistics to support IOStatistics

2020-10-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17271?focusedWorklogId=506253=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-506253 ] ASF GitHub Bot logged work on HADOOP-17271: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #2324: HADOOP-17271. S3A statistics to support IOStatistics

2020-10-29 Thread GitBox
hadoop-yetus removed a comment on pull request #2324: URL: https://github.com/apache/hadoop/pull/2324#issuecomment-717947867 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:|

[jira] [Work logged] (HADOOP-17271) S3A statistics to support IOStatistics

2020-10-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17271?focusedWorklogId=506254=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-506254 ] ASF GitHub Bot logged work on HADOOP-17271: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on pull request #2324: HADOOP-17271. S3A statistics to support IOStatistics

2020-10-29 Thread GitBox
steveloughran commented on pull request #2324: URL: https://github.com/apache/hadoop/pull/2324#issuecomment-718847026 test run failure ``` [INFO] Running org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB [ERROR] Tests run: 20, Failures: 0, Errors: 1, Skipped: 1, Time

[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #2324: HADOOP-17271. S3A statistics to support IOStatistics

2020-10-29 Thread GitBox
hadoop-yetus removed a comment on pull request #2324: URL: https://github.com/apache/hadoop/pull/2324#issuecomment-707319515 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:|

[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #2324: HADOOP-17271. S3A statistics to support IOStatistics

2020-10-29 Thread GitBox
hadoop-yetus removed a comment on pull request #2324: URL: https://github.com/apache/hadoop/pull/2324#issuecomment-704583811 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:|

[jira] [Work logged] (HADOOP-16830) Add public IOStatistics API

2020-10-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16830?focusedWorklogId=506247=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-506247 ] ASF GitHub Bot logged work on HADOOP-16830: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on pull request #2323: HADOOP-16830. Add public IOStatistics API.

2020-10-29 Thread GitBox
steveloughran commented on pull request #2323: URL: https://github.com/apache/hadoop/pull/2323#issuecomment-718839259 This PR won't merge right now due to the move to shaded Preconditons. The PR #2324 has been rebased; I'll be rebuilding this PR from that one as a trunk + one big fat

[jira] [Work logged] (HADOOP-17271) S3A statistics to support IOStatistics

2020-10-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17271?focusedWorklogId=506243=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-506243 ] ASF GitHub Bot logged work on HADOOP-17271: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #2324: HADOOP-17271. S3A statistics to support IOStatistics

2020-10-29 Thread GitBox
hadoop-yetus commented on pull request #2324: URL: https://github.com/apache/hadoop/pull/2324#issuecomment-718827022 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] hadoop-yetus commented on pull request #2368: Hadoop-17296. ABFS: Force reads to be always of buffer size

2020-10-29 Thread GitBox
hadoop-yetus commented on pull request #2368: URL: https://github.com/apache/hadoop/pull/2368#issuecomment-718737087 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Commented] (HADOOP-16881) PseudoAuthenticator does not disconnect HttpURLConnection leading to CLOSE_WAIT cnxns

2020-10-29 Thread Attila Magyar (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222866#comment-17222866 ] Attila Magyar commented on HADOOP-16881: Hey @[~ste...@apache.org], do you have any insights on

[jira] [Work logged] (HADOOP-17311) ABFS: Logs should redact SAS signature

2020-10-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17311?focusedWorklogId=506179=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-506179 ] ASF GitHub Bot logged work on HADOOP-17311: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #2422: HADOOP-17311. ABFS: Masking SAS signatures from logs

2020-10-29 Thread GitBox
hadoop-yetus commented on pull request #2422: URL: https://github.com/apache/hadoop/pull/2422#issuecomment-718698208 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Work logged] (HADOOP-17311) ABFS: Logs should redact SAS signature

2020-10-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17311?focusedWorklogId=506175=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-506175 ] ASF GitHub Bot logged work on HADOOP-17311: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #2422: HADOOP-17311. ABFS: Masking SAS signatures from logs

2020-10-29 Thread GitBox
hadoop-yetus commented on pull request #2422: URL: https://github.com/apache/hadoop/pull/2422#issuecomment-718695545 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Work logged] (HADOOP-17311) ABFS: Logs should redact SAS signature

2020-10-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17311?focusedWorklogId=506174=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-506174 ] ASF GitHub Bot logged work on HADOOP-17311: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #2422: HADOOP-17311. ABFS: Masking SAS signatures from logs

2020-10-29 Thread GitBox
hadoop-yetus commented on pull request #2422: URL: https://github.com/apache/hadoop/pull/2422#issuecomment-718692046 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Work logged] (HADOOP-17313) FileSystem.get to support slow-to-instantiate FS clients

2020-10-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17313?focusedWorklogId=506157=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-506157 ] ASF GitHub Bot logged work on HADOOP-17313: --- Author: ASF GitHub Bot

[GitHub] [hadoop] mehakmeet commented on a change in pull request #2396: HADOOP-17313. FileSystem.get to support slow-to-instantiate FS clients.

2020-10-29 Thread GitBox
mehakmeet commented on a change in pull request #2396: URL: https://github.com/apache/hadoop/pull/2396#discussion_r514165609 ## File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java ## @@ -3517,33 +3546,86 @@ private FileSystem

[jira] [Assigned] (HADOOP-17311) ABFS: Logs should redact SAS signature

2020-10-29 Thread Bilahari T H (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H reassigned HADOOP-17311: - Assignee: Bilahari T H (was: Sneha Vijayarajan) > ABFS: Logs should redact SAS

[jira] [Updated] (HADOOP-17311) ABFS: Logs should redact SAS signature

2020-10-29 Thread Bilahari T H (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17311: -- Status: Patch Available (was: Open) > ABFS: Logs should redact SAS signature >

[jira] [Work logged] (HADOOP-17311) ABFS: Logs should redact SAS signature

2020-10-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17311?focusedWorklogId=506146=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-506146 ] ASF GitHub Bot logged work on HADOOP-17311: --- Author: ASF GitHub Bot

[jira] [Updated] (HADOOP-17311) ABFS: Logs should redact SAS signature

2020-10-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-17311: Labels: pull-request-available (was: ) > ABFS: Logs should redact SAS signature >

[GitHub] [hadoop] bilaharith opened a new pull request #2422: HADOOP-17311. ABFS: Masking SAS signatures from logs

2020-10-29 Thread GitBox
bilaharith opened a new pull request #2422: URL: https://github.com/apache/hadoop/pull/2422 Masking SAS signatures from logs HNS-OAuth [INFO] Results: [INFO] [INFO] Tests run: 88, Failures: 0, Errors: 0, Skipped: 0 [INFO] Results: [INFO]

[jira] [Updated] (HADOOP-17329) mvn site commands fails due to MetricsSystemImpl changes

2020-10-29 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HADOOP-17329: - Fix Version/s: 3.2.3 > mvn site commands fails due to MetricsSystemImpl changes >

[jira] [Commented] (HADOOP-17329) mvn site commands fails due to MetricsSystemImpl changes

2020-10-29 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222798#comment-17222798 ] Xiaoqiao He commented on HADOOP-17329: -- Thanks [~sunilg] for your reviews and commit, add 3.2.3 as

[GitHub] [hadoop] smengcl edited a comment on pull request #2412: [branch-3.2] YARN-10314. YarnClient throws NoClassDefFoundError for WebSocketException with only shaded client jars

2020-10-29 Thread GitBox
smengcl edited a comment on pull request #2412: URL: https://github.com/apache/hadoop/pull/2412#issuecomment-718466315 CI shadedclient also passed. Please kindly take another look at the patch @jojochuang. This is an

[GitHub] [hadoop] smengcl commented on pull request #2412: [branch-3.2] YARN-10314. YarnClient throws NoClassDefFoundError for WebSocketException with only shaded client jars

2020-10-29 Thread GitBox
smengcl commented on pull request #2412: URL: https://github.com/apache/hadoop/pull/2412#issuecomment-718466315 CI shadedclient also passed. Please kindly take another look @jojochuang at the patch. This is an automated

[GitHub] [hadoop] hadoop-yetus commented on pull request #2412: [branch-3.2] YARN-10314. YarnClient throws NoClassDefFoundError for WebSocketException with only shaded client jars

2020-10-29 Thread GitBox
hadoop-yetus commented on pull request #2412: URL: https://github.com/apache/hadoop/pull/2412#issuecomment-718456812 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 16m

[GitHub] [hadoop] ayushtkn commented on a change in pull request #2377: HDFS-15624. fix the function of setting quota by storage type

2020-10-29 Thread GitBox
ayushtkn commented on a change in pull request #2377: URL: https://github.com/apache/hadoop/pull/2377#discussion_r514032406 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java ## @@ -3569,6 +3569,9 @@ void

[GitHub] [hadoop] smengcl commented on pull request #2412: [branch-3.2] YARN-10314. YarnClient throws NoClassDefFoundError for WebSocketException with only shaded client jars

2020-10-29 Thread GitBox
smengcl commented on pull request #2412: URL: https://github.com/apache/hadoop/pull/2412#issuecomment-718402415 I ran `mvn verify -pl :hadoop-client-minicluster` on the latest commit. It passed ``` [WARNING] maven-shade-plugin has detected that some class files are [WARNING]