[GitHub] [hadoop] hadoop-yetus commented on pull request #2625: HDFS-15683. Allow configuring DISK/ARCHIVE capacity for individual volumes.

2021-02-08 Thread GitBox
hadoop-yetus commented on pull request #2625: URL: https://github.com/apache/hadoop/pull/2625#issuecomment-774952753 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] anoopsjohn commented on a change in pull request #2646: HADOOP-17038 Support disabling buffered reads in ABFS positional reads.

2021-02-08 Thread GitBox
anoopsjohn commented on a change in pull request #2646: URL: https://github.com/apache/hadoop/pull/2646#discussion_r571844939 ## File path: hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/ITestAbfsPread.java ## @@ -0,0 +1,128 @@ +/** + * Licensed

[GitHub] [hadoop] anoopsjohn commented on a change in pull request #2646: HADOOP-17038 Support disabling buffered reads in ABFS positional reads.

2021-02-08 Thread GitBox
anoopsjohn commented on a change in pull request #2646: URL: https://github.com/apache/hadoop/pull/2646#discussion_r571845277 ## File path: hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAbfsInputStreamStatistics.java ## @@ -399,6 +406,89 @@ public

[jira] [Work logged] (HADOOP-17038) Support disabling buffered reads in ABFS positional reads

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17038?focusedWorklogId=549422&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549422 ] ASF GitHub Bot logged work on HADOOP-17038: --- Author: ASF GitHub B

[jira] [Work logged] (HADOOP-17038) Support disabling buffered reads in ABFS positional reads

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17038?focusedWorklogId=549423&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549423 ] ASF GitHub Bot logged work on HADOOP-17038: --- Author: ASF GitHub B

[GitHub] [hadoop] anoopsjohn commented on pull request #2646: HADOOP-17038 Support disabling buffered reads in ABFS positional reads.

2021-02-08 Thread GitBox
anoopsjohn commented on pull request #2646: URL: https://github.com/apache/hadoop/pull/2646#issuecomment-774956936 Thanks for the reviews.. Addressed comments. Added reply for 2 comments. Can you pls confirm @steveloughran whether those reply looks ok ?

[jira] [Work logged] (HADOOP-17038) Support disabling buffered reads in ABFS positional reads

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17038?focusedWorklogId=549424&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549424 ] ASF GitHub Bot logged work on HADOOP-17038: --- Author: ASF GitHub B

[GitHub] [hadoop] anoopsjohn commented on a change in pull request #2646: HADOOP-17038 Support disabling buffered reads in ABFS positional reads.

2021-02-08 Thread GitBox
anoopsjohn commented on a change in pull request #2646: URL: https://github.com/apache/hadoop/pull/2646#discussion_r571847078 ## File path: hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/ITestAbfsPread.java ## @@ -0,0 +1,128 @@ +/** + * Licensed

[jira] [Work logged] (HADOOP-17038) Support disabling buffered reads in ABFS positional reads

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17038?focusedWorklogId=549425&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549425 ] ASF GitHub Bot logged work on HADOOP-17038: --- Author: ASF GitHub B

[GitHub] [hadoop] anoopsjohn commented on pull request #2646: HADOOP-17038 Support disabling buffered reads in ABFS positional reads.

2021-02-08 Thread GitBox
anoopsjohn commented on pull request #2646: URL: https://github.com/apache/hadoop/pull/2646#issuecomment-774959132 > Production code looks good. Added some comments in tests. > Also pending checkstyle > `./hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/

[jira] [Work logged] (HADOOP-17038) Support disabling buffered reads in ABFS positional reads

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17038?focusedWorklogId=549427&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549427 ] ASF GitHub Bot logged work on HADOOP-17038: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus commented on pull request #2639: HDFS-15785. Datanode to support using DNS to resolve nameservices to IP addresses to get list of namenodes.

2021-02-08 Thread GitBox
hadoop-yetus commented on pull request #2639: URL: https://github.com/apache/hadoop/pull/2639#issuecomment-774974914 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] jianghuazhu opened a new pull request #2686: HDFS-15826. Solve the problem of incorrect progress of delegation tokens when loading FsImage.

2021-02-08 Thread GitBox
jianghuazhu opened a new pull request #2686: URL: https://github.com/apache/hadoop/pull/2686 …ens when loading FsImage. ## NOTICE Please create an issue in ASF JIRA before opening a pull request, and you need to set the title of the pull request which starts with the corr

[GitHub] [hadoop] hadoop-yetus commented on pull request #2646: HADOOP-17038 Support disabling buffered reads in ABFS positional reads.

2021-02-08 Thread GitBox
hadoop-yetus commented on pull request #2646: URL: https://github.com/apache/hadoop/pull/2646#issuecomment-775003443 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[jira] [Work logged] (HADOOP-17038) Support disabling buffered reads in ABFS positional reads

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17038?focusedWorklogId=549449&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549449 ] ASF GitHub Bot logged work on HADOOP-17038: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on pull request #2684: HADOOP-16906. Abortable

2021-02-08 Thread GitBox
steveloughran commented on pull request #2684: URL: https://github.com/apache/hadoop/pull/2684#issuecomment-775058042 Thanks @mukund-thakur Had some thoughts over the w/e 1. The call should return something, `AbortOutcome` which can include a list of non-critical exceptions c

[jira] [Work logged] (HADOOP-16906) Add some Abortable.abort() interface for streams etc which can be terminated

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16906?focusedWorklogId=549508&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549508 ] ASF GitHub Bot logged work on HADOOP-16906: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on a change in pull request #2587: HADOOP-13327 Output Stream Specification.

2021-02-08 Thread GitBox
steveloughran commented on a change in pull request #2587: URL: https://github.com/apache/hadoop/pull/2587#discussion_r571955452 ## File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java ## @@ -137,8 +139,13 @@ public void ini

[jira] [Work logged] (HADOOP-13327) Add OutputStream + Syncable to the Filesystem Specification

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13327?focusedWorklogId=549510&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549510 ] ASF GitHub Bot logged work on HADOOP-13327: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus commented on pull request #2677: HDFS-15817. Rename snapshots while marking them deleted.

2021-02-08 Thread GitBox
hadoop-yetus commented on pull request #2677: URL: https://github.com/apache/hadoop/pull/2677#issuecomment-775102419 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[jira] [Commented] (HADOOP-13887) Encrypt S3A data client-side with AWS SDK (S3-CSE)

2021-02-08 Thread Mehakmeet Singh (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17280997#comment-17280997 ] Mehakmeet Singh commented on HADOOP-13887: -- I have been trying to add support

[GitHub] [hadoop] steveloughran commented on pull request #2587: HADOOP-13327 Output Stream Specification.

2021-02-08 Thread GitBox
steveloughran commented on pull request #2587: URL: https://github.com/apache/hadoop/pull/2587#issuecomment-775113631 FWIW, here are the IOStats logged on the runs ABFS (which supports hsync) ``` .dfs.core.windows.net/stevel-testing/test/testSyncable?action=flush&retainUncommit

[jira] [Work logged] (HADOOP-13327) Add OutputStream + Syncable to the Filesystem Specification

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13327?focusedWorklogId=549544&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549544 ] ASF GitHub Bot logged work on HADOOP-13327: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus commented on pull request #2686: HDFS-15826. Solve the problem of incorrect progress of delegation tokens when loading FsImage.

2021-02-08 Thread GitBox
hadoop-yetus commented on pull request #2686: URL: https://github.com/apache/hadoop/pull/2686#issuecomment-775178247 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] steveloughran merged pull request #2685: HADOOP-16767 Handle non-IO exceptions in reopen()

2021-02-08 Thread GitBox
steveloughran merged pull request #2685: URL: https://github.com/apache/hadoop/pull/2685 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[jira] [Work logged] (HADOOP-16767) S3AInputStream reopening does not handle non IO exceptions properly

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16767?focusedWorklogId=549672&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549672 ] ASF GitHub Bot logged work on HADOOP-16767: --- Author: ASF GitHub B

[jira] [Commented] (HADOOP-16767) S3AInputStream reopening does not handle non IO exceptions properly

2021-02-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17281164#comment-17281164 ] Steve Loughran commented on HADOOP-16767: - thanks, merged back > S3AInputStrea

[jira] [Updated] (HADOOP-16767) S3AInputStream reopening does not handle non IO exceptions properly

2021-02-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-16767: Fix Version/s: 2.10.2 > S3AInputStream reopening does not handle non IO exceptions proper

[GitHub] [hadoop] steveloughran commented on a change in pull request #2646: HADOOP-17038 Support disabling buffered reads in ABFS positional reads.

2021-02-08 Thread GitBox
steveloughran commented on a change in pull request #2646: URL: https://github.com/apache/hadoop/pull/2646#discussion_r572175379 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsInputStream.java ## @@ -135,6 +145,41 @@ public Str

[jira] [Work logged] (HADOOP-17038) Support disabling buffered reads in ABFS positional reads

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17038?focusedWorklogId=549677&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549677 ] ASF GitHub Bot logged work on HADOOP-17038: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on a change in pull request #2646: HADOOP-17038 Support disabling buffered reads in ABFS positional reads.

2021-02-08 Thread GitBox
steveloughran commented on a change in pull request #2646: URL: https://github.com/apache/hadoop/pull/2646#discussion_r572178766 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystemStore.java ## @@ -634,12 +638,15 @@ public Ab

[jira] [Work logged] (HADOOP-17038) Support disabling buffered reads in ABFS positional reads

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17038?focusedWorklogId=549681&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549681 ] ASF GitHub Bot logged work on HADOOP-17038: --- Author: ASF GitHub B

[jira] [Commented] (HADOOP-17518) Usage of incorrect regex range A-z

2021-02-08 Thread Kihwal Lee (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17281189#comment-17281189 ] Kihwal Lee commented on HADOOP-17518: - It is only referenced by a HttpFS test. We c

[jira] [Commented] (HADOOP-13551) Collect AwsSdkMetrics in S3A FileSystem IOStatistics

2021-02-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17281221#comment-17281221 ] Steve Loughran commented on HADOOP-13551: - Stack {code} [ERROR] testJobSubmi

[GitHub] [hadoop] hadoop-yetus commented on pull request #2683: HADOOP-17516. Upgrade ant to 1.10.9.

2021-02-08 Thread GitBox
hadoop-yetus commented on pull request #2683: URL: https://github.com/apache/hadoop/pull/2683#issuecomment-775316840 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[jira] [Work logged] (HADOOP-17516) Upgrade ant to 1.10.9

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17516?focusedWorklogId=549728&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549728 ] ASF GitHub Bot logged work on HADOOP-17516: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus commented on pull request #2680: HADOOP-17514. Remove trace subcommand from hadoop CLI.

2021-02-08 Thread GitBox
hadoop-yetus commented on pull request #2680: URL: https://github.com/apache/hadoop/pull/2680#issuecomment-775319496 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[jira] [Work logged] (HADOOP-17514) Remove trace subcommand from hadoop CLI

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17514?focusedWorklogId=549732&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549732 ] ASF GitHub Bot logged work on HADOOP-17514: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran opened a new pull request #2687: HADOOP-13551. Wire up AWS SDK metrics to IOStatistics

2021-02-08 Thread GitBox
steveloughran opened a new pull request #2687: URL: https://github.com/apache/hadoop/pull/2687 tests which fail on the new builder API. Need to understand its config better *this PR doesn't fix the problem; it's the starting point to fixing it* ---

[GitHub] [hadoop] steveloughran commented on pull request #2687: HADOOP-13551. Wire up AWS SDK metrics to IOStatistics

2021-02-08 Thread GitBox
steveloughran commented on pull request #2687: URL: https://github.com/apache/hadoop/pull/2687#issuecomment-775337429 @mehakmeet FYI This is an automated message from the Apache Git Service. To respond to the message, please

[jira] [Updated] (HADOOP-13551) Collect AwsSdkMetrics in S3A FileSystem IOStatistics

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-13551: Labels: pull-request-available (was: ) > Collect AwsSdkMetrics in S3A FileSystem IOStati

[jira] [Work logged] (HADOOP-13551) Collect AwsSdkMetrics in S3A FileSystem IOStatistics

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13551?focusedWorklogId=549748&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549748 ] ASF GitHub Bot logged work on HADOOP-13551: --- Author: ASF GitHub B

[jira] [Work logged] (HADOOP-13551) Collect AwsSdkMetrics in S3A FileSystem IOStatistics

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13551?focusedWorklogId=549750&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549750 ] ASF GitHub Bot logged work on HADOOP-13551: --- Author: ASF GitHub B

[jira] [Created] (HADOOP-17519) Decommission S3Guard

2021-02-08 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17519: --- Summary: Decommission S3Guard Key: HADOOP-17519 URL: https://issues.apache.org/jira/browse/HADOOP-17519 Project: Hadoop Common Issue Type: Improvement

[jira] [Created] (HADOOP-17520) S3A to downgraded to unguarded if the DDB table cannot be found

2021-02-08 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17520: --- Summary: S3A to downgraded to unguarded if the DDB table cannot be found Key: HADOOP-17520 URL: https://issues.apache.org/jira/browse/HADOOP-17520 Project: Hado

[jira] [Assigned] (HADOOP-17520) S3A to downgraded to unguarded if the DDB table cannot be found

2021-02-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-17520: --- Assignee: Steve Loughran > S3A to downgraded to unguarded if the DDB table cannot

[GitHub] [hadoop] steveloughran commented on a change in pull request #2654: HADOOP-17497. refactoring the signature of S3ClientFactory breaks hboss compile.

2021-02-08 Thread GitBox
steveloughran commented on a change in pull request #2654: URL: https://github.com/apache/hadoop/pull/2654#discussion_r572265444 ## File path: hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3ClientFactory.java ## @@ -30,6 +30,9 @@ /** * Factory for creati

[jira] [Work logged] (HADOOP-17497) refactoring the signature of S3ClientFactory breaks hboss compile

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17497?focusedWorklogId=549753&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549753 ] ASF GitHub Bot logged work on HADOOP-17497: --- Author: ASF GitHub B

[GitHub] [hadoop] yzhangal opened a new pull request #2688: Branch 2.10

2021-02-08 Thread GitBox
yzhangal opened a new pull request #2688: URL: https://github.com/apache/hadoop/pull/2688 ## NOTICE Please create an issue in ASF JIRA before opening a pull request, and you need to set the title of the pull request which starts with the corresponding JIRA issue number. (e.g. HA

[GitHub] [hadoop] hadoop-yetus commented on pull request #2688: Branch 2.10

2021-02-08 Thread GitBox
hadoop-yetus commented on pull request #2688: URL: https://github.com/apache/hadoop/pull/2688#issuecomment-775347943 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 0

[jira] [Work started] (HADOOP-17520) S3A to downgraded to unguarded if the DDB table cannot be found

2021-02-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HADOOP-17520 started by Steve Loughran. --- > S3A to downgraded to unguarded if the DDB table cannot be found >

[GitHub] [hadoop] yzhangal commented on pull request #2685: HADOOP-16767 Handle non-IO exceptions in reopen()

2021-02-08 Thread GitBox
yzhangal commented on pull request #2685: URL: https://github.com/apache/hadoop/pull/2685#issuecomment-775353597 Thanks a lot @steveloughran ! I just uploaded HADOOP-17338 PR for 2.10 branch, but somehow it's not visible in the jira yet. Wonder if you were able to see. Thanks. ---

[jira] [Work logged] (HADOOP-16767) S3AInputStream reopening does not handle non IO exceptions properly

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16767?focusedWorklogId=549767&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549767 ] ASF GitHub Bot logged work on HADOOP-16767: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on pull request #2671: HADOOP-17498 mkdir operation.

2021-02-08 Thread GitBox
steveloughran commented on pull request #2671: URL: https://github.com/apache/hadoop/pull/2671#issuecomment-775360729 Needs: ITest, could do a unit test for completeness, it being isolated enough. With some counters of probes/createFakeDir we could verify that the operation did dir

[jira] [Work logged] (HADOOP-17498) Add a MkdirOperation for chained S3 operations during mkdir

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17498?focusedWorklogId=549781&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549781 ] ASF GitHub Bot logged work on HADOOP-17498: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on pull request #2685: HADOOP-16767 Handle non-IO exceptions in reopen()

2021-02-08 Thread GitBox
steveloughran commented on pull request #2685: URL: https://github.com/apache/hadoop/pull/2685#issuecomment-775362444 Has yetus started on the PR? as that's what really matters. You can always just link the JIRA to the PR by hand ---

[jira] [Work logged] (HADOOP-16767) S3AInputStream reopening does not handle non IO exceptions properly

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16767?focusedWorklogId=549783&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549783 ] ASF GitHub Bot logged work on HADOOP-16767: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on a change in pull request #1925: HADOOP-16948. Support single writer dirs.

2021-02-08 Thread GitBox
steveloughran commented on a change in pull request #1925: URL: https://github.com/apache/hadoop/pull/1925#discussion_r572289020 ## File path: hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemLease.java ## @@ -0,0 +1,295 @@ +/** + *

[jira] [Work logged] (HADOOP-16948) ABFS: Support single writer dirs

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16948?focusedWorklogId=549785&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549785 ] ASF GitHub Bot logged work on HADOOP-16948: --- Author: ASF GitHub B

[GitHub] [hadoop] yzhangal commented on pull request #2685: HADOOP-16767 Handle non-IO exceptions in reopen()

2021-02-08 Thread GitBox
yzhangal commented on pull request #2685: URL: https://github.com/apache/hadoop/pull/2685#issuecomment-775367808 https://github.com/apache/hadoop/pull/2688 Seems the build failed, maybe because I uploaded the diff before your merge thus the base is different. As of now, I don't see your m

[jira] [Work logged] (HADOOP-16767) S3AInputStream reopening does not handle non IO exceptions properly

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16767?focusedWorklogId=549791&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549791 ] ASF GitHub Bot logged work on HADOOP-16767: --- Author: ASF GitHub B

[GitHub] [hadoop] Jing9 commented on pull request #2625: HDFS-15683. Allow configuring DISK/ARCHIVE capacity for individual volumes.

2021-02-08 Thread GitBox
Jing9 commented on pull request #2625: URL: https://github.com/apache/hadoop/pull/2625#issuecomment-775369725 The javac warning looks unrelated. It reported "generated 14 new + 580 unchanged - 14 fixed = 594 total (was 594)" and the warnings are not caused by this PR. But we can use this c

[GitHub] [hadoop] Jing9 merged pull request #2625: HDFS-15683. Allow configuring DISK/ARCHIVE capacity for individual volumes.

2021-02-08 Thread GitBox
Jing9 merged pull request #2625: URL: https://github.com/apache/hadoop/pull/2625 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hadoop] LeonGao91 commented on pull request #2625: HDFS-15683. Allow configuring DISK/ARCHIVE capacity for individual volumes.

2021-02-08 Thread GitBox
LeonGao91 commented on pull request #2625: URL: https://github.com/apache/hadoop/pull/2625#issuecomment-775370895 thx @Jing9 for the review! I will open a jira for the javac issues. This is an automated message from the Apach

[GitHub] [hadoop] steveloughran commented on a change in pull request #1925: HADOOP-16948. Support single writer dirs.

2021-02-08 Thread GitBox
steveloughran commented on a change in pull request #1925: URL: https://github.com/apache/hadoop/pull/1925#discussion_r572291660 ## File path: hadoop-tools/hadoop-azure/src/site/markdown/abfs.md ## @@ -877,6 +877,21 @@ enabled for your Azure Storage account." The directories c

[jira] [Work logged] (HADOOP-16948) ABFS: Support single writer dirs

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16948?focusedWorklogId=549799&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549799 ] ASF GitHub Bot logged work on HADOOP-16948: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus commented on pull request #2687: HADOOP-13551. Wire up AWS SDK metrics to IOStatistics

2021-02-08 Thread GitBox
hadoop-yetus commented on pull request #2687: URL: https://github.com/apache/hadoop/pull/2687#issuecomment-775382658 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[jira] [Work logged] (HADOOP-13551) Collect AwsSdkMetrics in S3A FileSystem IOStatistics

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13551?focusedWorklogId=549817&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549817 ] ASF GitHub Bot logged work on HADOOP-13551: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus commented on pull request #2654: HADOOP-17497. refactoring the signature of S3ClientFactory breaks hboss compile.

2021-02-08 Thread GitBox
hadoop-yetus commented on pull request #2654: URL: https://github.com/apache/hadoop/pull/2654#issuecomment-775390596 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[jira] [Work logged] (HADOOP-17497) refactoring the signature of S3ClientFactory breaks hboss compile

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17497?focusedWorklogId=549825&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549825 ] ASF GitHub Bot logged work on HADOOP-17497: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus commented on pull request #2684: HADOOP-16906. Abortable

2021-02-08 Thread GitBox
hadoop-yetus commented on pull request #2684: URL: https://github.com/apache/hadoop/pull/2684#issuecomment-775397931 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[jira] [Work logged] (HADOOP-16906) Add some Abortable.abort() interface for streams etc which can be terminated

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16906?focusedWorklogId=549836&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549836 ] ASF GitHub Bot logged work on HADOOP-16906: --- Author: ASF GitHub B

[jira] [Commented] (HADOOP-17520) S3A to downgraded to unguarded if the DDB table cannot be found

2021-02-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17281347#comment-17281347 ] Steve Loughran commented on HADOOP-17520: - Mostly straightforward. One troubles

[jira] [Commented] (HADOOP-17520) S3A to downgraded to unguarded if the DDB table cannot be found

2021-02-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17281349#comment-17281349 ] Steve Loughran commented on HADOOP-17520: - this is related to dynamic creation

[GitHub] [hadoop] steveloughran opened a new pull request #2689: HADOOP-17520. Downgrade missing DDB table to unguarded client

2021-02-08 Thread GitBox
steveloughran opened a new pull request #2689: URL: https://github.com/apache/hadoop/pull/2689 If an s3a client starts up and cannot find/read the DDB table then it downgrades to being unguarded This is an automated mess

[jira] [Updated] (HADOOP-17520) S3A to downgraded to unguarded if the DDB table cannot be found

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-17520: Labels: pull-request-available (was: ) > S3A to downgraded to unguarded if the DDB table

[jira] [Work logged] (HADOOP-17520) S3A to downgraded to unguarded if the DDB table cannot be found

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17520?focusedWorklogId=549859&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549859 ] ASF GitHub Bot logged work on HADOOP-17520: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus commented on pull request #2677: HDFS-15817. Rename snapshots while marking them deleted.

2021-02-08 Thread GitBox
hadoop-yetus commented on pull request #2677: URL: https://github.com/apache/hadoop/pull/2677#issuecomment-775468282 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[jira] [Resolved] (HADOOP-17517) Non-randomized password used

2021-02-08 Thread Kihwal Lee (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee resolved HADOOP-17517. - Resolution: Invalid > Non-randomized password used > > >

[GitHub] [hadoop] zehaoc2 opened a new pull request #2690: HDFS-15821. Add metrics for in-service datanodes

2021-02-08 Thread GitBox
zehaoc2 opened a new pull request #2690: URL: https://github.com/apache/hadoop/pull/2690 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[GitHub] [hadoop] hadoop-yetus commented on pull request #2689: HADOOP-17520. Downgrade missing DDB table to unguarded client

2021-02-08 Thread GitBox
hadoop-yetus commented on pull request #2689: URL: https://github.com/apache/hadoop/pull/2689#issuecomment-775489652 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[jira] [Work logged] (HADOOP-17520) S3A to downgraded to unguarded if the DDB table cannot be found

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17520?focusedWorklogId=549921&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549921 ] ASF GitHub Bot logged work on HADOOP-17520: --- Author: ASF GitHub B

[GitHub] [hadoop] fengnanli commented on a change in pull request #2639: HDFS-15785. Datanode to support using DNS to resolve nameservices to IP addresses to get list of namenodes.

2021-02-08 Thread GitBox
fengnanli commented on a change in pull request #2639: URL: https://github.com/apache/hadoop/pull/2639#discussion_r572411463 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java ## @@ -1557,6 +1557,17 @@ public static final d

[GitHub] [hadoop] hadoop-yetus commented on pull request #2684: HADOOP-16906. Abortable

2021-02-08 Thread GitBox
hadoop-yetus commented on pull request #2684: URL: https://github.com/apache/hadoop/pull/2684#issuecomment-775499774 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[jira] [Work logged] (HADOOP-16906) Add some Abortable.abort() interface for streams etc which can be terminated

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16906?focusedWorklogId=549939&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549939 ] ASF GitHub Bot logged work on HADOOP-16906: --- Author: ASF GitHub B

[GitHub] [hadoop] LeonGao91 commented on a change in pull request #2639: HDFS-15785. Datanode to support using DNS to resolve nameservices to IP addresses to get list of namenodes.

2021-02-08 Thread GitBox
LeonGao91 commented on a change in pull request #2639: URL: https://github.com/apache/hadoop/pull/2639#discussion_r572419061 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java ## @@ -1557,6 +1557,17 @@ public static final d

[GitHub] [hadoop] LeonGao91 commented on a change in pull request #2639: HDFS-15785. Datanode to support using DNS to resolve nameservices to IP addresses to get list of namenodes.

2021-02-08 Thread GitBox
LeonGao91 commented on a change in pull request #2639: URL: https://github.com/apache/hadoop/pull/2639#discussion_r572426138 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSUtil.java ## @@ -647,6 +634,58 @@ public static String addKeySuffi

[GitHub] [hadoop] tomscut commented on pull request #2676: Make some parameters configurable for DataNodeDiskMetrics

2021-02-08 Thread GitBox
tomscut commented on pull request #2676: URL: https://github.com/apache/hadoop/pull/2676#issuecomment-775602310 @hanishakoneru Hi Hanisha Koneru, could you please review the code for me? Thanks! This is an automated messa

[GitHub] [hadoop] tomscut commented on pull request #2668: HDFS-15808. Add metrics for FSNamesystem read/write lock warnings

2021-02-08 Thread GitBox
tomscut commented on pull request #2668: URL: https://github.com/apache/hadoop/pull/2668#issuecomment-775605267 @xkrogen Hi Erik Krogen, could you please review the code? Thanks a lot! This is an automated message from th

[GitHub] [hadoop] tangzhankun merged pull request #2665: YARN-9650. Set thread names for CapacityScheduler AsyncScheduleThread

2021-02-08 Thread GitBox
tangzhankun merged pull request #2665: URL: https://github.com/apache/hadoop/pull/2665 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [hadoop] hadoop-yetus commented on pull request #2690: HDFS-15821. Add metrics for in-service datanodes

2021-02-08 Thread GitBox
hadoop-yetus commented on pull request #2690: URL: https://github.com/apache/hadoop/pull/2690#issuecomment-775627526 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] yzhangal opened a new pull request #2691: Feature/hadoop 17338

2021-02-08 Thread GitBox
yzhangal opened a new pull request #2691: URL: https://github.com/apache/hadoop/pull/2691 ## NOTICE Please create an issue in ASF JIRA before opening a pull request, and you need to set the title of the pull request which starts with the corresponding JIRA issue number. (e.g. HA

[GitHub] [hadoop] yzhangal opened a new pull request #2692: HADOOP-17338 Intermittent S3AInputStream failures: Premature end of C…

2021-02-08 Thread GitBox
yzhangal opened a new pull request #2692: URL: https://github.com/apache/hadoop/pull/2692 …ontent-Length delimited message body etc ## NOTICE Please create an issue in ASF JIRA before opening a pull request, and you need to set the title of the pull request which starts with

[jira] [Work logged] (HADOOP-17338) Intermittent S3AInputStream failures: Premature end of Content-Length delimited message body etc

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17338?focusedWorklogId=550038&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-550038 ] ASF GitHub Bot logged work on HADOOP-17338: --- Author: ASF GitHub B

[jira] [Commented] (HADOOP-16767) S3AInputStream reopening does not handle non IO exceptions properly

2021-02-08 Thread Yongjun Zhang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17281519#comment-17281519 ] Yongjun Zhang commented on HADOOP-16767: Thanks [~ste...@apache.org]! > S3AInp

[GitHub] [hadoop] yzhangal commented on pull request #2692: HADOOP-17338 Intermittent S3AInputStream failures: Premature end of C…

2021-02-08 Thread GitBox
yzhangal commented on pull request #2692: URL: https://github.com/apache/hadoop/pull/2692#issuecomment-775654938 HI @steveloughran , Thanks for reviewing and merging HADOOP-16767. This is the PR for HADOOPs-17338 on branch-2.10, the test was done previously together with HADOOP-167

[jira] [Work logged] (HADOOP-17338) Intermittent S3AInputStream failures: Premature end of Content-Length delimited message body etc

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17338?focusedWorklogId=550040&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-550040 ] ASF GitHub Bot logged work on HADOOP-17338: --- Author: ASF GitHub B

[GitHub] [hadoop] anoopsjohn commented on a change in pull request #2646: HADOOP-17038 Support disabling buffered reads in ABFS positional reads.

2021-02-08 Thread GitBox
anoopsjohn commented on a change in pull request #2646: URL: https://github.com/apache/hadoop/pull/2646#discussion_r572594015 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystemStore.java ## @@ -634,12 +638,15 @@ public AbfsI

[jira] [Work logged] (HADOOP-17038) Support disabling buffered reads in ABFS positional reads

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17038?focusedWorklogId=550041&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-550041 ] ASF GitHub Bot logged work on HADOOP-17038: --- Author: ASF GitHub B

[GitHub] [hadoop] anoopsjohn commented on a change in pull request #2646: HADOOP-17038 Support disabling buffered reads in ABFS positional reads.

2021-02-08 Thread GitBox
anoopsjohn commented on a change in pull request #2646: URL: https://github.com/apache/hadoop/pull/2646#discussion_r572594171 ## File path: hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAbfsInputStreamStatistics.java ## @@ -399,6 +406,89 @@ public

  1   2   >