[jira] [Updated] (HADOOP-17269) [JDK 11] False-positive findbugs warnings

2020-10-08 Thread Akira Ajisaka (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-17269: --- Target Version/s: 3.3.1, 3.4.0 (was: 3.4.0) > [JDK 11] False-positive findbugs warnings >

[jira] [Updated] (HADOOP-17269) [JDK 11] False-positive findbugs warnings

2020-10-08 Thread Akira Ajisaka (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-17269: --- Target Version/s: 3.4.0 > [JDK 11] False-positive findbugs warnings > -

[GitHub] [hadoop] aajisaka opened a new pull request #2374: HADOOP-17269. [JDK 11] Upgrade SpotBugs to 4.1.3 to fix false-positive warnings

2020-10-08 Thread GitBox
aajisaka opened a new pull request #2374: URL: https://github.com/apache/hadoop/pull/2374 JIRA: https://issues.apache.org/jira/browse/HADOOP-17269 Manually tested with Java 11.0.7 ``` $ mvn install -DskipTests -DskipShade $ mvn findbugs:findbugs -pl hadoop-common-project/

[jira] [Work logged] (HADOOP-17269) [JDK 11] False-positive findbugs warnings

2020-10-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17269?focusedWorklogId=497778&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-497778 ] ASF GitHub Bot logged work on HADOOP-17269: --- Author: ASF GitHub B

[jira] [Updated] (HADOOP-17269) [JDK 11] False-positive findbugs warnings

2020-10-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-17269: Labels: pull-request-available (was: ) > [JDK 11] False-positive findbugs warnings > ---

[jira] [Assigned] (HADOOP-17269) [JDK 11] False-positive findbugs warnings

2020-10-08 Thread Akira Ajisaka (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka reassigned HADOOP-17269: -- Assignee: Akira Ajisaka > [JDK 11] False-positive findbugs warnings > --

[jira] [Commented] (HADOOP-17269) [JDK 11] False-positive findbugs warnings

2020-10-08 Thread Akira Ajisaka (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17210635#comment-17210635 ] Akira Ajisaka commented on HADOOP-17269: SpotBugs 4.1.3 has been released. http

[GitHub] [hadoop] hadoop-yetus commented on pull request #2373: HDFS-15616. [SBN] Disable Observers to trigger edit Log roll

2020-10-08 Thread GitBox
hadoop-yetus commented on pull request #2373: URL: https://github.com/apache/hadoop/pull/2373#issuecomment-705988371 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] dongjoon-hyun commented on pull request #2315: HADOOP-17258. Make MagicS3GuardCommitter overwrite the existing pendingSet file if exists

2020-10-08 Thread GitBox
dongjoon-hyun commented on pull request #2315: URL: https://github.com/apache/hadoop/pull/2315#issuecomment-705976033 Thank you for taking over this, @steveloughran . This is an automated message from the Apache Git Service.

[jira] [Work logged] (HADOOP-17258) MagicS3GuardCommitter fails with `pendingset` already exists

2020-10-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17258?focusedWorklogId=497763&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-497763 ] ASF GitHub Bot logged work on HADOOP-17258: --- Author: ASF GitHub B

[jira] [Work logged] (HADOOP-17258) MagicS3GuardCommitter fails with `pendingset` already exists

2020-10-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17258?focusedWorklogId=497761&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-497761 ] ASF GitHub Bot logged work on HADOOP-17258: --- Author: ASF GitHub B

[GitHub] [hadoop] dongjoon-hyun closed pull request #2315: HADOOP-17258. Make MagicS3GuardCommitter overwrite the existing pendingSet file if exists

2020-10-08 Thread GitBox
dongjoon-hyun closed pull request #2315: URL: https://github.com/apache/hadoop/pull/2315 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[GitHub] [hadoop] symious opened a new pull request #2373: HDFS-15616. [SBN] Disable Observers to trigger edit Log roll

2020-10-08 Thread GitBox
symious opened a new pull request #2373: URL: https://github.com/apache/hadoop/pull/2373 Jira ticket: https://issues.apache.org/jira/browse/HDFS-15616 This is an automated message from the Apache Git Service. To respond to th

[GitHub] [hadoop] hadoop-yetus commented on pull request #2372: MAPREDUCE-7301: Expose Mini MR Cluster attribute for testing

2020-10-08 Thread GitBox
hadoop-yetus commented on pull request #2372: URL: https://github.com/apache/hadoop/pull/2372#issuecomment-705922259 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[GitHub] [hadoop] ferhui commented on pull request #2363: HDFS-13293. RBF: The RouterRPCServer should transfer client IP via CallerContext to NamenodeRpcServer

2020-10-08 Thread GitBox
ferhui commented on pull request #2363: URL: https://github.com/apache/hadoop/pull/2363#issuecomment-705908873 @goiri @aajisaka Thanks for review! Will merge later if no further comments. This is an automated message from the

[jira] [Work logged] (HADOOP-17287) Support new Instance by non default constructor by ReflectionUtils

2020-10-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17287?focusedWorklogId=497687&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-497687 ] ASF GitHub Bot logged work on HADOOP-17287: --- Author: ASF GitHub B

[GitHub] [hadoop] maobaolong commented on pull request #2341: HADOOP-17287. Support new Instance by non default constructor by ReflectionUtils

2020-10-08 Thread GitBox
maobaolong commented on pull request #2341: URL: https://github.com/apache/hadoop/pull/2341#issuecomment-705904691 @sunchao Thanks for your review and merge this PR. This is an automated message from the Apache Git Service. T

[jira] [Commented] (HADOOP-17068) client fails forever when namenode ipaddr changed

2020-10-08 Thread Mingliang Liu (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17210542#comment-17210542 ] Mingliang Liu commented on HADOOP-17068: Does this apply to Hadoop 2? If so cou

[GitHub] [hadoop] swaroopak opened a new pull request #2372: MAPREDUCE-7301: Expose Mini MR Cluster attribute for testing

2020-10-08 Thread GitBox
swaroopak opened a new pull request #2372: URL: https://github.com/apache/hadoop/pull/2372 ## NOTICE Please create an issue in ASF JIRA before opening a pull request, and you need to set the title of the pull request which starts with the corresponding JIRA issue number. (e.g. H

[GitHub] [hadoop] hadoop-yetus commented on pull request #2370: HDFS-15614. Initialize snapshot trash root during NameNode startup if enabled

2020-10-08 Thread GitBox
hadoop-yetus commented on pull request #2370: URL: https://github.com/apache/hadoop/pull/2370#issuecomment-705817726 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[jira] [Work logged] (HADOOP-17258) MagicS3GuardCommitter fails with `pendingset` already exists

2020-10-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17258?focusedWorklogId=497565&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-497565 ] ASF GitHub Bot logged work on HADOOP-17258: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus commented on pull request #2371: HADOOP-17258. Make MagicS3GuardCommitter overwrite the existing pendingSet file if exists

2020-10-08 Thread GitBox
hadoop-yetus commented on pull request #2371: URL: https://github.com/apache/hadoop/pull/2371#issuecomment-705803820 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[jira] [Work logged] (HADOOP-17258) MagicS3GuardCommitter fails with `pendingset` already exists

2020-10-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17258?focusedWorklogId=497522&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-497522 ] ASF GitHub Bot logged work on HADOOP-17258: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on pull request #2371: HADOOP-17258. Make MagicS3GuardCommitter overwrite the existing pendingSet file if exists

2020-10-08 Thread GitBox
steveloughran commented on pull request #2371: URL: https://github.com/apache/hadoop/pull/2371#issuecomment-705767136 testing against AWS london in progress. new tests all happy, lets see about terasort &c This is an automat

[GitHub] [hadoop] steveloughran commented on pull request #2315: HADOOP-17258. Make MagicS3GuardCommitter overwrite the existing pendingSet file if exists

2020-10-08 Thread GitBox
steveloughran commented on pull request #2315: URL: https://github.com/apache/hadoop/pull/2315#issuecomment-705766677 #2371 adds the new test case to verify that the last committed task attempt is the one which succeeds This

[jira] [Work logged] (HADOOP-17258) MagicS3GuardCommitter fails with `pendingset` already exists

2020-10-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17258?focusedWorklogId=497521&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-497521 ] ASF GitHub Bot logged work on HADOOP-17258: --- Author: ASF GitHub B

[jira] [Work logged] (HADOOP-17258) MagicS3GuardCommitter fails with `pendingset` already exists

2020-10-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17258?focusedWorklogId=497520&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-497520 ] ASF GitHub Bot logged work on HADOOP-17258: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran opened a new pull request #2371: HADOOP-17258. Make MagicS3GuardCommitter overwrite the existing pendingSet file if exists

2020-10-08 Thread GitBox
steveloughran opened a new pull request #2371: URL: https://github.com/apache/hadoop/pull/2371 Built on #2315 -adds the test This is an automated message from the Apache Git Service. To respond to the message, please log

[jira] [Work logged] (HADOOP-17258) MagicS3GuardCommitter fails with `pendingset` already exists

2020-10-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17258?focusedWorklogId=497518&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-497518 ] ASF GitHub Bot logged work on HADOOP-17258: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on pull request #2315: HADOOP-17258. Make MagicS3GuardCommitter overwrite the existing pendingSet file if exists

2020-10-08 Thread GitBox
steveloughran commented on pull request #2315: URL: https://github.com/apache/hadoop/pull/2315#issuecomment-705765571 ok, adding a new test for this. production side of patch will be your work., I think I'll also add the task attempt ID to the pendingset file so you can see which task atte

[jira] [Work logged] (HADOOP-17271) S3A statistics to support IOStatistics

2020-10-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17271?focusedWorklogId=497478&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-497478 ] ASF GitHub Bot logged work on HADOOP-17271: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus commented on pull request #2324: HADOOP-17271. S3A statistics to support IOStatistics

2020-10-08 Thread GitBox
hadoop-yetus commented on pull request #2324: URL: https://github.com/apache/hadoop/pull/2324#issuecomment-705731165 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[GitHub] [hadoop] smengcl commented on a change in pull request #2370: HDFS-15614. Initialize snapshot trash root during NameNode startup if enabled

2020-10-08 Thread GitBox
smengcl commented on a change in pull request #2370: URL: https://github.com/apache/hadoop/pull/2370#discussion_r501900056 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java ## @@ -8524,6 +8530,39 @@ void checkA

[GitHub] [hadoop] smengcl opened a new pull request #2370: HDFS-15614. Initialize snapshot trash root during NameNode startup if enabled

2020-10-08 Thread GitBox
smengcl opened a new pull request #2370: URL: https://github.com/apache/hadoop/pull/2370 https://issues.apache.org/jira/browse/HDFS-15614 Added `checkAndProvisionSnapshotTrashRoots` in `FSNamesystem`. Triggered in `NameNode` on startup if `dfs.namenode.snapshot.trashroot.enabled` set

[jira] [Work logged] (HADOOP-17293) S3A to always probe S3 in S3A getFileStatus on non-auth paths

2020-10-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17293?focusedWorklogId=497418&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-497418 ] ASF GitHub Bot logged work on HADOOP-17293: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus commented on pull request #2361: HADOOP-17293. Aways probe S3 when opening/getFileStatus on non-auth paths

2020-10-08 Thread GitBox
hadoop-yetus commented on pull request #2361: URL: https://github.com/apache/hadoop/pull/2361#issuecomment-705660591 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[GitHub] [hadoop] steveloughran closed pull request #2364: Merge pull request #1 from apache/trunk

2020-10-08 Thread GitBox
steveloughran closed pull request #2364: URL: https://github.com/apache/hadoop/pull/2364 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[GitHub] [hadoop] steveloughran commented on pull request #2364: Merge pull request #1 from apache/trunk

2020-10-08 Thread GitBox
steveloughran commented on pull request #2364: URL: https://github.com/apache/hadoop/pull/2364#issuecomment-705644115 while contributions to open source projects are always welcome, please create the apache first, then submit PRs which reference them. Thanks --

[GitHub] [hadoop] aajisaka commented on pull request #2363: HDFS-13293. RBF: The RouterRPCServer should transfer client IP via CallerContext to NamenodeRpcServer

2020-10-08 Thread GitBox
aajisaka commented on pull request #2363: URL: https://github.com/apache/hadoop/pull/2363#issuecomment-705642911 The filesystem contract test failures are related to HADOOP-17281. This is an automated message from the Apache

[GitHub] [hadoop] steveloughran commented on a change in pull request #2369: HADOOP-17301. ABFS: Fix bug introduced in HADOOP-16852 which reports read-ahead error back

2020-10-08 Thread GitBox
steveloughran commented on a change in pull request #2369: URL: https://github.com/apache/hadoop/pull/2369#discussion_r501695425 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/ReadBufferManager.java ## @@ -242,13 +243,29 @@ private

[jira] [Work logged] (HADOOP-17301) ABFS: Fix bug introduced in HADOOP-16852 which reports read-ahead error back

2020-10-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17301?focusedWorklogId=497403&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-497403 ] ASF GitHub Bot logged work on HADOOP-17301: --- Author: ASF GitHub B

[jira] [Work logged] (HADOOP-17293) S3A to always probe S3 in S3A getFileStatus on non-auth paths

2020-10-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17293?focusedWorklogId=497400&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-497400 ] ASF GitHub Bot logged work on HADOOP-17293: --- Author: ASF GitHub B

[jira] [Work logged] (HADOOP-17293) S3A to always probe S3 in S3A getFileStatus on non-auth paths

2020-10-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17293?focusedWorklogId=497398&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-497398 ] ASF GitHub Bot logged work on HADOOP-17293: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on pull request #2361: HADOOP-17293. Aways probe S3 when opening/getFileStatus on non-auth paths

2020-10-08 Thread GitBox
steveloughran commented on pull request #2361: URL: https://github.com/apache/hadoop/pull/2361#issuecomment-705636416 patch merged to 3.3.x and trunk, thanks! This is an automated message from the Apache Git Service. To respo

[GitHub] [hadoop] steveloughran closed pull request #2361: HADOOP-17293. Aways probe S3 when opening/getFileStatus on non-auth paths

2020-10-08 Thread GitBox
steveloughran closed pull request #2361: URL: https://github.com/apache/hadoop/pull/2361 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[GitHub] [hadoop] steveloughran commented on a change in pull request #2361: HADOOP-17293. Aways probe S3 when opening/getFileStatus on non-auth paths

2020-10-08 Thread GitBox
steveloughran commented on a change in pull request #2361: URL: https://github.com/apache/hadoop/pull/2361#discussion_r501799545 ## File path: hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java ## @@ -2956,55 +2956,30 @@ S3AFileStatus innerGetFil

[jira] [Work logged] (HADOOP-17293) S3A to always probe S3 in S3A getFileStatus on non-auth paths

2020-10-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17293?focusedWorklogId=497397&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-497397 ] ASF GitHub Bot logged work on HADOOP-17293: --- Author: ASF GitHub B

[jira] [Work logged] (HADOOP-17272) ABFS Streams to support IOStatistics API

2020-10-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17272?focusedWorklogId=497396&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-497396 ] ASF GitHub Bot logged work on HADOOP-17272: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus commented on pull request #2353: HADOOP-17272. ABFS Streams to support IOStatistics API

2020-10-08 Thread GitBox
hadoop-yetus commented on pull request #2353: URL: https://github.com/apache/hadoop/pull/2353#issuecomment-705635541 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] mukund-thakur commented on pull request #2368: Hadoop-17296. ABFS: Force reads to be always of buffer size

2020-10-08 Thread GitBox
mukund-thakur commented on pull request #2368: URL: https://github.com/apache/hadoop/pull/2368#issuecomment-705634731 Reviewed the production code. Will review the test code as well once I get some time. This is an automate

[GitHub] [hadoop] mukund-thakur commented on a change in pull request #2368: Hadoop-17296. ABFS: Force reads to be always of buffer size

2020-10-08 Thread GitBox
mukund-thakur commented on a change in pull request #2368: URL: https://github.com/apache/hadoop/pull/2368#discussion_r501750162 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsInputStream.java ## @@ -89,9 +91,24 @@ public AbfsI

[jira] [Work logged] (HADOOP-17293) S3A to always probe S3 in S3A getFileStatus on non-auth paths

2020-10-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17293?focusedWorklogId=497382&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-497382 ] ASF GitHub Bot logged work on HADOOP-17293: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus commented on pull request #2361: HADOOP-17293. Aways probe S3 when opening/getFileStatus on non-auth paths

2020-10-08 Thread GitBox
hadoop-yetus commented on pull request #2361: URL: https://github.com/apache/hadoop/pull/2361#issuecomment-705619305 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[jira] [Commented] (HADOOP-17293) S3A to always probe S3 in S3A getFileStatus on non-auth paths

2020-10-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17210255#comment-17210255 ] Steve Loughran commented on HADOOP-17293: - Final PR completely ignores TTL on g

[jira] [Resolved] (HADOOP-17293) S3A to always probe S3 in S3A getFileStatus on non-auth paths

2020-10-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17293. - Fix Version/s: 3.3.1 Resolution: Fixed > S3A to always probe S3 in S3A getFileS

[jira] [Updated] (HADOOP-17293) S3A to always probe S3 in S3A getFileStatus on non-auth paths

2020-10-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17293: Summary: S3A to always probe S3 in S3A getFileStatus on non-auth paths (was: refreshing

[jira] [Work logged] (HADOOP-16830) Add public IOStatistics API

2020-10-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16830?focusedWorklogId=497369&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-497369 ] ASF GitHub Bot logged work on HADOOP-16830: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on pull request #2323: HADOOP-16830. Add public IOStatistics API.

2020-10-08 Thread GitBox
steveloughran commented on pull request #2323: URL: https://github.com/apache/hadoop/pull/2323#issuecomment-705609349 javac failures on newly deprecated static fs.impl.FutureIOSupport methods I'm retaining to avoiding breaking things; checkstyle on 4 javadoc lines which are too wide becaus

[jira] [Work logged] (HADOOP-17293) refreshing S3Guard records after TTL-triggered-HEAD breaks some workflows

2020-10-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17293?focusedWorklogId=497362&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-497362 ] ASF GitHub Bot logged work on HADOOP-17293: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on a change in pull request #2361: HADOOP-17293. Aways probe S3 when opening/getFileStatus on non-auth paths

2020-10-08 Thread GitBox
steveloughran commented on a change in pull request #2361: URL: https://github.com/apache/hadoop/pull/2361#discussion_r501759422 ## File path: hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3ARemoteFileChanged.java ## @@ -326,7 +326,7 @@ protected Path pa

[GitHub] [hadoop] steveloughran commented on a change in pull request #2361: HADOOP-17293. Aways probe S3 when opening/getFileStatus on non-auth paths

2020-10-08 Thread GitBox
steveloughran commented on a change in pull request #2361: URL: https://github.com/apache/hadoop/pull/2361#discussion_r501703520 ## File path: hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3ARemoteFileChanged.java ## @@ -326,7 +326,7 @@ protected Path pa

[jira] [Work logged] (HADOOP-17293) refreshing S3Guard records after TTL-triggered-HEAD breaks some workflows

2020-10-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17293?focusedWorklogId=497356&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-497356 ] ASF GitHub Bot logged work on HADOOP-17293: --- Author: ASF GitHub B

[jira] [Commented] (HADOOP-16990) Update Mockserver

2020-10-08 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17210219#comment-17210219 ] Hadoop QA commented on HADOOP-16990: | (/) *{color:green}+1 overall{color}* | \\ \\

[jira] [Work logged] (HADOOP-17293) refreshing S3Guard records after TTL-triggered-HEAD breaks some workflows

2020-10-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17293?focusedWorklogId=497323&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-497323 ] ASF GitHub Bot logged work on HADOOP-17293: --- Author: ASF GitHub B

[GitHub] [hadoop] bgaborg commented on pull request #2361: HADOOP-17293. Aways probe S3 when opening/getFileStatus on non-auth paths

2020-10-08 Thread GitBox
bgaborg commented on pull request #2361: URL: https://github.com/apache/hadoop/pull/2361#issuecomment-70714 (also, tests run against Ireland was successful) This is an automated message from the Apache Git Service. To res

[jira] [Work logged] (HADOOP-17293) refreshing S3Guard records after TTL-triggered-HEAD breaks some workflows

2020-10-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17293?focusedWorklogId=497321&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-497321 ] ASF GitHub Bot logged work on HADOOP-17293: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on a change in pull request #2361: HADOOP-17293. Aways probe S3 when opening/getFileStatus on non-auth paths

2020-10-08 Thread GitBox
steveloughran commented on a change in pull request #2361: URL: https://github.com/apache/hadoop/pull/2361#discussion_r501703520 ## File path: hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3ARemoteFileChanged.java ## @@ -326,7 +326,7 @@ protected Path pa

[jira] [Work logged] (HADOOP-17301) ABFS: Fix bug introduced in HADOOP-16852 which reports read-ahead error back

2020-10-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17301?focusedWorklogId=497318&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-497318 ] ASF GitHub Bot logged work on HADOOP-17301: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #2369: HADOOP-17301. ABFS: Fix bug introduced in HADOOP-16852 which reports read-ahead error back

2020-10-08 Thread GitBox
hadoop-yetus removed a comment on pull request #2369: URL: https://github.com/apache/hadoop/pull/2369#issuecomment-705303355 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:|

[jira] [Commented] (HADOOP-16990) Update Mockserver

2020-10-08 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17210163#comment-17210163 ] Hadoop QA commented on HADOOP-16990: | (x) *{color:red}-1 overall{color}* | \\ \\ |

[jira] [Updated] (HADOOP-16990) Update Mockserver

2020-10-08 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HADOOP-16990: -- Attachment: (was: HADOOP-16990-branch-3.1.003.patch) > Update Mockserver > --

[jira] [Updated] (HADOOP-16990) Update Mockserver

2020-10-08 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HADOOP-16990: -- Attachment: HADOOP-16990-branch-3.1.004.patch > Update Mockserver > -

[jira] [Work logged] (HADOOP-17272) ABFS Streams to support IOStatistics API

2020-10-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17272?focusedWorklogId=497275&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-497275 ] ASF GitHub Bot logged work on HADOOP-17272: --- Author: ASF GitHub B

[GitHub] [hadoop] mehakmeet commented on a change in pull request #2353: HADOOP-17272. ABFS Streams to support IOStatistics API

2020-10-08 Thread GitBox
mehakmeet commented on a change in pull request #2353: URL: https://github.com/apache/hadoop/pull/2353#discussion_r501670851 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsInputStream.java ## @@ -279,7 +289,15 @@ int readRemote

[GitHub] [hadoop] hadoop-yetus commented on pull request #2368: Hadoop-17296. ABFS: Force reads to be always of buffer size

2020-10-08 Thread GitBox
hadoop-yetus commented on pull request #2368: URL: https://github.com/apache/hadoop/pull/2368#issuecomment-705526565 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[GitHub] [hadoop] steveloughran merged pull request #1993: HADOOP-17021. Add concat fs command

2020-10-08 Thread GitBox
steveloughran merged pull request #1993: URL: https://github.com/apache/hadoop/pull/1993 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #2323: HADOOP-16830. Add public IOStatistics API.

2020-10-08 Thread GitBox
hadoop-yetus removed a comment on pull request #2323: URL: https://github.com/apache/hadoop/pull/2323#issuecomment-704549467 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[jira] [Work logged] (HADOOP-16830) Add public IOStatistics API

2020-10-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16830?focusedWorklogId=497237&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-497237 ] ASF GitHub Bot logged work on HADOOP-16830: --- Author: ASF GitHub B

[GitHub] [hadoop] bgaborg commented on a change in pull request #2361: HADOOP-17293. Aways probe S3 when opening/getFileStatus on non-auth paths

2020-10-08 Thread GitBox
bgaborg commented on a change in pull request #2361: URL: https://github.com/apache/hadoop/pull/2361#discussion_r501626796 ## File path: hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java ## @@ -2956,55 +2956,30 @@ S3AFileStatus innerGetFileStatu

[jira] [Work logged] (HADOOP-17293) refreshing S3Guard records after TTL-triggered-HEAD breaks some workflows

2020-10-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17293?focusedWorklogId=497232&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-497232 ] ASF GitHub Bot logged work on HADOOP-17293: --- Author: ASF GitHub B

[jira] [Updated] (HADOOP-17296) ABFS: Allow Random Reads to be of Buffer Size

2020-10-08 Thread Sneha Vijayarajan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sneha Vijayarajan updated HADOOP-17296: --- Description: ADLS Gen2/ABFS driver is optimized to read only the bytes that are req

[jira] [Work logged] (HADOOP-17301) ABFS: Fix bug introduced in HADOOP-16852 which reports read-ahead error back

2020-10-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17301?focusedWorklogId=497216&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-497216 ] ASF GitHub Bot logged work on HADOOP-17301: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus commented on pull request #2369: HADOOP-17301. ABFS: Fix bug introduced in HADOOP-16852 which reports read-ahead error back

2020-10-08 Thread GitBox
hadoop-yetus commented on pull request #2369: URL: https://github.com/apache/hadoop/pull/2369#issuecomment-705474826 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[GitHub] [hadoop] steveloughran commented on pull request #2149: HADOOP-13230. S3A to optionally retain directory markers

2020-10-08 Thread GitBox
steveloughran commented on pull request #2149: URL: https://github.com/apache/hadoop/pull/2149#issuecomment-705455586 @liuml07 Looking forward to a Hadoop 2 backport! I'll do a hadoop-2 fixup of list time changes, but if you look #2310 delete gets complex, especially handling partial

[jira] [Work logged] (HADOOP-13230) S3A to optionally retain directory markers

2020-10-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13230?focusedWorklogId=497205&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-497205 ] ASF GitHub Bot logged work on HADOOP-13230: --- Author: ASF GitHub B

[jira] [Updated] (HADOOP-17021) Add concat fs command

2020-10-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17021: Fix Version/s: 3.3.1 Resolution: Fixed Status: Resolved (was: Patch Ava

[jira] [Updated] (HADOOP-17021) Add concat fs command

2020-10-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17021: Component/s: fs > Add concat fs command > - > > Key:

[jira] [Commented] (HADOOP-17021) Add concat fs command

2020-10-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17210107#comment-17210107 ] Steve Loughran commented on HADOOP-17021: - +1, merged to branch 3.3 & Trunk >

[jira] [Updated] (HADOOP-16990) Update Mockserver

2020-10-08 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HADOOP-16990: -- Attachment: HADOOP-16990-branch-3.1.003.patch > Update Mockserver > -

[jira] [Work logged] (HADOOP-17021) Add concat fs command

2020-10-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17021?focusedWorklogId=497203&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-497203 ] ASF GitHub Bot logged work on HADOOP-17021: --- Author: ASF GitHub B

[jira] [Commented] (HADOOP-16990) Update Mockserver

2020-10-08 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17210102#comment-17210102 ] Hadoop QA commented on HADOOP-16990: | (x) *{color:red}-1 overall{color}* | \\ \\ |

[GitHub] [hadoop] lokeshj1703 merged pull request #2365: HDFS-15610 Reduced datanode upgrade/hardlink thread from 12 to 6

2020-10-08 Thread GitBox
lokeshj1703 merged pull request #2365: URL: https://github.com/apache/hadoop/pull/2365 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[jira] [Updated] (HADOOP-16990) Update Mockserver

2020-10-08 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HADOOP-16990: -- Attachment: HADOOP-16990-branch-3.3.002.patch > Update Mockserver > -

[jira] [Work logged] (HADOOP-17301) ABFS: Fix bug introduced in HADOOP-16852 which reports read-ahead error back

2020-10-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17301?focusedWorklogId=497191&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-497191 ] ASF GitHub Bot logged work on HADOOP-17301: --- Author: ASF GitHub B

[GitHub] [hadoop] snvijaya commented on pull request #2369: HADOOP-17301. ABFS: Fix bug introduced in HADOOP-16852 which reports read-ahead error back

2020-10-08 Thread GitBox
snvijaya commented on pull request #2369: URL: https://github.com/apache/hadoop/pull/2369#issuecomment-705436858 Tests were run on accounts on East US 2 region. ### HNS-OAuth [INFO] Results: [INFO] [INFO] Tests run: 88, Failures: 0, Errors:

[jira] [Comment Edited] (HADOOP-17296) ABFS: Allow Random Reads to be of Buffer Size

2020-10-08 Thread Sneha Vijayarajan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17209897#comment-17209897 ] Sneha Vijayarajan edited comment on HADOOP-17296 at 10/8/20, 9:03 AM: ---

[GitHub] [hadoop] ferhui commented on pull request #2363: HDFS-13293. RBF: The RouterRPCServer should transfer client IP via CallerContext to NamenodeRpcServer

2020-10-08 Thread GitBox
ferhui commented on pull request #2363: URL: https://github.com/apache/hadoop/pull/2363#issuecomment-705414755 Failed tests are unrelated and pass locally This is an automated message from the Apache Git Service. To respond t

[GitHub] [hadoop] hadoop-yetus commented on pull request #2363: HDFS-13293. RBF: The RouterRPCServer should transfer client IP via CallerContext to NamenodeRpcServer

2020-10-08 Thread GitBox
hadoop-yetus commented on pull request #2363: URL: https://github.com/apache/hadoop/pull/2363#issuecomment-705398703 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] lokeshj1703 commented on pull request #2365: HDFS-15610 Reduced datanode upgrade/hardlink thread from 12 to 6

2020-10-08 Thread GitBox
lokeshj1703 commented on pull request #2365: URL: https://github.com/apache/hadoop/pull/2365#issuecomment-705382799 The failed tests pass locally. This is an automated message from the Apache Git Service. To respond to the me