[jira] [Work logged] (HADOOP-17572) [branch-2.10] Docker image build fails due to the removal of openjdk-7-jdk package

2021-03-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17572?focusedWorklogId=563534&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-563534 ] ASF GitHub Bot logged work on HADOOP-17572: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus commented on pull request #2755: HADOOP-17572. [branch-2.10] Docker image build fails due to the removal of openjdk-7-jdk package

2021-03-09 Thread GitBox
hadoop-yetus commented on pull request #2755: URL: https://github.com/apache/hadoop/pull/2755#issuecomment-795036761 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://ci-hadoop.apac

[jira] [Updated] (HADOOP-17572) [branch-2.10] Docker image build fails due to the removal of openjdk-7-jdk package

2021-03-09 Thread Akira Ajisaka (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-17572: --- Affects Version/s: 2.10.1 > [branch-2.10] Docker image build fails due to the removal of op

[jira] [Updated] (HADOOP-17572) [branch-2.10] Docker image build fails due to the removal of openjdk-7-jdk package

2021-03-09 Thread Akira Ajisaka (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-17572: --- Status: Patch Available (was: Open) > [branch-2.10] Docker image build fails due to the re

[jira] [Updated] (HADOOP-17572) [branch-2.10] Docker image build fails due to the removal of openjdk-7-jdk package

2021-03-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-17572: Labels: pull-request-available (was: ) > [branch-2.10] Docker image build fails due to t

[jira] [Work logged] (HADOOP-17572) [branch-2.10] Docker image build fails due to the removal of openjdk-7-jdk package

2021-03-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17572?focusedWorklogId=563533&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-563533 ] ASF GitHub Bot logged work on HADOOP-17572: --- Author: ASF GitHub B

[GitHub] [hadoop] aajisaka opened a new pull request #2755: HADOOP-17572. [branch-2.10] Docker image build fails due to the removal of openjdk-7-jdk package

2021-03-09 Thread GitBox
aajisaka opened a new pull request #2755: URL: https://github.com/apache/hadoop/pull/2755 - Use Zulu OpenJDK 7. - Enable multijdk option to compile with both Java 7 and 8 in GitHub PR. JIRA: https://issues.apache.org/jira/browse/HADOOP-17572 --

[jira] [Assigned] (HADOOP-17572) [branch-2.10] Docker image build fails due to the removal of openjdk-7-jdk package

2021-03-09 Thread Akira Ajisaka (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka reassigned HADOOP-17572: -- Assignee: Akira Ajisaka > [branch-2.10] Docker image build fails due to the removal

[GitHub] [hadoop] jianghuazhu commented on pull request #2741: HDFS-15855.Solve the problem of incorrect EC progress when loading FsImage.

2021-03-09 Thread GitBox
jianghuazhu commented on pull request #2741: URL: https://github.com/apache/hadoop/pull/2741#issuecomment-795023854 @goiri , thanks for your comment. I will add some unit tests. This is an automated message from the Apach

[jira] [Commented] (HADOOP-17571) Upgrade com.fasterxml.woodstox:woodstox-core for security reasons

2021-03-09 Thread Viraj Jasani (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17298592#comment-17298592 ] Viraj Jasani commented on HADOOP-17571: --- Thank you [~liuml07] > Upgrade com.fast

[jira] [Commented] (HADOOP-17572) [branch-2.10] Docker image build fails due to the removal of openjdk-7-jdk package

2021-03-09 Thread Akira Ajisaka (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17298577#comment-17298577 ] Akira Ajisaka commented on HADOOP-17572: I'm trying to create a patch to use Zu

[jira] [Created] (HADOOP-17572) [branch-2.10] Docker image build fails due to the removal of openjdk-7-jdk package

2021-03-09 Thread Akira Ajisaka (Jira)
Akira Ajisaka created HADOOP-17572: -- Summary: [branch-2.10] Docker image build fails due to the removal of openjdk-7-jdk package Key: HADOOP-17572 URL: https://issues.apache.org/jira/browse/HADOOP-17572

[jira] [Commented] (HADOOP-17571) Upgrade com.fasterxml.woodstox:woodstox-core for security reasons

2021-03-09 Thread Mingliang Liu (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17298551#comment-17298551 ] Mingliang Liu commented on HADOOP-17571: Thank you filing this one. I have adde

[jira] [Assigned] (HADOOP-17571) Upgrade com.fasterxml.woodstox:woodstox-core for security reasons

2021-03-09 Thread Mingliang Liu (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu reassigned HADOOP-17571: -- Assignee: Viraj Jasani > Upgrade com.fasterxml.woodstox:woodstox-core for security r

[GitHub] [hadoop] ferhui commented on pull request #2746: HDFS-15875. Check whether file is being truncated before truncate

2021-03-09 Thread GitBox
ferhui commented on pull request #2746: URL: https://github.com/apache/hadoop/pull/2746#issuecomment-794943728 @ayushtkn Thanks for review ! Merged This is an automated message from the Apache Git Service. To respond to t

[GitHub] [hadoop] ferhui merged pull request #2746: HDFS-15875. Check whether file is being truncated before truncate

2021-03-09 Thread GitBox
ferhui merged pull request #2746: URL: https://github.com/apache/hadoop/pull/2746 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[jira] [Work logged] (HADOOP-17536) Suport for customer provided encrption key

2021-03-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17536?focusedWorklogId=563506&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-563506 ] ASF GitHub Bot logged work on HADOOP-17536: --- Author: ASF GitHub B

[GitHub] [hadoop] vinaysbadami commented on a change in pull request #2707: HADOOP-17536. ABFS: Supporting customer provided encryption key

2021-03-09 Thread GitBox
vinaysbadami commented on a change in pull request #2707: URL: https://github.com/apache/hadoop/pull/2707#discussion_r591074440 ## File path: hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAbfsClientCustomerProvidedKey.java ## @@ -0,0 +1,623 @@ +/**

[jira] [Created] (HADOOP-17571) Upgrade com.fasterxml.woodstox:woodstox-core for security reasons

2021-03-09 Thread Viraj Jasani (Jira)
Viraj Jasani created HADOOP-17571: - Summary: Upgrade com.fasterxml.woodstox:woodstox-core for security reasons Key: HADOOP-17571 URL: https://issues.apache.org/jira/browse/HADOOP-17571 Project: Hadoop

[jira] [Commented] (HADOOP-17571) Upgrade com.fasterxml.woodstox:woodstox-core for security reasons

2021-03-09 Thread Viraj Jasani (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17298536#comment-17298536 ] Viraj Jasani commented on HADOOP-17571: --- Can someone please provide me the contri

[jira] [Work logged] (HADOOP-16870) Use spotbugs-maven-plugin instead of findbugs-maven-plugin

2021-03-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16870?focusedWorklogId=563487&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-563487 ] ASF GitHub Bot logged work on HADOOP-16870: --- Author: ASF GitHub B

[GitHub] [hadoop] asf-cloudbees-jenkins-ci-hadoop[bot] commented on pull request #2753: HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin

2021-03-09 Thread GitBox
asf-cloudbees-jenkins-ci-hadoop[bot] commented on pull request #2753: URL: https://github.com/apache/hadoop/pull/2753#issuecomment-794841846 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at

[GitHub] [hadoop] ferhui commented on pull request #2746: HDFS-15875. Check whether file is being truncated before truncate

2021-03-09 Thread GitBox
ferhui commented on pull request #2746: URL: https://github.com/apache/hadoop/pull/2746#issuecomment-794677960 @ayushtkn Any further questions? Thanks! This is an automated message from the Apache Git Service. To respond to t

[GitHub] [hadoop] sungpeo commented on a change in pull request #1942: MAPREDUCE-7270. TestHistoryViewerPrinter could be failed when the locale isn't English.

2021-03-09 Thread GitBox
sungpeo commented on a change in pull request #1942: URL: https://github.com/apache/hadoop/pull/1942#discussion_r590794948 ## File path: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/jobhistory/TestHistoryViewer

[jira] [Work logged] (HADOOP-16080) hadoop-aws does not work with hadoop-client-api

2021-03-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16080?focusedWorklogId=56&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-56 ] ASF GitHub Bot logged work on HADOOP-16080: --- Author: ASF GitHub B

[GitHub] [hadoop] sunchao commented on pull request #2575: HADOOP-16080. hadoop-aws does not work with hadoop-client-api

2021-03-09 Thread GitBox
sunchao commented on pull request #2575: URL: https://github.com/apache/hadoop/pull/2575#issuecomment-794470421 thanks @steveloughran ! This is an automated message from the Apache Git Service. To respond to the message, plea

[jira] [Work logged] (HADOOP-17464) Create hadoop-compression module

2021-03-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17464?focusedWorklogId=563311&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-563311 ] ASF GitHub Bot logged work on HADOOP-17464: --- Author: ASF GitHub B

[GitHub] [hadoop] viirya commented on pull request #2611: HADOOP-17464. Create hadoop-compression module

2021-03-09 Thread GitBox
viirya commented on pull request #2611: URL: https://github.com/apache/hadoop/pull/2611#issuecomment-794389089 > What do we think will break? Specifically will people be expecting snappy/lz4 to be there? I think so. Previously people may include hadoop-common and expect snappy/lz4 c

[jira] [Work logged] (HADOOP-16080) hadoop-aws does not work with hadoop-client-api

2021-03-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16080?focusedWorklogId=563310&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-563310 ] ASF GitHub Bot logged work on HADOOP-16080: --- Author: ASF GitHub B

[jira] [Work logged] (HADOOP-17472) Upgrade hadoop default version in hadoop.tools.dynamometer.TestDynamometerInfra

2021-03-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17472?focusedWorklogId=563309&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-563309 ] ASF GitHub Bot logged work on HADOOP-17472: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran merged pull request #2575: HADOOP-16080. hadoop-aws does not work with hadoop-client-api

2021-03-09 Thread GitBox
steveloughran merged pull request #2575: URL: https://github.com/apache/hadoop/pull/2575 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[GitHub] [hadoop] viirya commented on pull request #2622: HADOOP-17472. Upgrade hadoop default version in hadoop.tools.dynamometer.TestDynamometerInfra

2021-03-09 Thread GitBox
viirya commented on pull request #2622: URL: https://github.com/apache/hadoop/pull/2622#issuecomment-794382273 Does #2537 help this issue? Let me sync up with trunk. This is an automated message from the Apache Git Service. T

[jira] [Work logged] (HADOOP-16080) hadoop-aws does not work with hadoop-client-api

2021-03-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16080?focusedWorklogId=563308&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-563308 ] ASF GitHub Bot logged work on HADOOP-16080: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on pull request #2575: HADOOP-16080. hadoop-aws does not work with hadoop-client-api

2021-03-09 Thread GitBox
steveloughran commented on pull request #2575: URL: https://github.com/apache/hadoop/pull/2575#issuecomment-794380727 I've been running on trunk for a while, everything is happy there. Going to +1 and merge this backport

[jira] [Commented] (HADOOP-17528) Not closing an SFTP File System instance prevents JVM from exiting.

2021-03-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17298302#comment-17298302 ] Steve Loughran commented on HADOOP-17528: - patch is in 3.3.1+. Thanks > Not cl

[jira] [Updated] (HADOOP-17528) Not closing an SFTP File System instance prevents JVM from exiting.

2021-03-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17528: Fix Version/s: 3.3.1 > Not closing an SFTP File System instance prevents JVM from exiting

[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results

2021-03-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=563306&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-563306 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on pull request #2597: HADOOP-17191. ABFS: Run the tests with various combinations of configurations and publish a consolidated results

2021-03-09 Thread GitBox
steveloughran commented on pull request #2597: URL: https://github.com/apache/hadoop/pull/2597#issuecomment-794374377 I'm not worried about shellcheck if its some version mismatch there. Is the change actually enabling a check which is disabled on the later branch? if so: interesting

[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results

2021-03-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=563305&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-563305 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub B

[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results

2021-03-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=563304&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-563304 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #2597: HADOOP-17191. ABFS: Run the tests with various combinations of configurations and publish a consolidated results

2021-03-09 Thread GitBox
hadoop-yetus removed a comment on pull request #2597: URL: https://github.com/apache/hadoop/pull/2597#issuecomment-757514175 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #2597: HADOOP-17191. ABFS: Run the tests with various combinations of configurations and publish a consolidated results

2021-03-09 Thread GitBox
hadoop-yetus removed a comment on pull request #2597: URL: https://github.com/apache/hadoop/pull/2597#issuecomment-755239267 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[jira] [Work logged] (HADOOP-17464) Create hadoop-compression module

2021-03-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17464?focusedWorklogId=563303&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-563303 ] ASF GitHub Bot logged work on HADOOP-17464: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on pull request #2611: HADOOP-17464. Create hadoop-compression module

2021-03-09 Thread GitBox
steveloughran commented on pull request #2611: URL: https://github.com/apache/hadoop/pull/2611#issuecomment-794368943 OK, so where are we with this patch? * new module is there * hdfs is happy * compression tests are moved over * all codecs other than snappy are still in hado

[jira] [Assigned] (HADOOP-17472) Upgrade hadoop default version in hadoop.tools.dynamometer.TestDynamometerInfra

2021-03-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-17472: --- Assignee: L. C. Hsieh > Upgrade hadoop default version in > hadoop.tools.dynamome

[jira] [Work logged] (HADOOP-17472) Upgrade hadoop default version in hadoop.tools.dynamometer.TestDynamometerInfra

2021-03-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17472?focusedWorklogId=563299&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-563299 ] ASF GitHub Bot logged work on HADOOP-17472: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on pull request #2622: HADOOP-17472. Upgrade hadoop default version in hadoop.tools.dynamometer.TestDynamometerInfra

2021-03-09 Thread GitBox
steveloughran commented on pull request #2622: URL: https://github.com/apache/hadoop/pull/2622#issuecomment-794360543 With #2537 in, what is the status of this? This is an automated message from the Apache Git Service. To res

[jira] [Work logged] (HADOOP-17464) Create hadoop-compression module

2021-03-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17464?focusedWorklogId=563298&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-563298 ] ASF GitHub Bot logged work on HADOOP-17464: --- Author: ASF GitHub B

[jira] [Work logged] (HADOOP-17464) Create hadoop-compression module

2021-03-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17464?focusedWorklogId=563296&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-563296 ] ASF GitHub Bot logged work on HADOOP-17464: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #2611: HADOOP-17464. Create hadoop-compression module

2021-03-09 Thread GitBox
hadoop-yetus removed a comment on pull request #2611: URL: https://github.com/apache/hadoop/pull/2611#issuecomment-760625516 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:|

[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #2611: HADOOP-17464. Create hadoop-compression module

2021-03-09 Thread GitBox
hadoop-yetus removed a comment on pull request #2611: URL: https://github.com/apache/hadoop/pull/2611#issuecomment-759193916 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[jira] [Work logged] (HADOOP-17402) Add GCS FS impl reference to core-default.xml

2021-03-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17402?focusedWorklogId=563292&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-563292 ] ASF GitHub Bot logged work on HADOOP-17402: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on a change in pull request #2638: HADOOP-17402. Add GCS config to the core-site

2021-03-09 Thread GitBox
steveloughran commented on a change in pull request #2638: URL: https://github.com/apache/hadoop/pull/2638#discussion_r590663774 ## File path: hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestCommonConfigurationFields.java ## @@ -142,6 +141,7 @@ pub

[jira] [Work logged] (HADOOP-17531) DistCp: Reduce memory usage on copying huge directories

2021-03-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17531?focusedWorklogId=563285&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-563285 ] ASF GitHub Bot logged work on HADOOP-17531: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on a change in pull request #2732: HADOOP-17531. DistCp: Reduce memory usage on copying huge directories.

2021-03-09 Thread GitBox
steveloughran commented on a change in pull request #2732: URL: https://github.com/apache/hadoop/pull/2732#discussion_r590624549 ## File path: hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/SimpleCopyListing.java ## @@ -730,4 +660,155 @@ private void writeToF

[jira] [Updated] (HADOOP-16721) Improve S3A rename resilience

2021-03-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-16721: Hadoop Flags: Incompatible change Release Note: The S3A connector's rename() operation

[GitHub] [hadoop] tomscut opened a new pull request #2754: HDFS-15884. RBF: Remove unused method getCreateLocation in RouterRpcS…

2021-03-09 Thread GitBox
tomscut opened a new pull request #2754: URL: https://github.com/apache/hadoop/pull/2754 JIRA: [HDFS-15884](https://issues.apache.org/jira/browse/HDFS-15884) Remove unused method org.apache.hadoop.hdfs.server.federation.router.RouterRpcServer#getCreateLocation.

[jira] [Work logged] (HADOOP-16721) Improve S3A rename resilience

2021-03-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16721?focusedWorklogId=563113&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-563113 ] ASF GitHub Bot logged work on HADOOP-16721: --- Author: ASF GitHub B

[jira] [Work logged] (HADOOP-16721) Improve S3A rename resilience

2021-03-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16721?focusedWorklogId=563114&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-563114 ] ASF GitHub Bot logged work on HADOOP-16721: --- Author: ASF GitHub B

[GitHub] [hadoop] tomscut commented on pull request #2748: HDFS-15879. Exclude slow nodes when choose targets for blocks

2021-03-09 Thread GitBox
tomscut commented on pull request #2748: URL: https://github.com/apache/hadoop/pull/2748#issuecomment-794007692 Hi @dineshchitlangia @tasanuma @ayushtkn @qizhu-lucas , could you please help review the code? Thank you. This i

[GitHub] [hadoop] steveloughran commented on a change in pull request #2742: HADOOP-16721. Improve S3A rename resilience

2021-03-09 Thread GitBox
steveloughran commented on a change in pull request #2742: URL: https://github.com/apache/hadoop/pull/2742#discussion_r590443239 ## File path: hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/impl/ITestlRenameDeleteRace.java ## @@ -0,0 +1,200 @@ +/* + * Licensed

[GitHub] [hadoop] steveloughran commented on a change in pull request #2742: HADOOP-16721. Improve S3A rename resilience

2021-03-09 Thread GitBox
steveloughran commented on a change in pull request #2742: URL: https://github.com/apache/hadoop/pull/2742#discussion_r590442351 ## File path: hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/troubleshooting_s3a.md ## @@ -1126,6 +1126,26 @@ We also recommend using ap

[jira] [Work logged] (HADOOP-16721) Improve S3A rename resilience

2021-03-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16721?focusedWorklogId=563112&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-563112 ] ASF GitHub Bot logged work on HADOOP-16721: --- Author: ASF GitHub B

[GitHub] [hadoop] iwasakims commented on pull request #2742: HADOOP-16721. Improve S3A rename resilience

2021-03-09 Thread GitBox
iwasakims commented on pull request #2742: URL: https://github.com/apache/hadoop/pull/2742#issuecomment-794005013 > 3. don't downgrade FileNotFoundException to 'false' if source doesn't exist > 4. raise FileAlreadyExistsException if dest path exists or parent is a file. Should thi

[jira] [Work logged] (HADOOP-16721) Improve S3A rename resilience

2021-03-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16721?focusedWorklogId=563110&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-563110 ] ASF GitHub Bot logged work on HADOOP-16721: --- Author: ASF GitHub B

[GitHub] [hadoop] iwasakims commented on pull request #2742: HADOOP-16721. Improve S3A rename resilience

2021-03-09 Thread GitBox
iwasakims commented on pull request #2742: URL: https://github.com/apache/hadoop/pull/2742#issuecomment-794003742 I got error on ITestS3AContractDistCp but this seems not to be related. I can not reproduce the error by running the ITestS3AContractDistCp alone by `-Dit.test=ITestS3AContract

[jira] [Work logged] (HADOOP-16721) Improve S3A rename resilience

2021-03-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16721?focusedWorklogId=563107&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-563107 ] ASF GitHub Bot logged work on HADOOP-16721: --- Author: ASF GitHub B

[GitHub] [hadoop] iwasakims commented on pull request #2742: HADOOP-16721. Improve S3A rename resilience

2021-03-09 Thread GitBox
iwasakims commented on pull request #2742: URL: https://github.com/apache/hadoop/pull/2742#issuecomment-793993687 LGTM overall. Tests against Tokyo region worked. I just left comments for nits. ``` $ mvn verify -Dtest=x -Dit.test=ITestS3AContractRename,ITestlRenameDeleteRace -Ds3

[GitHub] [hadoop] iwasakims commented on a change in pull request #2742: HADOOP-16721. Improve S3A rename resilience

2021-03-09 Thread GitBox
iwasakims commented on a change in pull request #2742: URL: https://github.com/apache/hadoop/pull/2742#discussion_r590433369 ## File path: hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/impl/ITestlRenameDeleteRace.java ## @@ -0,0 +1,200 @@ +/* + * Licensed to t

[jira] [Work logged] (HADOOP-16721) Improve S3A rename resilience

2021-03-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16721?focusedWorklogId=563106&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-563106 ] ASF GitHub Bot logged work on HADOOP-16721: --- Author: ASF GitHub B

[jira] [Work logged] (HADOOP-16721) Improve S3A rename resilience

2021-03-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16721?focusedWorklogId=562986&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-562986 ] ASF GitHub Bot logged work on HADOOP-16721: --- Author: ASF GitHub B

[GitHub] [hadoop] steveloughran commented on a change in pull request #2742: HADOOP-16721. Improve S3A rename resilience

2021-03-09 Thread GitBox
steveloughran commented on a change in pull request #2742: URL: https://github.com/apache/hadoop/pull/2742#discussion_r590194822 ## File path: hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/impl/ITestlRenameDeleteRace.java ## @@ -0,0 +1,200 @@ +/* + * Licensed