[jira] [Work started] (HADOOP-16753) Refactor HAAdmin

2020-01-06 Thread Xieming Li (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HADOOP-16753 started by Xieming Li. --- > Refactor HAAdmin > > > Key: HADOOP-16753 >

[jira] [Updated] (HADOOP-16753) Refactor HAAdmin

2020-01-06 Thread Xieming Li (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xieming Li updated HADOOP-16753: Status: Open (was: Patch Available) > Refactor HAAdmin > > > Ke

[jira] [Commented] (HADOOP-16756) distcp -update to S3A always overwrites due to block size mismatch

2020-01-06 Thread Daisuke Kobayashi (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17008634#comment-17008634 ] Daisuke Kobayashi commented on HADOOP-16756: Thanks guys for looking at thi

[GitHub] [hadoop] bilaharith commented on a change in pull request #1790: changes for combined append+flush calls for blockblob(traditionan) an…

2020-01-06 Thread GitBox
bilaharith commented on a change in pull request #1790: changes for combined append+flush calls for blockblob(traditionan) an… URL: https://github.com/apache/hadoop/pull/1790#discussion_r363219914 ## File path: hadoop-tools/hadoop-azure/pom.xml ## @@ -143,6 +143,14 @@

[GitHub] [hadoop] bilaharith commented on a change in pull request #1790: changes for combined append+flush calls for blockblob(traditionan) an…

2020-01-06 Thread GitBox
bilaharith commented on a change in pull request #1790: changes for combined append+flush calls for blockblob(traditionan) an… URL: https://github.com/apache/hadoop/pull/1790#discussion_r363222173 ## File path: hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/s

[GitHub] [hadoop] bilaharith commented on a change in pull request #1790: changes for combined append+flush calls for blockblob(traditionan) an…

2020-01-06 Thread GitBox
bilaharith commented on a change in pull request #1790: changes for combined append+flush calls for blockblob(traditionan) an… URL: https://github.com/apache/hadoop/pull/1790#discussion_r363222173 ## File path: hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/s

[GitHub] [hadoop] bilaharith commented on a change in pull request #1790: changes for combined append+flush calls for blockblob(traditionan) an…

2020-01-06 Thread GitBox
bilaharith commented on a change in pull request #1790: changes for combined append+flush calls for blockblob(traditionan) an… URL: https://github.com/apache/hadoop/pull/1790#discussion_r363223682 ## File path: hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/s

[GitHub] [hadoop] bilaharith commented on a change in pull request #1790: changes for combined append+flush calls for blockblob(traditionan) an…

2020-01-06 Thread GitBox
bilaharith commented on a change in pull request #1790: changes for combined append+flush calls for blockblob(traditionan) an… URL: https://github.com/apache/hadoop/pull/1790#discussion_r363225156 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/A

[GitHub] [hadoop] bilaharith commented on a change in pull request #1790: changes for combined append+flush calls for blockblob(traditionan) an…

2020-01-06 Thread GitBox
bilaharith commented on a change in pull request #1790: changes for combined append+flush calls for blockblob(traditionan) an… URL: https://github.com/apache/hadoop/pull/1790#discussion_r363225156 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/A

[GitHub] [hadoop] taojieterry opened a new pull request #1794: HADOOP-15887: Add an option to avoid writing data locally in Distcp

2020-01-06 Thread GitBox
taojieterry opened a new pull request #1794: HADOOP-15887: Add an option to avoid writing data locally in Distcp URL: https://github.com/apache/hadoop/pull/1794 In https://issues.apache.org/jira/browse/HDFS-3702 we add a flag in DFSClient to avoid replicating to the local datanode. We can

[GitHub] [hadoop] steveloughran commented on a change in pull request #1792: HADOOP-16790: Add Write Convenience Methods

2020-01-06 Thread GitBox
steveloughran commented on a change in pull request #1792: HADOOP-16790: Add Write Convenience Methods URL: https://github.com/apache/hadoop/pull/1792#discussion_r363253470 ## File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java #

[GitHub] [hadoop] steveloughran commented on a change in pull request #1792: HADOOP-16790: Add Write Convenience Methods

2020-01-06 Thread GitBox
steveloughran commented on a change in pull request #1792: HADOOP-16790: Add Write Convenience Methods URL: https://github.com/apache/hadoop/pull/1792#discussion_r363252414 ## File path: hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileUtil.java #

[GitHub] [hadoop] steveloughran commented on a change in pull request #1792: HADOOP-16790: Add Write Convenience Methods

2020-01-06 Thread GitBox
steveloughran commented on a change in pull request #1792: HADOOP-16790: Add Write Convenience Methods URL: https://github.com/apache/hadoop/pull/1792#discussion_r363253006 ## File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java #

[GitHub] [hadoop] steveloughran commented on a change in pull request #1792: HADOOP-16790: Add Write Convenience Methods

2020-01-06 Thread GitBox
steveloughran commented on a change in pull request #1792: HADOOP-16790: Add Write Convenience Methods URL: https://github.com/apache/hadoop/pull/1792#discussion_r363252650 ## File path: hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileUtil.java #

[GitHub] [hadoop] hadoop-yetus removed a comment on issue #1792: HADOOP-16790: Add Write Convenience Methods

2020-01-06 Thread GitBox
hadoop-yetus removed a comment on issue #1792: HADOOP-16790: Add Write Convenience Methods URL: https://github.com/apache/hadoop/pull/1792#issuecomment-570751348 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|

[jira] [Updated] (HADOOP-16621) [pb-upgrade] spark-hive doesn't compile against hadoop trunk because of Token's marshalling

2020-01-06 Thread Vinayakumar B (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HADOOP-16621: --- Target Version/s: 3.3.0 > [pb-upgrade] spark-hive doesn't compile against hadoop trunk beca

[jira] [Updated] (HADOOP-16358) Add an ARM CI for Hadoop

2020-01-06 Thread Vinayakumar B (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HADOOP-16358: --- Issue Type: Task (was: Improvement) > Add an ARM CI for Hadoop >

[jira] [Resolved] (HADOOP-16358) Add an ARM CI for Hadoop

2020-01-06 Thread Vinayakumar B (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B resolved HADOOP-16358. Fix Version/s: 3.3.0 Resolution: Fixed A Jenkins Job has been created to run night

[jira] [Commented] (HADOOP-16791) ABFS: Have all external dependent module execution tracked with DurationInfo

2020-01-06 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17008747#comment-17008747 ] Steve Loughran commented on HADOOP-16791: - Sounds good. We've just added some

[jira] [Assigned] (HADOOP-16358) Add an ARM CI for Hadoop

2020-01-06 Thread Vinayakumar B (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B reassigned HADOOP-16358: -- Assignee: Zhenyu Zheng > Add an ARM CI for Hadoop > > >

[jira] [Commented] (HADOOP-16358) Add an ARM CI for Hadoop

2020-01-06 Thread Vinayakumar B (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17008748#comment-17008748 ] Vinayakumar B commented on HADOOP-16358: Thanks Everyone > Add an ARM CI for H

[jira] [Commented] (HADOOP-15887) Add an option to avoid writing data locally in Distcp

2020-01-06 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-15887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17008813#comment-17008813 ] Hadoop QA commented on HADOOP-15887: | (/) *{color:green}+1 overall{color}* | \\ \\

[GitHub] [hadoop] hadoop-yetus commented on issue #1794: HADOOP-15887: Add an option to avoid writing data locally in Distcp

2020-01-06 Thread GitBox
hadoop-yetus commented on issue #1794: HADOOP-15887: Add an option to avoid writing data locally in Distcp URL: https://github.com/apache/hadoop/pull/1794#issuecomment-571126994 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|---

[jira] [Assigned] (HADOOP-16621) [pb-upgrade] spark-hive doesn't compile against hadoop trunk because of Token's marshalling

2020-01-06 Thread Vinayakumar B (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B reassigned HADOOP-16621: -- Assignee: Vinayakumar B > [pb-upgrade] spark-hive doesn't compile against hadoop tru

[GitHub] [hadoop] bgaborg commented on issue #1774: HADOOP-16772. Extract version numbers to head of pom.xml (addendum)

2020-01-06 Thread GitBox
bgaborg commented on issue #1774: HADOOP-16772. Extract version numbers to head of pom.xml (addendum) URL: https://github.com/apache/hadoop/pull/1774#issuecomment-571132180 +1, compile works for me, change seems ok. This is a

[GitHub] [hadoop] bgaborg merged pull request #1774: HADOOP-16772. Extract version numbers to head of pom.xml (addendum)

2020-01-06 Thread GitBox
bgaborg merged pull request #1774: HADOOP-16772. Extract version numbers to head of pom.xml (addendum) URL: https://github.com/apache/hadoop/pull/1774 This is an automated message from the Apache Git Service. To respond to t

[jira] [Commented] (HADOOP-16772) Extract version numbers to head of pom.xml (addendum)

2020-01-06 Thread Gabor Bota (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17008832#comment-17008832 ] Gabor Bota commented on HADOOP-16772: - +1 on PR #1774 > Extract version numbers to

[jira] [Resolved] (HADOOP-16772) Extract version numbers to head of pom.xml (addendum)

2020-01-06 Thread Gabor Bota (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Bota resolved HADOOP-16772. - Resolution: Fixed > Extract version numbers to head of pom.xml (addendum) > ---

[GitHub] [hadoop] steveloughran commented on issue #1747: HDFS-15042 Add more tests for ByteBufferPositionedReadable.

2020-01-06 Thread GitBox
steveloughran commented on issue #1747: HDFS-15042 Add more tests for ByteBufferPositionedReadable. URL: https://github.com/apache/hadoop/pull/1747#issuecomment-571136496 hdfs failures clearly something else ``` [INFO] [INFO] Results: [INFO] [ERROR] Failures: [ERROR]

[GitHub] [hadoop] hadoop-yetus removed a comment on issue #1707: HADOOP-16697. Tune/audit auth mode

2020-01-06 Thread GitBox
hadoop-yetus removed a comment on issue #1707: HADOOP-16697. Tune/audit auth mode URL: https://github.com/apache/hadoop/pull/1707#issuecomment-570640020 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:--

[GitHub] [hadoop] hadoop-yetus removed a comment on issue #1707: HADOOP-16697. Tune/audit auth mode

2020-01-06 Thread GitBox
hadoop-yetus removed a comment on issue #1707: HADOOP-16697. Tune/audit auth mode URL: https://github.com/apache/hadoop/pull/1707#issuecomment-570636585 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:--

[GitHub] [hadoop] hadoop-yetus removed a comment on issue #1707: HADOOP-16697. Tune/audit auth mode

2020-01-06 Thread GitBox
hadoop-yetus removed a comment on issue #1707: HADOOP-16697. Tune/audit auth mode URL: https://github.com/apache/hadoop/pull/1707#issuecomment-570346561 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:-

[GitHub] [hadoop] steveloughran commented on issue #1707: HADOOP-16697. Tune/audit auth mode

2020-01-06 Thread GitBox
steveloughran commented on issue #1707: HADOOP-16697. Tune/audit auth mode URL: https://github.com/apache/hadoop/pull/1707#issuecomment-571136940 oops, deleted last yetus review. I'll kick off a new one. This is an automated m

[jira] [Updated] (HADOOP-16595) [pb-upgrade] Create hadoop-thirdparty artifact to have shaded protobuf

2020-01-06 Thread Vinayakumar B (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HADOOP-16595: --- Target Version/s: thirdparty-1.0.0 > [pb-upgrade] Create hadoop-thirdparty artifact to have

[jira] [Reopened] (HADOOP-16772) Extract version numbers to head of pom.xml (addendum)

2020-01-06 Thread Tamas Penzes (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tamas Penzes reopened HADOOP-16772: --- > Extract version numbers to head of pom.xml (addendum) > -

[jira] [Commented] (HADOOP-16772) Extract version numbers to head of pom.xml (addendum)

2020-01-06 Thread Tamas Penzes (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17008868#comment-17008868 ] Tamas Penzes commented on HADOOP-16772: --- [~gabor.bota] thanks for committing #177

[GitHub] [hadoop] steveloughran commented on issue #1791: HADOOP-16785: wasb to raise IOE if write() invoked on a closed stream

2020-01-06 Thread GitBox
steveloughran commented on issue #1791: HADOOP-16785: wasb to raise IOE if write() invoked on a closed stream URL: https://github.com/apache/hadoop/pull/1791#issuecomment-571175415 I can't replicate that in the test -but can see how it does surface...it's in the close() of a try-with-r

[jira] [Commented] (HADOOP-16789) In TestZKFailoverController, restore changes from HADOOP-11149 that were dropped by HDFS-6440

2020-01-06 Thread Jim Brennan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17008959#comment-17008959 ] Jim Brennan commented on HADOOP-16789: -- Thanks [~vagarychen]!   > In TestZKFailo

[GitHub] [hadoop] steveloughran commented on issue #1791: HADOOP-16785: wasb to raise IOE if write() invoked on a closed stream

2020-01-06 Thread GitBox
steveloughran commented on issue #1791: HADOOP-16785: wasb to raise IOE if write() invoked on a closed stream URL: https://github.com/apache/hadoop/pull/1791#issuecomment-571193435 tested -azure wales. All good except for HADOOP-16706 and org.apache.hadoop.fs.azurebfs.ITestClientUrlScheme

[jira] [Updated] (HADOOP-16785) Improve wasb and abfs resilience on double close() calls

2020-01-06 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-16785: Summary: Improve wasb and abfs resilience on double close() calls (was: NativeAzureFsOut

[jira] [Updated] (HADOOP-16785) Improve wasb and abfs resilience on double close() calls

2020-01-06 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-16785: Description: # if you call write() after the NativeAzureFsOutputStream is closed it throw

[jira] [Updated] (HADOOP-16785) Improve wasb and abfs resilience on double close() calls

2020-01-06 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-16785: Priority: Major (was: Blocker) > Improve wasb and abfs resilience on double close() call

[jira] [Commented] (HADOOP-16785) Improve wasb and abfs resilience on double close() calls

2020-01-06 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17008964#comment-17008964 ] Steve Loughran commented on HADOOP-16785: - and abfs: {code} java.lang.IllegalAr

[GitHub] [hadoop] hadoop-yetus commented on issue #1635: HADOOP-16596. [pb-upgrade] Use shaded protobuf classes from hadoop-thirdparty dependency

2020-01-06 Thread GitBox
hadoop-yetus commented on issue #1635: HADOOP-16596. [pb-upgrade] Use shaded protobuf classes from hadoop-thirdparty dependency URL: https://github.com/apache/hadoop/pull/1635#issuecomment-571201484 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comm

[jira] [Commented] (HADOOP-16756) distcp -update to S3A always overwrites due to block size mismatch

2020-01-06 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17008978#comment-17008978 ] Steve Loughran commented on HADOOP-16756: - think we should still add some -p fo

[GitHub] [hadoop] hadoop-yetus commented on issue #1791: HADOOP-16785: wasb to raise IOE if write() invoked on a closed stream

2020-01-06 Thread GitBox
hadoop-yetus commented on issue #1791: HADOOP-16785: wasb to raise IOE if write() invoked on a closed stream URL: https://github.com/apache/hadoop/pull/1791#issuecomment-571209228 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|-

[jira] [Commented] (HADOOP-16785) Improve wasb and abfs resilience on double close() calls

2020-01-06 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17008979#comment-17008979 ] Hadoop QA commented on HADOOP-16785: | (/) *{color:green}+1 overall{color}* | \\ \\

[jira] [Commented] (HADOOP-16621) [pb-upgrade] spark-hive doesn't compile against hadoop trunk because of Token's marshalling

2020-01-06 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17008981#comment-17008981 ] Steve Loughran commented on HADOOP-16621: - I'm for #1, maybe with a backport of

[GitHub] [hadoop] hadoop-yetus commented on issue #1791: HADOOP-16785: wasb to raise IOE if write() invoked on a closed stream

2020-01-06 Thread GitBox
hadoop-yetus commented on issue #1791: HADOOP-16785: wasb to raise IOE if write() invoked on a closed stream URL: https://github.com/apache/hadoop/pull/1791#issuecomment-571215129 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|-

[GitHub] [hadoop] steveloughran commented on a change in pull request #1794: HADOOP-15887: Add an option to avoid writing data locally in Distcp

2020-01-06 Thread GitBox
steveloughran commented on a change in pull request #1794: HADOOP-15887: Add an option to avoid writing data locally in Distcp URL: https://github.com/apache/hadoop/pull/1794#discussion_r363376799 ## File path: hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred

[GitHub] [hadoop] steveloughran commented on a change in pull request #1794: HADOOP-15887: Add an option to avoid writing data locally in Distcp

2020-01-06 Thread GitBox
steveloughran commented on a change in pull request #1794: HADOOP-15887: Add an option to avoid writing data locally in Distcp URL: https://github.com/apache/hadoop/pull/1794#discussion_r363378165 ## File path: hadoop-tools/hadoop-distcp/src/site/markdown/DistCp.md.vm ## @

[GitHub] [hadoop] steveloughran commented on a change in pull request #1794: HADOOP-15887: Add an option to avoid writing data locally in Distcp

2020-01-06 Thread GitBox
steveloughran commented on a change in pull request #1794: HADOOP-15887: Add an option to avoid writing data locally in Distcp URL: https://github.com/apache/hadoop/pull/1794#discussion_r363378426 ## File path: hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestIn

[GitHub] [hadoop] steveloughran commented on a change in pull request #1794: HADOOP-15887: Add an option to avoid writing data locally in Distcp

2020-01-06 Thread GitBox
steveloughran commented on a change in pull request #1794: HADOOP-15887: Add an option to avoid writing data locally in Distcp URL: https://github.com/apache/hadoop/pull/1794#discussion_r363379090 ## File path: hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/contra

[GitHub] [hadoop] steveloughran commented on a change in pull request #1794: HADOOP-15887: Add an option to avoid writing data locally in Distcp

2020-01-06 Thread GitBox
steveloughran commented on a change in pull request #1794: HADOOP-15887: Add an option to avoid writing data locally in Distcp URL: https://github.com/apache/hadoop/pull/1794#discussion_r363378663 ## File path: hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOp

[GitHub] [hadoop] steveloughran commented on a change in pull request #1794: HADOOP-15887: Add an option to avoid writing data locally in Distcp

2020-01-06 Thread GitBox
steveloughran commented on a change in pull request #1794: HADOOP-15887: Add an option to avoid writing data locally in Distcp URL: https://github.com/apache/hadoop/pull/1794#discussion_r363378804 ## File path: hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOp

[jira] [Commented] (HADOOP-16785) Improve wasb and abfs resilience on double close() calls

2020-01-06 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17008990#comment-17008990 ] Hadoop QA commented on HADOOP-16785: | (/) *{color:green}+1 overall{color}* | \\ \\

[GitHub] [hadoop] hadoop-yetus removed a comment on issue #1620: HADOOP-16642. ITestDynamoDBMetadataStoreScale fails when throttled.

2020-01-06 Thread GitBox
hadoop-yetus removed a comment on issue #1620: HADOOP-16642. ITestDynamoDBMetadataStoreScale fails when throttled. URL: https://github.com/apache/hadoop/pull/1620#issuecomment-539632520 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |:-

[GitHub] [hadoop] hadoop-yetus removed a comment on issue #1620: HADOOP-16642. ITestDynamoDBMetadataStoreScale fails when throttled.

2020-01-06 Thread GitBox
hadoop-yetus removed a comment on issue #1620: HADOOP-16642. ITestDynamoDBMetadataStoreScale fails when throttled. URL: https://github.com/apache/hadoop/pull/1620#issuecomment-546320706 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |:-

[GitHub] [hadoop] steveloughran commented on issue #1620: HADOOP-16642. ITestDynamoDBMetadataStoreScale fails when throttled.

2020-01-06 Thread GitBox
steveloughran commented on issue #1620: HADOOP-16642. ITestDynamoDBMetadataStoreScale fails when throttled. URL: https://github.com/apache/hadoop/pull/1620#issuecomment-571219014 rebase and retesting. @bgaborg -can you take a quick look at this? ---

[GitHub] [hadoop] steveloughran commented on a change in pull request #1404: HDFS-13660 Copy file till the source file length during distcp

2020-01-06 Thread GitBox
steveloughran commented on a change in pull request #1404: HDFS-13660 Copy file till the source file length during distcp URL: https://github.com/apache/hadoop/pull/1404#discussion_r363389171 ## File path: hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/Test

[GitHub] [hadoop] steveloughran commented on a change in pull request #1404: HDFS-13660 Copy file till the source file length during distcp

2020-01-06 Thread GitBox
steveloughran commented on a change in pull request #1404: HDFS-13660 Copy file till the source file length during distcp URL: https://github.com/apache/hadoop/pull/1404#discussion_r363389669 ## File path: hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/Test

[GitHub] [hadoop] steveloughran commented on a change in pull request #1404: HDFS-13660 Copy file till the source file length during distcp

2020-01-06 Thread GitBox
steveloughran commented on a change in pull request #1404: HDFS-13660 Copy file till the source file length during distcp URL: https://github.com/apache/hadoop/pull/1404#discussion_r363390092 ## File path: hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/Test

[GitHub] [hadoop] steveloughran commented on a change in pull request #1404: HDFS-13660 Copy file till the source file length during distcp

2020-01-06 Thread GitBox
steveloughran commented on a change in pull request #1404: HDFS-13660 Copy file till the source file length during distcp URL: https://github.com/apache/hadoop/pull/1404#discussion_r363390247 ## File path: hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestDi

[GitHub] [hadoop] steveloughran commented on a change in pull request #1404: HDFS-13660 Copy file till the source file length during distcp

2020-01-06 Thread GitBox
steveloughran commented on a change in pull request #1404: HDFS-13660 Copy file till the source file length during distcp URL: https://github.com/apache/hadoop/pull/1404#discussion_r363390848 ## File path: hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestDi

[GitHub] [hadoop] steveloughran commented on a change in pull request #1404: HDFS-13660 Copy file till the source file length during distcp

2020-01-06 Thread GitBox
steveloughran commented on a change in pull request #1404: HDFS-13660 Copy file till the source file length during distcp URL: https://github.com/apache/hadoop/pull/1404#discussion_r363390684 ## File path: hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestDi

[GitHub] [hadoop] steveloughran commented on issue #1404: HDFS-13660 Copy file till the source file length during distcp

2020-01-06 Thread GitBox
steveloughran commented on issue #1404: HDFS-13660 Copy file till the source file length during distcp URL: https://github.com/apache/hadoop/pull/1404#issuecomment-571225773 sorry, my mistake. Thought this was in and was doing soem more reviews. Ignore that. I'm just trying to run through

[GitHub] [hadoop] ThomasMarquardt commented on a change in pull request #1791: HADOOP-16785: wasb to raise IOE if write() invoked on a closed stream

2020-01-06 Thread GitBox
ThomasMarquardt commented on a change in pull request #1791: HADOOP-16785: wasb to raise IOE if write() invoked on a closed stream URL: https://github.com/apache/hadoop/pull/1791#discussion_r363403741 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/

[GitHub] [hadoop] ThomasMarquardt commented on a change in pull request #1791: HADOOP-16785: wasb to raise IOE if write() invoked on a closed stream

2020-01-06 Thread GitBox
ThomasMarquardt commented on a change in pull request #1791: HADOOP-16785: wasb to raise IOE if write() invoked on a closed stream URL: https://github.com/apache/hadoop/pull/1791#discussion_r363405980 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/

[GitHub] [hadoop] hadoop-yetus commented on issue #1791: HADOOP-16785: wasb to raise IOE if write() invoked on a closed stream

2020-01-06 Thread GitBox
hadoop-yetus commented on issue #1791: HADOOP-16785: wasb to raise IOE if write() invoked on a closed stream URL: https://github.com/apache/hadoop/pull/1791#issuecomment-571242097 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--

[jira] [Commented] (HADOOP-16785) Improve wasb and abfs resilience on double close() calls

2020-01-06 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17009061#comment-17009061 ] Hadoop QA commented on HADOOP-16785: | (x) *{color:red}-1 overall{color}* | \\ \\ |

[GitHub] [hadoop] hadoop-yetus commented on issue #1707: HADOOP-16697. Tune/audit auth mode

2020-01-06 Thread GitBox
hadoop-yetus commented on issue #1707: HADOOP-16697. Tune/audit auth mode URL: https://github.com/apache/hadoop/pull/1707#issuecomment-571248639 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| |

[GitHub] [hadoop] hadoop-yetus commented on issue #1620: HADOOP-16642. ITestDynamoDBMetadataStoreScale fails when throttled.

2020-01-06 Thread GitBox
hadoop-yetus commented on issue #1620: HADOOP-16642. ITestDynamoDBMetadataStoreScale fails when throttled. URL: https://github.com/apache/hadoop/pull/1620#issuecomment-571253386 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|---

[GitHub] [hadoop] ThomasMarquardt commented on a change in pull request #1791: HADOOP-16785: wasb to raise IOE if write() invoked on a closed stream

2020-01-06 Thread GitBox
ThomasMarquardt commented on a change in pull request #1791: HADOOP-16785: wasb to raise IOE if write() invoked on a closed stream URL: https://github.com/apache/hadoop/pull/1791#discussion_r363422859 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azureb

[GitHub] [hadoop] ThomasMarquardt commented on a change in pull request #1791: HADOOP-16785: wasb to raise IOE if write() invoked on a closed stream

2020-01-06 Thread GitBox
ThomasMarquardt commented on a change in pull request #1791: HADOOP-16785: wasb to raise IOE if write() invoked on a closed stream URL: https://github.com/apache/hadoop/pull/1791#discussion_r363431500 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/

[GitHub] [hadoop] ThomasMarquardt commented on a change in pull request #1791: HADOOP-16785: wasb to raise IOE if write() invoked on a closed stream

2020-01-06 Thread GitBox
ThomasMarquardt commented on a change in pull request #1791: HADOOP-16785: wasb to raise IOE if write() invoked on a closed stream URL: https://github.com/apache/hadoop/pull/1791#discussion_r363432586 ## File path: hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azureb

[GitHub] [hadoop] hadoop-yetus commented on issue #1635: HADOOP-16596. [pb-upgrade] Use shaded protobuf classes from hadoop-thirdparty dependency

2020-01-06 Thread GitBox
hadoop-yetus commented on issue #1635: HADOOP-16596. [pb-upgrade] Use shaded protobuf classes from hadoop-thirdparty dependency URL: https://github.com/apache/hadoop/pull/1635#issuecomment-571265883 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comm

[GitHub] [hadoop] ThomasMarquardt commented on a change in pull request #1791: HADOOP-16785: wasb to raise IOE if write() invoked on a closed stream

2020-01-06 Thread GitBox
ThomasMarquardt commented on a change in pull request #1791: HADOOP-16785: wasb to raise IOE if write() invoked on a closed stream URL: https://github.com/apache/hadoop/pull/1791#discussion_r363435811 ## File path: hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azureb

[GitHub] [hadoop] steveloughran commented on issue #1702: HDFS-14788 Use dynamic regex filter to ignore copy of source files in…

2020-01-06 Thread GitBox
steveloughran commented on issue #1702: HDFS-14788 Use dynamic regex filter to ignore copy of source files in… URL: https://github.com/apache/hadoop/pull/1702#issuecomment-571270584 +1 committed to trunk after a full distcp retest. Thanks! ---

[GitHub] [hadoop] steveloughran closed pull request #1702: HDFS-14788 Use dynamic regex filter to ignore copy of source files in…

2020-01-06 Thread GitBox
steveloughran closed pull request #1702: HDFS-14788 Use dynamic regex filter to ignore copy of source files in… URL: https://github.com/apache/hadoop/pull/1702 This is an automated message from the Apache Git Service. To res

[GitHub] [hadoop] steveloughran commented on issue #1712: HADOOP-16699: Add verbose TRACE logging to ABFS

2020-01-06 Thread GitBox
steveloughran commented on issue #1712: HADOOP-16699: Add verbose TRACE logging to ABFS URL: https://github.com/apache/hadoop/pull/1712#issuecomment-571274373 well, provided you are aware that moving to DurationInfo will essentially require rolling back this logging, I am happy to merge it

[GitHub] [hadoop] steveloughran commented on a change in pull request #1791: HADOOP-16785: wasb to raise IOE if write() invoked on a closed stream

2020-01-06 Thread GitBox
steveloughran commented on a change in pull request #1791: HADOOP-16785: wasb to raise IOE if write() invoked on a closed stream URL: https://github.com/apache/hadoop/pull/1791#discussion_r363444377 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/Na

[GitHub] [hadoop] steveloughran commented on a change in pull request #1791: HADOOP-16785: wasb to raise IOE if write() invoked on a closed stream

2020-01-06 Thread GitBox
steveloughran commented on a change in pull request #1791: HADOOP-16785: wasb to raise IOE if write() invoked on a closed stream URL: https://github.com/apache/hadoop/pull/1791#discussion_r363444853 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs

[GitHub] [hadoop] steveloughran commented on a change in pull request #1791: HADOOP-16785: wasb to raise IOE if write() invoked on a closed stream

2020-01-06 Thread GitBox
steveloughran commented on a change in pull request #1791: HADOOP-16785: wasb to raise IOE if write() invoked on a closed stream URL: https://github.com/apache/hadoop/pull/1791#discussion_r363445286 ## File path: hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs

[GitHub] [hadoop] steveloughran commented on a change in pull request #1791: HADOOP-16785: wasb to raise IOE if write() invoked on a closed stream

2020-01-06 Thread GitBox
steveloughran commented on a change in pull request #1791: HADOOP-16785: wasb to raise IOE if write() invoked on a closed stream URL: https://github.com/apache/hadoop/pull/1791#discussion_r363445962 ## File path: hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs

[GitHub] [hadoop] hadoop-yetus removed a comment on issue #1791: HADOOP-16785: wasb to raise IOE if write() invoked on a closed stream

2020-01-06 Thread GitBox
hadoop-yetus removed a comment on issue #1791: HADOOP-16785: wasb to raise IOE if write() invoked on a closed stream URL: https://github.com/apache/hadoop/pull/1791#issuecomment-571215129 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |

[GitHub] [hadoop] hadoop-yetus removed a comment on issue #1791: HADOOP-16785: wasb to raise IOE if write() invoked on a closed stream

2020-01-06 Thread GitBox
hadoop-yetus removed a comment on issue #1791: HADOOP-16785: wasb to raise IOE if write() invoked on a closed stream URL: https://github.com/apache/hadoop/pull/1791#issuecomment-571209228 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |

[GitHub] [hadoop] steveloughran commented on issue #1791: HADOOP-16785: wasb to raise IOE if write() invoked on a closed stream

2020-01-06 Thread GitBox
steveloughran commented on issue #1791: HADOOP-16785: wasb to raise IOE if write() invoked on a closed stream URL: https://github.com/apache/hadoop/pull/1791#issuecomment-571278717 checkstyle ``` /hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsOutp

[GitHub] [hadoop] steveloughran commented on issue #1668: HADOOP-16645. S3A Delegation Token extension point to use StoreContext.

2020-01-06 Thread GitBox
steveloughran commented on issue #1668: HADOOP-16645. S3A Delegation Token extension point to use StoreContext. URL: https://github.com/apache/hadoop/pull/1668#issuecomment-571285652 thanks! This is an automated message f

[GitHub] [hadoop] ThomasMarquardt commented on a change in pull request #1791: HADOOP-16785: wasb to raise IOE if write() invoked on a closed stream

2020-01-06 Thread GitBox
ThomasMarquardt commented on a change in pull request #1791: HADOOP-16785: wasb to raise IOE if write() invoked on a closed stream URL: https://github.com/apache/hadoop/pull/1791#discussion_r363453384 ## File path: hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azureb

[GitHub] [hadoop] ThomasMarquardt commented on a change in pull request #1791: HADOOP-16785: wasb to raise IOE if write() invoked on a closed stream

2020-01-06 Thread GitBox
ThomasMarquardt commented on a change in pull request #1791: HADOOP-16785: wasb to raise IOE if write() invoked on a closed stream URL: https://github.com/apache/hadoop/pull/1791#discussion_r363454016 ## File path: hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azureb

[jira] [Commented] (HADOOP-16727) KMS Jetty server does not startup if trust store password is null

2020-01-06 Thread Hanisha Koneru (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17009208#comment-17009208 ] Hanisha Koneru commented on HADOOP-16727: - Thank you [~weichiu]. I have retrigg

[jira] [Commented] (HADOOP-16727) KMS Jetty server does not startup if trust store password is null

2020-01-06 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17009261#comment-17009261 ] Hadoop QA commented on HADOOP-16727: | (x) *{color:red}-1 overall{color}* | \\ \\ |

[jira] [Created] (HADOOP-16792) Let s3 clients configure request timeout

2020-01-06 Thread Mustafa Iman (Jira)
Mustafa Iman created HADOOP-16792: - Summary: Let s3 clients configure request timeout Key: HADOOP-16792 URL: https://issues.apache.org/jira/browse/HADOOP-16792 Project: Hadoop Common Issue Ty

[GitHub] [hadoop] belugabehr commented on issue #1792: HADOOP-16790: Add Write Convenience Methods

2020-01-06 Thread GitBox
belugabehr commented on issue #1792: HADOOP-16790: Add Write Convenience Methods URL: https://github.com/apache/hadoop/pull/1792#issuecomment-571387538 @steveloughran Thanks for the review! > can't point to a good alternative place right now. Neither could I. > Passing i

[GitHub] [hadoop] belugabehr commented on a change in pull request #1792: HADOOP-16790: Add Write Convenience Methods

2020-01-06 Thread GitBox
belugabehr commented on a change in pull request #1792: HADOOP-16790: Add Write Convenience Methods URL: https://github.com/apache/hadoop/pull/1792#discussion_r363554355 ## File path: hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileUtil.java

[GitHub] [hadoop] ramesh0201 commented on a change in pull request #1768: HADOOP-16769. LocalDirAllocator to provide diagnostics when file creation fails

2020-01-06 Thread GitBox
ramesh0201 commented on a change in pull request #1768: HADOOP-16769. LocalDirAllocator to provide diagnostics when file creation fails URL: https://github.com/apache/hadoop/pull/1768#discussion_r363555060 ## File path: hadoop-common-project/hadoop-common/src/test/java/org/apache/h

[GitHub] [hadoop] belugabehr commented on a change in pull request #1792: HADOOP-16790: Add Write Convenience Methods

2020-01-06 Thread GitBox
belugabehr commented on a change in pull request #1792: HADOOP-16790: Add Write Convenience Methods URL: https://github.com/apache/hadoop/pull/1792#discussion_r363555279 ## File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java ##

[GitHub] [hadoop] mustafaiman opened a new pull request #1795: HADOOP-16792: Make S3 client request timeout configurable

2020-01-06 Thread GitBox
mustafaiman opened a new pull request #1795: HADOOP-16792: Make S3 client request timeout configurable URL: https://github.com/apache/hadoop/pull/1795 ## NOTICE Please create an issue in ASF JIRA before opening a pull request, and you need to set the title of the pull request whic

[GitHub] [hadoop] ramesh0201 commented on issue #1768: HADOOP-16769. LocalDirAllocator to provide diagnostics when file creation fails

2020-01-06 Thread GitBox
ramesh0201 commented on issue #1768: HADOOP-16769. LocalDirAllocator to provide diagnostics when file creation fails URL: https://github.com/apache/hadoop/pull/1768#issuecomment-571394894 I will address the above change to catch and throw the two errors, so they are nested into one another

[GitHub] [hadoop] ramesh0201 edited a comment on issue #1768: HADOOP-16769. LocalDirAllocator to provide diagnostics when file creation fails

2020-01-06 Thread GitBox
ramesh0201 edited a comment on issue #1768: HADOOP-16769. LocalDirAllocator to provide diagnostics when file creation fails URL: https://github.com/apache/hadoop/pull/1768#issuecomment-571394894 I will address the above change to catch and throw the two errors, so they are nested into one

  1   2   >