[jira] [Created] (HADOOP-18183) s3a audit logs to publish range start/end of GET requests in audit header

2022-03-28 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18183: --- Summary: s3a audit logs to publish range start/end of GET requests in audit header Key: HADOOP-18183 URL: https://issues.apache.org/jira/browse/HADOOP-18183 Pro

[jira] [Created] (HADOOP-18182) S3File to store reference to active S3Object in a field.

2022-03-28 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18182: --- Summary: S3File to store reference to active S3Object in a field. Key: HADOOP-18182 URL: https://issues.apache.org/jira/browse/HADOOP-18182 Project: Hadoop Commo

[jira] [Created] (HADOOP-18181) move org.apache.hadoop.fs.common package into hadoop-common module

2022-03-28 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18181: --- Summary: move org.apache.hadoop.fs.common package into hadoop-common module Key: HADOOP-18181 URL: https://issues.apache.org/jira/browse/HADOOP-18181 Project:

[jira] [Resolved] (HADOOP-18176) s3a prefetching stream to move off twitter FuturePool

2022-03-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18176. - Resolution: Duplicate duplicated by HADOOP-18180. which comes with a PR > s3a prefetch

[jira] [Created] (HADOOP-18179) Boost S3A Stream Read Performance

2022-03-28 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18179: --- Summary: Boost S3A Stream Read Performance Key: HADOOP-18179 URL: https://issues.apache.org/jira/browse/HADOOP-18179 Project: Hadoop Common Issue Type:

[jira] [Created] (HADOOP-18177) document use and design of prefetching s3a input stream

2022-03-25 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18177: --- Summary: document use and design of prefetching s3a input stream Key: HADOOP-18177 URL: https://issues.apache.org/jira/browse/HADOOP-18177 Project: Hadoop Common

[jira] [Created] (HADOOP-18176) s3a prefetching stream to move off twitter FuturePool

2022-03-25 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18176: --- Summary: s3a prefetching stream to move off twitter FuturePool Key: HADOOP-18176 URL: https://issues.apache.org/jira/browse/HADOOP-18176 Project: Hadoop Common

[jira] [Created] (HADOOP-18175) test failures with prefetching s3a input stream

2022-03-25 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18175: --- Summary: test failures with prefetching s3a input stream Key: HADOOP-18175 URL: https://issues.apache.org/jira/browse/HADOOP-18175 Project: Hadoop Common

[jira] [Resolved] (HADOOP-18160) `org.wildfly.openssl` should not be shaded by Hadoop build

2022-03-22 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18160. - Fix Version/s: 3.3.3 Resolution: Fixed > `org.wildfly.openssl` should not be sha

[jira] [Resolved] (HADOOP-13704) S3A getContentSummary() to move to listFiles(recursive) to count children; instrument use

2022-03-22 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-13704. - Fix Version/s: 3.3.3 Resolution: Fixed fixed in 3.3.3. thanks! > S3A getContent

[jira] [Created] (HADOOP-18168) ITestMarkerTool.testRunLimitedLandsatAudit failing -bucket purged?

2022-03-22 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18168: --- Summary: ITestMarkerTool.testRunLimitedLandsatAudit failing -bucket purged? Key: HADOOP-18168 URL: https://issues.apache.org/jira/browse/HADOOP-18168 Project: H

[jira] [Resolved] (HADOOP-17976) abfs etag extraction inconsistent between LIST and HEAD calls

2022-03-17 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17976. - Fix Version/s: 3.3.3 Resolution: Fixed > abfs etag extraction inconsistent betwe

[jira] [Resolved] (HADOOP-16204) ABFS tests to include terasort

2022-03-17 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-16204. - Fix Version/s: 3.3.3 Resolution: Fixed > ABFS tests to include terasort > --

[jira] [Resolved] (HADOOP-18163) hadoop-azure support for the Manifest Committer of MAPREDUCE-7341

2022-03-17 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18163. - Fix Version/s: 3.3.3 Resolution: Fixed > hadoop-azure support for the Manifest C

[jira] [Resolved] (HADOOP-18162) hadoop-common enhancements for the Manifest Committer of MAPREDUCE-7341

2022-03-17 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18162. - Fix Version/s: 3.3.3 Resolution: Fixed > hadoop-common enhancements for the Mani

[jira] [Created] (HADOOP-18163) hadoop-azure support for the Manifest Committer of MAPREDUCE-7341

2022-03-16 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18163: --- Summary: hadoop-azure support for the Manifest Committer of MAPREDUCE-7341 Key: HADOOP-18163 URL: https://issues.apache.org/jira/browse/HADOOP-18163 Project: Ha

[jira] [Created] (HADOOP-18162) Hadoop common enhancements for the Manifest Committer ofMAPREDUCE-7341

2022-03-16 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18162: --- Summary: Hadoop common enhancements for the Manifest Committer ofMAPREDUCE-7341 Key: HADOOP-18162 URL: https://issues.apache.org/jira/browse/HADOOP-18162 Projec

[jira] [Resolved] (HADOOP-18157) S3AFileSystem cannot find a fake dir any more

2022-03-10 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18157. - Resolution: Duplicate > S3AFileSystem cannot find a fake dir any more > ---

[jira] [Resolved] (HADOOP-18150) Fix ITestAuditManagerDisabled after S3A audit logging was enabled in HADOOP-18091

2022-03-03 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18150. - Fix Version/s: 3.3.3 Resolution: Fixed > Fix ITestAuditManagerDisabled after S3A

[jira] [Resolved] (HADOOP-18075) ABFS: Fix failure caused by listFiles() in ITestAbfsRestOperationException

2022-03-01 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18075. - Fix Version/s: 3.3.3 Resolution: Fixed > ABFS: Fix failure caused by listFiles()

[jira] [Resolved] (HADOOP-17862) ABFS: Fix unchecked cast compiler warning for AbfsListStatusRemoteIterator

2022-03-01 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17862. - Fix Version/s: 3.3.3 Resolution: Fixed > ABFS: Fix unchecked cast compiler warni

[jira] [Resolved] (HADOOP-17765) ABFS: Use Unique File Paths in Tests

2022-03-01 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17765. - Fix Version/s: 3.3.3 Resolution: Fixed > ABFS: Use Unique File Paths in Tests >

[jira] [Resolved] (HADOOP-18147) ITestAbfsRestOperationException failing

2022-02-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18147. - Resolution: Duplicate > ITestAbfsRestOperationException failing > -

[jira] [Created] (HADOOP-18147) ITestAbfsRestOperationException failing

2022-02-28 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18147: --- Summary: ITestAbfsRestOperationException failing Key: HADOOP-18147 URL: https://issues.apache.org/jira/browse/HADOOP-18147 Project: Hadoop Common Issue

[jira] [Resolved] (HADOOP-18071) ABFS: Set driver global timeout for ITestAzureBlobFileSystemBasics

2022-02-23 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18071. - Fix Version/s: 3.3.3 Resolution: Fixed > ABFS: Set driver global timeout for ITe

[jira] [Created] (HADOOP-18137) Verify FileUtils.unTar() handling of missing .tgz files

2022-02-21 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18137: --- Summary: Verify FileUtils.unTar() handling of missing .tgz files Key: HADOOP-18137 URL: https://issues.apache.org/jira/browse/HADOOP-18137 Project: Hadoop Common

[jira] [Created] (HADOOP-18136) Verify FileUtils.unTar() handling of missing .tar files

2022-02-21 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18136: --- Summary: Verify FileUtils.unTar() handling of missing .tar files Key: HADOOP-18136 URL: https://issues.apache.org/jira/browse/HADOOP-18136 Project: Hadoop Common

[jira] [Resolved] (HADOOP-18091) S3A auditing leaks memory through ThreadLocal references

2022-02-10 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18091. - Fix Version/s: 3.3.3 Resolution: Fixed > S3A auditing leaks memory through Threa

[jira] [Resolved] (HADOOP-18114) Documentation Syntax Error Fix > AWS Assumed Roles

2022-02-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18114. - Fix Version/s: 3.3.3 Resolution: Fixed thanks! > Documentation Syntax Error Fix

[jira] [Resolved] (HADOOP-18085) S3 SDK Upgrade causes AccessPoint ARN endpoint mistranslation

2022-02-04 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18085. - Fix Version/s: 3.3.3 Resolution: Fixed this in branch-3 now, so all branches wit

[jira] [Created] (HADOOP-18113) remove filtering of directory markers in s3a RenameOperation

2022-02-03 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18113: --- Summary: remove filtering of directory markers in s3a RenameOperation Key: HADOOP-18113 URL: https://issues.apache.org/jira/browse/HADOOP-18113 Project: Hadoop

[jira] [Created] (HADOOP-18095) s3a connector to fully support AWS partitions,

2022-01-24 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18095: --- Summary: s3a connector to fully support AWS partitions, Key: HADOOP-18095 URL: https://issues.apache.org/jira/browse/HADOOP-18095 Project: Hadoop Common

[jira] [Resolved] (HADOOP-18094) Disable S3A auditing by default.

2022-01-24 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18094. - Fix Version/s: 3.3.2 Resolution: Fixed > Disable S3A auditing by default. >

[jira] [Created] (HADOOP-18094) Disable S3A auditing by default.

2022-01-24 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18094: --- Summary: Disable S3A auditing by default. Key: HADOOP-18094 URL: https://issues.apache.org/jira/browse/HADOOP-18094 Project: Hadoop Common Issue Type:

[jira] [Created] (HADOOP-18091) S3A auditing leaks memory through ThreadLocal references

2022-01-22 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18091: --- Summary: S3A auditing leaks memory through ThreadLocal references Key: HADOOP-18091 URL: https://issues.apache.org/jira/browse/HADOOP-18091 Project: Hadoop Commo

[jira] [Resolved] (HADOOP-18084) ABFS: Add testfilePath while verifying test contents are read correctly

2022-01-19 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18084. - Fix Version/s: 3.3.3 Resolution: Fixed > ABFS: Add testfilePath while verifying

[jira] [Resolved] (HADOOP-17409) Remove S3Guard - no longer needed

2022-01-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17409. - Fix Version/s: 3.3.3 (was: 3.4.0) Resolution: Fixed > Rem

[jira] [Resolved] (HADOOP-17457) intermittent ITestS3AInconsistency.testGetFileStatus failure.

2022-01-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17457. - Fix Version/s: 3.3.3 Resolution: Duplicate > intermittent ITestS3AInconsistency.

[jira] [Resolved] (HADOOP-18068) upgrade AWS SDK to 1.12.132

2022-01-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18068. - Fix Version/s: 3.3.3 Resolution: Fixed TestArn from HADOOP-17198 broke with this

[jira] [Created] (HADOOP-18081) FileNotFoundException in abfs mkdirs() call

2022-01-13 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18081: --- Summary: FileNotFoundException in abfs mkdirs() call Key: HADOOP-18081 URL: https://issues.apache.org/jira/browse/HADOOP-18081 Project: Hadoop Common I

[jira] [Resolved] (HADOOP-17954) org.apache.spark.SparkException: Task failed while writing rows S3

2022-01-10 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17954. - Resolution: Cannot Reproduce > org.apache.spark.SparkException: Task failed while writi

[jira] [Resolved] (HADOOP-14334) S3 SSEC tests to downgrade when running against a mandatory encryption object store

2022-01-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-14334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-14334. - Fix Version/s: 3.3.3 Resolution: Fixed > S3 SSEC tests to downgrade when runnin

[jira] [Resolved] (HADOOP-17736) Über-JIRA: abfs phase III: Hadoop 3.3.2 features & fixes

2022-01-07 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17736. - Resolution: Done > Über-JIRA: abfs phase III: Hadoop 3.3.2 features & fixes > -

[jira] [Resolved] (HADOOP-17492) abfs listLocatedStatus to support incremental/async page fetching

2022-01-07 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17492. - Fix Version/s: 3.3.3 Resolution: Fixed done in HADOOP-17979 so etags are returne

[jira] [Created] (HADOOP-18072) Über-JIRA: abfs phase III: Hadoop 3.3.3 features & fixes

2022-01-07 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18072: --- Summary: Über-JIRA: abfs phase III: Hadoop 3.3.3 features & fixes Key: HADOOP-18072 URL: https://issues.apache.org/jira/browse/HADOOP-18072 Project: Hadoop Commo

[jira] [Created] (HADOOP-18068) upgrade AWS SDK to 1.12.x

2022-01-05 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18068: --- Summary: upgrade AWS SDK to 1.12.x Key: HADOOP-18068 URL: https://issues.apache.org/jira/browse/HADOOP-18068 Project: Hadoop Common Issue Type: Sub-tas

[jira] [Resolved] (HADOOP-17566) Über-jira: S3A Hadoop 3.3.2 features

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17566. - Fix Version/s: 3.3.2 Resolution: Done > Über-jira: S3A Hadoop 3.3.2 features > -

[jira] [Resolved] (HADOOP-15751) AWS Data read stack trace in S3a putObjectDirect

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-15751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-15751. - Resolution: Cannot Reproduce not seen for a long time; closing. re-open if it surfaces

[jira] [Resolved] (HADOOP-15955) declare that fs.s3a.ext. is a prefix for arbitrary extensions

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-15955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-15955. - Fix Version/s: 3.3.2 Resolution: Done > declare that fs.s3a.ext. is a prefix for

[jira] [Resolved] (HADOOP-16370) S3AFileSystem copyFile to propagate etag/version from getObjectMetadata to copy request

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-16370. - Resolution: Duplicate > S3AFileSystem copyFile to propagate etag/version from getObject

[jira] [Resolved] (HADOOP-14070) S3a: Failed to reset the request input stream/make S3A uploadPart() retriable

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-14070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-14070. - Resolution: Cannot Reproduce not seen for a long time; closing. re-open if it surfaces

[jira] [Resolved] (HADOOP-16188) s3a rename failed during copy, "Unable to copy part" + 200 error code

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-16188. - Resolution: Cannot Reproduce > s3a rename failed during copy, "Unable to copy part" + 2

[jira] [Resolved] (HADOOP-17226) Failure of ITestAssumeRole.testRestrictedCommitActions

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17226. - Fix Version/s: 3.3.2 Resolution: Fixed > Failure of ITestAssumeRole.testRestrict

[jira] [Resolved] (HADOOP-17263) transient ITestS3AFileContextStatistics failure -read buffer not filled

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17263. - Resolution: Won't Fix usual transient buffer underrun issue > transient ITestS3AFileCo

[jira] [Resolved] (HADOOP-17403) S3A ITestPartialRenamesDeletes.testRenameDirFailsInDelete failure: missing directory marker

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17403. - Resolution: Won't Fix remove s3guard and this whole feature gets cut, so no test failur

[jira] [Resolved] (HADOOP-17818) ITestS3AContractDistCp failing IllegalStateException: WFOPENSSL0030 Running handshake with buffered unwrapped data

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17818. - Resolution: Cannot Reproduce > ITestS3AContractDistCp failing IllegalStateException: WF

[jira] [Created] (HADOOP-18067) Über-jira: S3A Hadoop 3.3.3 features

2022-01-05 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18067: --- Summary: Über-jira: S3A Hadoop 3.3.3 features Key: HADOOP-18067 URL: https://issues.apache.org/jira/browse/HADOOP-18067 Project: Hadoop Common Issue Ty

[jira] [Resolved] (HADOOP-18064) replace URL.toString() with URL.toURI().toString()

2022-01-04 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18064. - Resolution: Invalid > replace URL.toString() with URL.toURI().toString() >

[jira] [Created] (HADOOP-18064) replace URL.toString() with URL.toURI().toString()

2022-01-04 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18064: --- Summary: replace URL.toString() with URL.toURI().toString() Key: HADOOP-18064 URL: https://issues.apache.org/jira/browse/HADOOP-18064 Project: Hadoop Common

[jira] [Resolved] (HADOOP-17989) ITestAzureBlobFileSystemDelete failing "Operations has null HTTP response"

2021-12-06 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17989. - Fix Version/s: 3.3.3 Resolution: Fixed > ITestAzureBlobFileSystemDelete failing

[jira] [Resolved] (HADOOP-18026) Fix default value of Magic committer

2021-12-03 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18026. - Fix Version/s: 3.3.3 Resolution: Fixed > Fix default value of Magic committer >

[jira] [Resolved] (HADOOP-18021) Provide a public wrapper of Configuration#substituteVars

2021-12-03 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18021. - Resolution: Fixed > Provide a public wrapper of Configuration#substituteVars >

[jira] [Resolved] (HADOOP-17979) Interface EtagSource to allow FileStatus subclasses to provide etags

2021-11-29 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17979. - Fix Version/s: 3.3.2 Resolution: Fixed > Interface EtagSource to allow FileStatu

[jira] [Created] (HADOOP-18013) ABFS: add resilient trash policy

2021-11-18 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18013: --- Summary: ABFS: add resilient trash policy Key: HADOOP-18013 URL: https://issues.apache.org/jira/browse/HADOOP-18013 Project: Hadoop Common Issue Type:

[jira] [Created] (HADOOP-18004) abfs and s3a disk buffer factories to use UUIDs for file prefixes

2021-11-11 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18004: --- Summary: abfs and s3a disk buffer factories to use UUIDs for file prefixes Key: HADOOP-18004 URL: https://issues.apache.org/jira/browse/HADOOP-18004 Project: Ha

[jira] [Resolved] (HADOOP-17981) Support etag-assisted renames in FileOutputCommitter

2021-11-10 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17981. - Resolution: Won't Fix closing as WONTFIX as even though the PR is complete, we don't wa

[jira] [Created] (HADOOP-18002) abfs

2021-11-10 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18002: --- Summary: abfs Key: HADOOP-18002 URL: https://issues.apache.org/jira/browse/HADOOP-18002 Project: Hadoop Common Issue Type: Bug Reporter: St

[jira] [Resolved] (HADOOP-17928) s3a: set fs.s3a.downgrade.syncable.exceptions = true by default

2021-11-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17928. - Fix Version/s: 3.3.2 Resolution: Fixed > s3a: set fs.s3a.downgrade.syncable.exce

[jira] [Reopened] (HADOOP-17873) ABFS: Fix transient failures in ITestAbfsStreamStatistics and ITestAbfsRestOperationException

2021-11-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reopened HADOOP-17873: - had to reopen it as now build rules keep the shaded guava @VislbleForTesting refs out; you

[jira] [Resolved] (HADOOP-17863) ABFS: Fix compiler deprecation warning in TextFileBasedIdentityHandler

2021-11-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17863. - Fix Version/s: 3.3.2 Resolution: Fixed > ABFS: Fix compiler deprecation warning

[jira] [Resolved] (HADOOP-17873) ABFS: Fix transient failures in ITestAbfsStreamStatistics and ITestAbfsRestOperationException

2021-11-04 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17873. - Fix Version/s: 3.4.0 Resolution: Fixed > ABFS: Fix transient failures in ITestAb

[jira] [Resolved] (HADOOP-17966) S3A SSE-KMS inconsistency issue during rename

2021-11-04 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17966. - Resolution: Duplicate > S3A SSE-KMS inconsistency issue during rename > ---

[jira] [Created] (HADOOP-17989) ITestAzureBlobFileSystemDelete failing "Operations has null HTTP response"

2021-11-04 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17989: --- Summary: ITestAzureBlobFileSystemDelete failing "Operations has null HTTP response" Key: HADOOP-17989 URL: https://issues.apache.org/jira/browse/HADOOP-17989 Pr

[jira] [Resolved] (HADOOP-17945) JsonSerialization raises EOFException reading JSON data stored on google GCS

2021-11-02 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17945. - Fix Version/s: (was: 3.4.0) 3.3.2 Resolution: Fixed > Jso

[jira] [Resolved] (HADOOP-17984) Hadoop-aws jar is unable to read file from S3 if used with third party like MINIO

2021-10-29 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17984. - Resolution: Invalid > Hadoop-aws jar is unable to read file from S3 if used with third

[jira] [Created] (HADOOP-17981) Support etag-assisted renames in FileOutputCommitter

2021-10-28 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17981: --- Summary: Support etag-assisted renames in FileOutputCommitter Key: HADOOP-17981 URL: https://issues.apache.org/jira/browse/HADOOP-17981 Project: Hadoop Common

[jira] [Resolved] (HADOOP-17980) Spark application stuck at ACCEPTED state (unset port issue)

2021-10-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17980. - Resolution: Invalid > Spark application stuck at ACCEPTED state (unset port issue) > --

[jira] [Created] (HADOOP-17979) Interface EtagFromFileStatus to allow FileStatus subclasses to provide etags

2021-10-26 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17979: --- Summary: Interface EtagFromFileStatus to allow FileStatus subclasses to provide etags Key: HADOOP-17979 URL: https://issues.apache.org/jira/browse/HADOOP-17979

[jira] [Created] (HADOOP-17976) abfs etag extraction inconsistent between LIST and HEAD calls

2021-10-22 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17976: --- Summary: abfs etag extraction inconsistent between LIST and HEAD calls Key: HADOOP-17976 URL: https://issues.apache.org/jira/browse/HADOOP-17976 Project: Hadoop

[jira] [Resolved] (HADOOP-17974) Fix the import statements in hadoop-aws module

2021-10-21 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17974. - Fix Version/s: 3.3.2 Resolution: Fixed merged trunk & branch-3.3; accidentally l

[jira] [Resolved] (HADOOP-17953) S3A: ITestS3AFileContextStatistics test to lookup global or per-bucket configuration for encryption algorithm

2021-10-19 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17953. - Fix Version/s: 3.4.0 Resolution: Fixed > S3A: ITestS3AFileContextStatistics test

[jira] [Resolved] (HADOOP-16646) Backport S3A enhancements and fixes from trunk to branch-3.2

2021-10-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-16646. - Resolution: Won't Fix > Backport S3A enhancements and fixes from trunk to branch-3.2 >

[jira] [Resolved] (HADOOP-17559) S3Guard import can OOM on large imports

2021-10-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17559. - Resolution: Won't Fix > S3Guard import can OOM on large imports > -

[jira] [Created] (HADOOP-17961) s3 and abfs incremental listing: ask for a smaller first batch

2021-10-11 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17961: --- Summary: s3 and abfs incremental listing: ask for a smaller first batch Key: HADOOP-17961 URL: https://issues.apache.org/jira/browse/HADOOP-17961 Project: Hadoo

[jira] [Resolved] (HADOOP-17951) AccessPoint verifyBucketExistsV2 always returns false

2021-10-04 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17951. - Fix Version/s: 3.4.0 Resolution: Fixed > AccessPoint verifyBucketExistsV2 always

[jira] [Resolved] (HADOOP-17198) Support S3 Access Points

2021-10-04 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17198. - Resolution: Fixed in trunk, will get into 3.3.x once the CSE stuff is in > Support S3

[jira] [Resolved] (HADOOP-17934) NullPointerException when no HTTP response set on AbfsRestOperation

2021-09-30 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17934. - Fix Version/s: 3.3.3 Resolution: Fixed > NullPointerException when no HTTP respo

[jira] [Resolved] (HADOOP-17520) S3A to downgraded to unguarded if the DDB table cannot be found

2021-09-30 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17520. - Resolution: Won't Fix we have the PR but I'm not going to merge it > S3A to downgraded

[jira] [Resolved] (HADOOP-17851) S3A to support user-specified content encoding

2021-09-29 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17851. - Fix Version/s: 3.4.0 Resolution: Fixed > S3A to support user-specified content e

[jira] [Created] (HADOOP-17945) JsonSerialization raises EOFException reading JSON off google GCS

2021-09-29 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17945: --- Summary: JsonSerialization raises EOFException reading JSON off google GCS Key: HADOOP-17945 URL: https://issues.apache.org/jira/browse/HADOOP-17945 Project: Ha

[jira] [Created] (HADOOP-17943) Add s3a tool to convert S3 server logs to avro/csv files

2021-09-28 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17943: --- Summary: Add s3a tool to convert S3 server logs to avro/csv files Key: HADOOP-17943 URL: https://issues.apache.org/jira/browse/HADOOP-17943 Project: Hadoop Commo

[jira] [Created] (HADOOP-17942) abfs & s3a FS instantiate triggers warning about deprecated io.bytes.per.checksum

2021-09-28 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17942: --- Summary: abfs & s3a FS instantiate triggers warning about deprecated io.bytes.per.checksum Key: HADOOP-17942 URL: https://issues.apache.org/jira/browse/HADOOP-17942

[jira] [Resolved] (HADOOP-7199) Crash due to reuse of checksum files

2021-09-27 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-7199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-7199. Resolution: Not A Problem > Crash due to reuse of checksum files > ---

[jira] [Created] (HADOOP-17937) .ITestS3ADeleteFilesOneByOne. testBulkRenameAndDelete OOM: Direct buffer memory

2021-09-24 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17937: --- Summary: .ITestS3ADeleteFilesOneByOne. testBulkRenameAndDelete OOM: Direct buffer memory Key: HADOOP-17937 URL: https://issues.apache.org/jira/browse/HADOOP-17937

[jira] [Resolved] (HADOOP-17195) Intermittent OutOfMemory error while performing hdfs CopyFromLocal to abfs

2021-09-22 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17195. - Fix Version/s: 3.3.3 Resolution: Fixed > Intermittent OutOfMemory error while pe

[jira] [Created] (HADOOP-17928) s3a: set fs.s3a.downgrade.syncable.exceptions = true by default

2021-09-21 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17928: --- Summary: s3a: set fs.s3a.downgrade.syncable.exceptions = true by default Key: HADOOP-17928 URL: https://issues.apache.org/jira/browse/HADOOP-17928 Project: Hado

[jira] [Resolved] (HADOOP-13637) improve setting of max connections in AWS client

2021-09-17 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-13637. - Fix Version/s: 3.3.2 Resolution: Duplicate > improve setting of max connections

[jira] [Created] (HADOOP-17915) ABFS AbfsDelegationTokenManager to generate canonicalServiceName if DT plugin doesn't

2021-09-15 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17915: --- Summary: ABFS AbfsDelegationTokenManager to generate canonicalServiceName if DT plugin doesn't Key: HADOOP-17915 URL: https://issues.apache.org/jira/browse/HADOOP-17915

[jira] [Resolved] (HADOOP-17894) CredentialProviderFactory.getProviders() recursion loading JCEKS file from s3a

2021-09-10 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17894. - Fix Version/s: 3.3.2 Resolution: Fixed > CredentialProviderFactory.getProviders(

[jira] [Created] (HADOOP-17906) AbstractAbfsIntegrationTest.createFilesystemForSASTests() leaks FS instances

2021-09-09 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17906: --- Summary: AbstractAbfsIntegrationTest.createFilesystemForSASTests() leaks FS instances Key: HADOOP-17906 URL: https://issues.apache.org/jira/browse/HADOOP-17906

<    2   3   4   5   6   7   8   9   10   11   >