[
https://issues.apache.org/jira/browse/HADOOP-18878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18067021#comment-18067021
]
ASF GitHub Bot commented on HADOOP-18878:
-----------------------------------------
steveloughran commented on PR #8353:
URL: https://github.com/apache/hadoop/pull/8353#issuecomment-4097368524
which s3 store did you test against?
> remove/deprecate fs.s3a.multipart.purge
> ---------------------------------------
>
> Key: HADOOP-18878
> URL: https://issues.apache.org/jira/browse/HADOOP-18878
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.3.6
> Reporter: Steve Loughran
> Assignee: Aaron Fabbri
> Priority: Major
> Labels: pull-request-available
>
> the fs.s3a.multipart.purge option has been in for a long time, to help avoid
> running up costs from incomplete uploads, especially during testing.
> but
> * adds overhead on startup (list + delete)
> * dangerous if one client has a very short lifespan; all active s3a committer
> jobs will be broken
> * obsoleted by s3 lifecycle rules
> * and "hadoop s3guard uploads" cli.
> proposed: deprecate for the next release, delete after.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
