[
https://issues.apache.org/jira/browse/HADOOP-14971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16222254#comment-16222254
]
ASF GitHub Bot commented on HADOOP-14971:
-----------------------------------------
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/hadoop/pull/282#discussion_r147394409
--- Diff:
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java
---
@@ -1130,17 +1359,23 @@ private void blockRootDelete(String key) throws
InvalidRequestException {
* Perform a bulk object delete operation.
* Increments the {@code OBJECT_DELETE_REQUESTS} and write
* operation statistics.
+ * Retry policy: retry untranslated; delete considered idempotent.
* @param deleteRequest keys to delete on the s3-backend
* @throws MultiObjectDeleteException one or more of the keys could not
* be deleted.
* @throws AmazonClientException amazon-layer failure.
*/
+ @Retries.RetryRaw
private void deleteObjects(DeleteObjectsRequest deleteRequest)
- throws MultiObjectDeleteException, AmazonClientException {
+ throws MultiObjectDeleteException, AmazonClientException,
IOException {
incrementWriteOperations();
- incrementStatistic(OBJECT_DELETE_REQUESTS, 1);
try {
- s3.deleteObjects(deleteRequest);
+ invoker.retryUntranslated("delete",
--- End diff --
Fundamentally, the core contract of a filesystem directory delete: it
changes from the state "is" to the state "isnt", in an O(1) atomic op, is not
satisified, we just pretend it does..
There's already ambiguity in our pretence. If an object disappears during a
delete, that is not detected. If a new object is added if it is in a large dir
and a later list() call finds it, it is deleted. Otherwise, it remains. Which
means that we don't even detect updates to a path, not unless after a delete()
you do another list to see if there has been a change (or do some leasing in
dynamo, but let's not go there).
AFAIK, delete() succeeds if any child is taken away; the outcome on an add
is undefined. rename is a bit more ambiguous w.r.t either.
> Merge S3A committers into trunk
> -------------------------------
>
> Key: HADOOP-14971
> URL: https://issues.apache.org/jira/browse/HADOOP-14971
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.0.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
>
> Merge the HADOOP-13786 committer into trunk. This branch is being set up as a
> github PR for review there & to keep it out the mailboxes of the watchers on
> the main JIRA
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
