[
https://issues.apache.org/jira/browse/HADOOP-15183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16332584#comment-16332584
]
Steve Loughran commented on HADOOP-15183:
-----------------------------------------
Attached: debug level log of operations leading to failure. Search for
"Renaming readonly files" to get the directory rename sequence which is
failing. One of the files being renamed "readonlyChild" was unsuccessfully
renamed earlier and then destDir/readonlyChild deleted
> S3Guard store becomes inconsistent after partial failure of rename
> ------------------------------------------------------------------
>
> Key: HADOOP-15183
> URL: https://issues.apache.org/jira/browse/HADOOP-15183
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.0.0
> Reporter: Steve Loughran
> Priority: Major
> Attachments: org.apache.hadoop.fs.s3a.auth.ITestAssumeRole-output.txt
>
>
> If an S3A rename() operation fails partway through, such as when the user
> doesn't have permissions to delete the source files after copying to the
> destination, then the s3guard view of the world ends up inconsistent. In
> particular the sequence
> (assuming src/file* is a list of files file1...file10 and read only to
> caller)
>
> # create file dest/file1
> # delete file dest/file1
> # rename src/file* dest/
> You will not see file1 in the listing, because it will have a tombstone
> marker and the update at the end of the rename() didn't take place: the old
> data is still there.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
