[
https://issues.apache.org/jira/browse/HADOOP-15894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16670413#comment-16670413
]
Steve Loughran commented on HADOOP-15894:
-----------------------------------------
to get a 404 just after a PUT means that the negative 404 response on the HEAD
call made before creation is still in the AWS load-balancer cache, which is a
fairly small window, and the longer the gap between open() and the
getFileChecksum() call, the less likely this is to happen.
What could be done here?
check s3guard state.
* File deleted -> FNFE
* file exists, remember this
* issue HEAD request
* if HEAD -> 404, conclude brief inconsistency, log and retry both the s3guard
check and the HEAD. (policy: #of attempts?)
> getFileChecksum() needs to adopt S3Guard
> ----------------------------------------
>
> Key: HADOOP-15894
> URL: https://issues.apache.org/jira/browse/HADOOP-15894
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.2.0
> Reporter: Steve Loughran
> Priority: Minor
>
> Encountered a 404 failure in
> {{ITestS3AMiscOperations.testNonEmptyFileChecksumsUnencrypted}}; newly
> created file wasn't seen. Even with S3guard enabled, that method isn't doing
> anything to query the store for it existing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
