Steve Loughran created HADOOP-18657:
---------------------------------------
Summary: Tune ABFS create() retry logic
Key: HADOOP-18657
URL: https://issues.apache.org/jira/browse/HADOOP-18657
Project: Hadoop Common
Issue Type: Improvement
Components: fs/azure
Affects Versions: 3.3.5
Reporter: Steve Loughran
Based on experience trying to debug this happening
# add debug statements when create() fails
# generated exception text to reference string shared with tests, path and
error code
# generated exception to include inner exception for full stack trace
Currently the retry logic is
# create(overwrite=false)
# if HTTP_CONFLICT/409 raised; call HEAD
# use etag in create(path, overwrite=true, etag)
# special handling of error HTTP_PRECON_FAILED = 412
There's a race condition here, which is if between 1 and 2 the file which
exists is deleted. The retry should succeed, but currently a 404 from the head
is escalated to a failure
proposed changes
# if HEAD is 404, leave etag == null and continue
# special handling of 412 also to handle 409
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
