Steve Loughran created HADOOP-15460:
---------------------------------------
Summary: S3A FS to add "s3a:no-existence-checks" to the builder
file creation option set
Key: HADOOP-15460
URL: https://issues.apache.org/jira/browse/HADOOP-15460
Project: Hadoop Common
Issue Type: Sub-task
Components: fs/s3
Affects Versions: 3.1.0
Reporter: Steve Loughran
As promised to [~StephanEwen]: add and s3a-specific option to the builder-API
to create files for all existence checks to be skipped.
This
# eliminates a few hundred milliseconds
# avoids any caching of negative HEAD/GET responses in the S3 load balancers.
Callers will be expected to know what what they are doing.
FWIW, we are doing some PUT calls in the committer which bypass this stuff, for
the same reason. If you've just created a directory, you know there's nothing
underneath, so no need to check.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
