Steve Loughran created HADOOP-14645:
---------------------------------------
Summary: add a predicate/option to probe an S3A FS for being
consistent
Key: HADOOP-14645
URL: https://issues.apache.org/jira/browse/HADOOP-14645
Project: Hadoop Common
Issue Type: Sub-task
Components: fs/s3
Affects Versions: HADOOP-13345
Reporter: Steve Loughran
The committer work shows It's useful for s3a-aware code to be able to ask if
the FS is consistent fir listings. AWS S3 is when s3guard is turned on, but
third-party S3 implementations may also be consistent.
The committer should be able to ask an FS if it is consistent, something which
can be
* off by default against AWS
* configured to be one for all buckets (external s3 server)
* dynamically set when s3guard is enabled.
This could be done with some predicate isListingConsistent(), or just probed as
an FS option (fs.s3a.consistent.listing) which could be set explicitly in
configs, and set dynamically when s3guard is on. That's actually the simpleset
route all round
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
