[
https://issues.apache.org/jira/browse/HADOOP-18996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-18996:
------------------------------------
Release Note: Hadoop S3A connector has explicit awareness of and support
for S3Express storage.A filesystem can now be probed for inconsistent
directoriy listings through fs.hasPathCapability(path,
"fs.capability.directory.listing.inconsistent"). If true, then treewalking code
SHOULD NOT report a failure if, when walking into a subdirectory, a
list/getFileStatus on that directory raises a FileNotFoundException. (was:
Hadoop S3A connector has explicit awareness of and support for S3Express
storage. Hadoop-common and hadoop-mapreduce treewalking code (shell commands,
FileInputFormat,...) can cope with paths with incomplete uploads being visible.
)
> S3A to provide full support for S3 Express One Zone
> ---------------------------------------------------
>
> Key: HADOOP-18996
> URL: https://issues.apache.org/jira/browse/HADOOP-18996
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.4.0
> Reporter: Ahmar Suhail
> Assignee: Steve Loughran
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.7-aws
>
>
> HADOOP-18995 upgrades the SDK version which allows connecting to a s3 express
> one zone support.
> Complete support needs to be added to address tests that fail with s3 express
> one zone, additional tests, documentation etc.
> * hadoop-common path capability to indicate that treewalking may encounter
> missing dirs
> * use this in treewalking code in shell, mapreduce FileInputFormat etc to not
> fail during treewalks
> * extra path capability for s3express too.
> * tests for this
> * anything else
> A filesystem can now be probed for inconsistent directoriy listings through
> {{fs.hasPathCapability(path, "fs.capability.directory.listing.inconsistent")}}
> If true, then treewalking code SHOULD NOT report a failure if, when walking
> into a subdirectory, a list/getFileStatus on that directory raises a
> FileNotFoundExceptin.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
