[jira] [Updated] (HADOOP-18996) S3A to provide full support for S3 Express One Zone
[
https://issues.apache.org/jira/browse/HADOOP-18996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-18996:
Description:
HADOOP-18995 upgrades the SDK version which allows connecting to a s3 express
one zone support.
Complete support needs to be added to address tests that fail with s3 express
one zone, additional tests, documentation etc.
* hadoop-common path capability to indicate that treewalking may encounter
missing dirs
* use this in treewalking code in shell, mapreduce FileInputFormat etc to not
fail during treewalks
* extra path capability for s3express too.
* tests for this
* anything else
A filesystem can now be probed for inconsistent directory listings through
{{fs.hasPathCapability(path, "fs.capability.directory.listing.inconsistent")}}
If true, then treewalking code SHOULD NOT report a failure if, when walking
into a subdirectory, a list/getFileStatus on that directory raises a
FileNotFoundExceptin.
was:
HADOOP-18995 upgrades the SDK version which allows connecting to a s3 express
one zone support.
Complete support needs to be added to address tests that fail with s3 express
one zone, additional tests, documentation etc.
* hadoop-common path capability to indicate that treewalking may encounter
missing dirs
* use this in treewalking code in shell, mapreduce FileInputFormat etc to not
fail during treewalks
* extra path capability for s3express too.
* tests for this
* anything else
A filesystem can now be probed for inconsistent directoriy listings through
{{fs.hasPathCapability(path, "fs.capability.directory.listing.inconsistent")}}
If true, then treewalking code SHOULD NOT report a failure if, when walking
into a subdirectory, a list/getFileStatus on that directory raises a
FileNotFoundExceptin.
> S3A to provide full support for S3 Express One Zone
> ---
>
> Key: HADOOP-18996
> URL: https://issues.apache.org/jira/browse/HADOOP-18996
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.4.0
>Reporter: Ahmar Suhail
>Assignee: Steve Loughran
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.7-aws
>
>
> HADOOP-18995 upgrades the SDK version which allows connecting to a s3 express
> one zone support.
> Complete support needs to be added to address tests that fail with s3 express
> one zone, additional tests, documentation etc.
> * hadoop-common path capability to indicate that treewalking may encounter
> missing dirs
> * use this in treewalking code in shell, mapreduce FileInputFormat etc to not
> fail during treewalks
> * extra path capability for s3express too.
> * tests for this
> * anything else
> A filesystem can now be probed for inconsistent directory listings through
> {{fs.hasPathCapability(path, "fs.capability.directory.listing.inconsistent")}}
> If true, then treewalking code SHOULD NOT report a failure if, when walking
> into a subdirectory, a list/getFileStatus on that directory raises a
> FileNotFoundExceptin.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18996) S3A to provide full support for S3 Express One Zone
[
https://issues.apache.org/jira/browse/HADOOP-18996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-18996:
Release Note: Hadoop S3A connector has explicit awareness of and support
for S3Express storage.A filesystem can now be probed for inconsistent
directoriy listings through fs.hasPathCapability(path,
"fs.capability.directory.listing.inconsistent"). If true, then treewalking code
SHOULD NOT report a failure if, when walking into a subdirectory, a
list/getFileStatus on that directory raises a FileNotFoundException. (was:
Hadoop S3A connector has explicit awareness of and support for S3Express
storage. Hadoop-common and hadoop-mapreduce treewalking code (shell commands,
FileInputFormat,...) can cope with paths with incomplete uploads being visible.
)
> S3A to provide full support for S3 Express One Zone
> ---
>
> Key: HADOOP-18996
> URL: https://issues.apache.org/jira/browse/HADOOP-18996
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.4.0
>Reporter: Ahmar Suhail
>Assignee: Steve Loughran
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.7-aws
>
>
> HADOOP-18995 upgrades the SDK version which allows connecting to a s3 express
> one zone support.
> Complete support needs to be added to address tests that fail with s3 express
> one zone, additional tests, documentation etc.
> * hadoop-common path capability to indicate that treewalking may encounter
> missing dirs
> * use this in treewalking code in shell, mapreduce FileInputFormat etc to not
> fail during treewalks
> * extra path capability for s3express too.
> * tests for this
> * anything else
> A filesystem can now be probed for inconsistent directoriy listings through
> {{fs.hasPathCapability(path, "fs.capability.directory.listing.inconsistent")}}
> If true, then treewalking code SHOULD NOT report a failure if, when walking
> into a subdirectory, a list/getFileStatus on that directory raises a
> FileNotFoundExceptin.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18996) S3A to provide full support for S3 Express One Zone
[
https://issues.apache.org/jira/browse/HADOOP-18996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-18996:
Description:
HADOOP-18995 upgrades the SDK version which allows connecting to a s3 express
one zone support.
Complete support needs to be added to address tests that fail with s3 express
one zone, additional tests, documentation etc.
* hadoop-common path capability to indicate that treewalking may encounter
missing dirs
* use this in treewalking code in shell, mapreduce FileInputFormat etc to not
fail during treewalks
* extra path capability for s3express too.
* tests for this
* anything else
A filesystem can now be probed for inconsistent directoriy listings through
{{fs.hasPathCapability(path, "fs.capability.directory.listing.inconsistent")}}
If true, then treewalking code SHOULD NOT report a failure if, when walking
into a subdirectory, a list/getFileStatus on that directory raises a
FileNotFoundExceptin.
was:
HADOOP-18995 upgrades the SDK version which allows connecting to a s3 express
one zone support.
Complete support needs to be added to address tests that fail with s3 express
one zone, additional tests, documentation etc.
* hadoop-common path capability to indicate that treewalking may encounter
missing dirs
* use this in treewalking code in shell, mapreduce FileInputFormat etc to not
fail during treewalks
* extra path capability for s3express too.
* tests for this
* anything else
> S3A to provide full support for S3 Express One Zone
> ---
>
> Key: HADOOP-18996
> URL: https://issues.apache.org/jira/browse/HADOOP-18996
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.4.0
>Reporter: Ahmar Suhail
>Assignee: Steve Loughran
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.7-aws
>
>
> HADOOP-18995 upgrades the SDK version which allows connecting to a s3 express
> one zone support.
> Complete support needs to be added to address tests that fail with s3 express
> one zone, additional tests, documentation etc.
> * hadoop-common path capability to indicate that treewalking may encounter
> missing dirs
> * use this in treewalking code in shell, mapreduce FileInputFormat etc to not
> fail during treewalks
> * extra path capability for s3express too.
> * tests for this
> * anything else
> A filesystem can now be probed for inconsistent directoriy listings through
> {{fs.hasPathCapability(path, "fs.capability.directory.listing.inconsistent")}}
> If true, then treewalking code SHOULD NOT report a failure if, when walking
> into a subdirectory, a list/getFileStatus on that directory raises a
> FileNotFoundExceptin.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18996) S3A to provide full support for S3 Express One Zone
[ https://issues.apache.org/jira/browse/HADOOP-18996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shilun Fan updated HADOOP-18996: Hadoop Flags: Reviewed Target Version/s: 3.4.0 > S3A to provide full support for S3 Express One Zone > --- > > Key: HADOOP-18996 > URL: https://issues.apache.org/jira/browse/HADOOP-18996 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.4.0 >Reporter: Ahmar Suhail >Assignee: Steve Loughran >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.7-aws > > > HADOOP-18995 upgrades the SDK version which allows connecting to a s3 express > one zone support. > Complete support needs to be added to address tests that fail with s3 express > one zone, additional tests, documentation etc. > * hadoop-common path capability to indicate that treewalking may encounter > missing dirs > * use this in treewalking code in shell, mapreduce FileInputFormat etc to not > fail during treewalks > * extra path capability for s3express too. > * tests for this > * anything else -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18996) S3A to provide full support for S3 Express One Zone
[ https://issues.apache.org/jira/browse/HADOOP-18996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmar Suhail updated HADOOP-18996: -- Fix Version/s: 3.3.7-aws (was: 3.3.6-aws) > S3A to provide full support for S3 Express One Zone > --- > > Key: HADOOP-18996 > URL: https://issues.apache.org/jira/browse/HADOOP-18996 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.4.0 >Reporter: Ahmar Suhail >Assignee: Steve Loughran >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.7-aws > > > HADOOP-18995 upgrades the SDK version which allows connecting to a s3 express > one zone support. > Complete support needs to be added to address tests that fail with s3 express > one zone, additional tests, documentation etc. > * hadoop-common path capability to indicate that treewalking may encounter > missing dirs > * use this in treewalking code in shell, mapreduce FileInputFormat etc to not > fail during treewalks > * extra path capability for s3express too. > * tests for this > * anything else -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18996) S3A to provide full support for S3 Express One Zone
[ https://issues.apache.org/jira/browse/HADOOP-18996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmar Suhail updated HADOOP-18996: -- Fix Version/s: 3.3.6-aws > S3A to provide full support for S3 Express One Zone > --- > > Key: HADOOP-18996 > URL: https://issues.apache.org/jira/browse/HADOOP-18996 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.4.0 >Reporter: Ahmar Suhail >Assignee: Steve Loughran >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.6-aws > > > HADOOP-18995 upgrades the SDK version which allows connecting to a s3 express > one zone support. > Complete support needs to be added to address tests that fail with s3 express > one zone, additional tests, documentation etc. > * hadoop-common path capability to indicate that treewalking may encounter > missing dirs > * use this in treewalking code in shell, mapreduce FileInputFormat etc to not > fail during treewalks > * extra path capability for s3express too. > * tests for this > * anything else -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18996) S3A to provide full support for S3 Express One Zone
[ https://issues.apache.org/jira/browse/HADOOP-18996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18996: Parent Issue: HADOOP-18886 (was: HADOOP-18477) > S3A to provide full support for S3 Express One Zone > --- > > Key: HADOOP-18996 > URL: https://issues.apache.org/jira/browse/HADOOP-18996 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.4.0 >Reporter: Ahmar Suhail >Assignee: Steve Loughran >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > > HADOOP-18995 upgrades the SDK version which allows connecting to a s3 express > one zone support. > Complete support needs to be added to address tests that fail with s3 express > one zone, additional tests, documentation etc. > * hadoop-common path capability to indicate that treewalking may encounter > missing dirs > * use this in treewalking code in shell, mapreduce FileInputFormat etc to not > fail during treewalks > * extra path capability for s3express too. > * tests for this > * anything else -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18996) S3A to provide full support for S3 Express One Zone
[ https://issues.apache.org/jira/browse/HADOOP-18996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18996: Release Note: Hadoop S3A connector has explicit awareness of and support for S3Express storage. Hadoop-common and hadoop-mapreduce treewalking code (shell commands, FileInputFormat,...) can cope with paths with incomplete uploads being visible. > S3A to provide full support for S3 Express One Zone > --- > > Key: HADOOP-18996 > URL: https://issues.apache.org/jira/browse/HADOOP-18996 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.4.0 >Reporter: Ahmar Suhail >Assignee: Steve Loughran >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > > HADOOP-18995 upgrades the SDK version which allows connecting to a s3 express > one zone support. > Complete support needs to be added to address tests that fail with s3 express > one zone, additional tests, documentation etc. > * hadoop-common path capability to indicate that treewalking may encounter > missing dirs > * use this in treewalking code in shell, mapreduce FileInputFormat etc to not > fail during treewalks > * extra path capability for s3express too. > * tests for this > * anything else -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18996) S3A to provide full support for S3 Express One Zone
[ https://issues.apache.org/jira/browse/HADOOP-18996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-18996: Labels: pull-request-available (was: ) > S3A to provide full support for S3 Express One Zone > --- > > Key: HADOOP-18996 > URL: https://issues.apache.org/jira/browse/HADOOP-18996 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.4.0 >Reporter: Ahmar Suhail >Assignee: Steve Loughran >Priority: Major > Labels: pull-request-available > > HADOOP-18995 upgrades the SDK version which allows connecting to a s3 express > one zone support. > Complete support needs to be added to address tests that fail with s3 express > one zone, additional tests, documentation etc. > * hadoop-common path capability to indicate that treewalking may encounter > missing dirs > * use this in treewalking code in shell, mapreduce FileInputFormat etc to not > fail during treewalks > * extra path capability for s3express too. > * tests for this > * anything else -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18996) S3A to provide full support for S3 Express One Zone
[ https://issues.apache.org/jira/browse/HADOOP-18996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18996: Description: HADOOP-18995 upgrades the SDK version which allows connecting to a s3 express one zone support. Complete support needs to be added to address tests that fail with s3 express one zone, additional tests, documentation etc. * hadoop-common path capability to indicate that treewalking may encounter missing dirs * use this in treewalking code in shell, mapreduce FileInputFormat etc to not fail during treewalks * extra path capability for s3express too. * tests for this * anything else was: HADOOP-18995 upgrades the SDK version which allows connecting to a s3 express one zone support. Complete support needs to be added to address tests that fail with s3 express one zone, additional tests, documentation etc. > S3A to provide full support for S3 Express One Zone > --- > > Key: HADOOP-18996 > URL: https://issues.apache.org/jira/browse/HADOOP-18996 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.4.0 >Reporter: Ahmar Suhail >Assignee: Steve Loughran >Priority: Major > > HADOOP-18995 upgrades the SDK version which allows connecting to a s3 express > one zone support. > Complete support needs to be added to address tests that fail with s3 express > one zone, additional tests, documentation etc. > * hadoop-common path capability to indicate that treewalking may encounter > missing dirs > * use this in treewalking code in shell, mapreduce FileInputFormat etc to not > fail during treewalks > * extra path capability for s3express too. > * tests for this > * anything else -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18996) S3A to provide full support for S3 Express One Zone
[ https://issues.apache.org/jira/browse/HADOOP-18996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18996: Summary: S3A to provide full support for S3 Express One Zone (was: Add necessary software support for S3 Express One Zone) > S3A to provide full support for S3 Express One Zone > --- > > Key: HADOOP-18996 > URL: https://issues.apache.org/jira/browse/HADOOP-18996 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.4.0 >Reporter: Ahmar Suhail >Assignee: Steve Loughran >Priority: Major > > HADOOP-18995 upgrades the SDK version which allows connecting to a s3 express > one zone support. > Complete support needs to be added to address tests that fail with s3 express > one zone, additional tests, documentation etc. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
