[
https://issues.apache.org/jira/browse/HADOOP-18908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17767499#comment-17767499
]
ASF GitHub Bot commented on HADOOP-18908:
-----------------------------------------
ahmarsuhail opened a new pull request, #6106:
URL: https://github.com/apache/hadoop/pull/6106
### Description of PR
Removes the region probe to S3 and reinstates the region resolution logic
that was there before the SDK V2 upgrade.
* Region probe is no longer needed as SDKV2 now supports cross region
support.
* If endpoint is configured and it is not the central (s3.amazonaws.com)
one, parse the endpoint for the region. If unable to
parse region from endpoint (or endpoint is null), look at the the region
configuration.
* If region configured, set it in the client. If it is null, use US_EAST_1.
If config is empty string, use SDK region resolution chain.
### How was this patch tested?
Testing in progress.
> Improve s3a region handling, including determining from endpoint
> ----------------------------------------------------------------
>
> Key: HADOOP-18908
> URL: https://issues.apache.org/jira/browse/HADOOP-18908
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.4.0
> Reporter: Steve Loughran
> Assignee: Ahmar Suhail
> Priority: Major
>
> s3a now requires the fs.s3a.endpoint.region to be set; and while it can
> determine it from a network call, this takes time and doesn't work for third
> party stores.
> proposed
> * reinstate parsing of the fs.3a.endpoint url to automatically determine
> region from well known endoints (and vplink ones)
> * don't try to talk to AWS S3 if endpoint isn't an aws one: for that caller
> must declare (HADOOP-18673)
> * document this in v2 migration, including stack traces of falures
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
