[
https://issues.apache.org/jira/browse/HADOOP-17771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-17771:
------------------------------------
Summary: S3AFS creation fails without region set in ~/.aws/config (was:
S3AFS creation fails without region set ~/.aws/config)
> S3AFS creation fails without region set in ~/.aws/config
> --------------------------------------------------------
>
> Key: HADOOP-17771
> URL: https://issues.apache.org/jira/browse/HADOOP-17771
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.3.1
> Environment: Host outside EC2 and without the file ~/.aws/config or
> without a region set in it
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Blocker
>
> If you don't have {{fs.s3a.endpoint}} set and lack a region set in
> env var {{AWS_REGION_ENV_VAR}}, system property {{aws.region}} or the file
> ~/.aws/config
> then S3A FS creation fails with the message
> "Unable to find a region via the region provider chain."
> This is caused by the move to the AWS S3 client builder API in HADOOP-13551
> This is pretty dramatic and no doubt everyone will be asking "why didn't you
> notice this?",
> But in fact there are some reasons.
> # when running in EC2, all is well. Meaning our big test runs were all happy.
> # if a developer has fs.s3a.endpoint set for the test bucket, all is well.
> Those of us who work with buckets in the "regions tend to do this, not
> least because it can save a HEAD request every time an FS is created.
> # if you have a region set in ~/.aws/config then all is well
> reason #3 is the real surprise and the one which has really caught out. Even
> my tests against buckets in usw-2 through central didn't fail because of
> course I, like my colleagues, have the AWS S3 client installed locally. This
> was sufficient to make the problem go away. It is also why this has been an
> intermittent problem on test clusters outside AWS infra: it really depended
> on the VM/docker image whether things worked or not.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
