[ https://issues.apache.org/jira/browse/HADOOP-17705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17617658#comment-17617658 ]
Steve Loughran commented on HADOOP-17705: ----------------------------------------- you can put it up. usual test process applies. it was a pretty traumatic change with the factory etc -things broke and we didn't find out until after shipping. I am scared of this > S3A to add option fs.s3a.endpoint.region to set AWS region > ---------------------------------------------------------- > > Key: HADOOP-17705 > URL: https://issues.apache.org/jira/browse/HADOOP-17705 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 > Reporter: Mehakmeet Singh > Assignee: Mehakmeet Singh > Priority: Major > Labels: pull-request-available > Fix For: 3.3.2 > > Time Spent: 3h > Remaining Estimate: 0h > > Currently, AWS region is either constructed via the endpoint URL, by making > an assumption that the 2nd component after delimiter "." is the region in > endpoint URL, which doesn't work for private links and sets the default to > us-east-1 thus causing authorization issue w.r.t the private link. > The option fs.s3a.endpoint.region allows this to be explicitly set > h2. how to set the s3 region on older hadoop releases > For anyone who needs to set the signing region on older versions of the s3a > client *you do not need this festure*. instead just provide a custom endpoint > to region mapping json file > # Download the default region mapping file > [awssdk_config_default.json|https://github.com/aws/aws-sdk-java/blob/master/aws-java-sdk-core/src/main/resources/com/amazonaws/internal/config/awssdk_config_default.json] > # Add a new regular expression to map the endpoint/hostname to the target > region > # Save the file as {{/etc/hadoop/conf/awssdk_config_override.json}} > # verify basic hadop fs -ls commands work > # copy to the rest of the cluster. > # There should be no need to restart any services -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org