[ https://issues.apache.org/jira/browse/HADOOP-13674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543380#comment-15543380 ]
Chris Nauroth commented on HADOOP-13674: ---------------------------------------- Here is the error I see when I attempt to access a bucket in us-west-2, but with {{fs.s3a.endpoint}} pointing to us-west-1. {code} > hadoop fs -D fs.s3a.endpoint=s3-us-west-1.amazonaws.com -ls > s3a://cnauroth-test-aws-s3a-logs/ ls: getFileStatus on : com.amazonaws.services.s3.model.AmazonS3Exception: The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint. (Service: Amazon S3; Status Code: 301; Error Code: PermanentRedirect; Request ID: EC6C7FCF8B40B27C), S3 Extended Request ID: EQ1h4MW2CRLV4ZJBGs2xz2CVXwsfGS5X+ByWfyl1tdzbXbf7bFn5DI5pejcWWCmu1/P/uDEOjaU=: The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint. (Service: Amazon S3; Status Code: 301; Error Code: PermanentRedirect; Request ID: EC6C7FCF8B40B27C) {code} It says that the endpoint is wrong, but it doesn't say which endpoint is correct. Turning on debug logging shows that the information does come back in the HTTP 301 response: {code} > hadoop --loglevel DEBUG fs -D fs.s3a.endpoint=s3-us-west-1.amazonaws.com -ls > s3a://cnauroth-test-aws-s3a-logs/ ... 16/10/03 13:35:28 DEBUG http.wire: << "<Error><Code>PermanentRedirect</Code><Message>The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.</Message><Bucket>cnauroth-test-aws-s3a-logs</Bucket><Endpoint>cnauroth-test-aws-s3a-logs.s3-us-west-2.amazonaws.com</Endpoint><RequestId>995927D9C5DD8F90</RequestId><HostId>LK/kvbR/gdnxyr5JXj1L41TOfcO4VBF6MtT8FkwOXXyRdjhasccrHc2bux+b4uHSqJmiBEgHJcI=</HostId></Error>" ... {code} It appears that the AWS SDK maps the {{<Endpoint>}} element into the map returned by [{{AmazonS3Exception#getAdditionalDetails()}}|http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/model/AmazonS3Exception.html#getAdditionalDetails--]. We can use that to get the information and put it into the exception thrown from S3A. > S3A can provide a more detailed error message when accessing a bucket through > an incorrect S3 endpoint. > ------------------------------------------------------------------------------------------------------- > > Key: HADOOP-13674 > URL: https://issues.apache.org/jira/browse/HADOOP-13674 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 > Reporter: Chris Nauroth > Assignee: Chris Nauroth > Priority: Minor > > When accessing the S3 service through a region-specific endpoint, the bucket > must be located in that region. If the client attempts to access a bucket > that is not located in that region, then the service replies with a 301 > redirect and the correct region endpoint. However, the exception thrown by > S3A does not include the correct endpoint. If we included that information > in the exception, it would make it easier for users to diagnose and fix > incorrect configuration. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org