[ https://issues.apache.org/jira/browse/HADOOP-14620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16073971#comment-16073971 ]
Steve Loughran commented on HADOOP-14620: ----------------------------------------- Here's the XML options I'm using to talk to the bucket {{s3a://hwdev-steve-london}} hosted in London, which is v4 only. try talking to the London bucket yourself, to see if its the special case of s3 ireland at play. {code} <property> <name>fs.s3a.bucket.hwdev-steve-london.endpoint</name> <value>${london.endpoint}</value> </property> <property> <name>london.endpoint</name> <value>s3.eu-west-2.amazonaws.com</value> </property> {code} > S3A authentication failure for regions other than us-east-1 > ----------------------------------------------------------- > > Key: HADOOP-14620 > URL: https://issues.apache.org/jira/browse/HADOOP-14620 > Project: Hadoop Common > Issue Type: Bug > Components: fs/s3 > Affects Versions: 2.8.0, 2.7.3 > Reporter: Ilya Fourmanov > Attachments: s3-403.txt > > > hadoop fs s3a:// operations fail authentication for s3 buckets hosted in > regions other than default us-east-1 > Steps to reproduce: > # create s3 bucket in eu-west-1 > # Using IAM instance profile or fs.s3a.access.key/fs.s3a.secret.key run > following command: > {code} > hadoop --loglevel DEBUG -D fs.s3a.endpoint=s3.eu-west-1.amazonaws.com -ls > s3a://your-eu-west-1-hosted-bucket/ > {code} > Expected behaviour: > You will see listing of the bucket > Actual behaviour: > You will get 403 Authentication Denied response for AWS S3. > Reason is mismatch in string to sign as defined in > http://docs.aws.amazon.com/AmazonS3/latest/dev/RESTAuthentication.html > provided by hadoop and expected by AWS. > If you use https://aws.amazon.com/code/199 to analyse StringToSignBytes > returned by AWS, you will see that AWS expects CanonicalizedResource to be in > form > /your-eu-west-1-hosted-bucket{color:red}.s3.eu-west-1.amazonaws.com{color}/. > Hadoop provides it as /your-eu-west-1-hosted-bucket/ > Note that AWS documentation doesn't explicitly state that endpoint or full > dns address should be appended to CanonicalizedResource however practice > shows it is actually required. > I've also submitted this to AWS for them to correct behaviour or > documentation. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org