Mehakmeet Singh created HADOOP-18201: ----------------------------------------
Summary: Remove base and bucket overrides for endpoint in ITestS3ARequesterPays.java Key: HADOOP-18201 URL: https://issues.apache.org/jira/browse/HADOOP-18201 Project: Hadoop Common Issue Type: Bug Components: fs/s3 Affects Versions: 3.3.3 Reporter: Mehakmeet Singh Assignee: Daniel Carl Jones If a user has set the endpoint for their bucket in a test environment, we should ignore that in a test that won't use the bucket that the endpoint is set for. In this case, we are using "s3a://usgs-landsat/" which is in the region us-west-2, and would fail if the user has explicitly set the endpoint to something else. Example (I have set the endpoint to ap-south-1): {code:java} [ERROR] testRequesterPaysDisabledFails(org.apache.hadoop.fs.s3a.ITestS3ARequesterPays) Time elapsed: 9.323 s <<< ERROR! org.apache.hadoop.fs.s3a.AWSRedirectException: getFileStatus on s3a://usgs-landsat/collection02/catalog.json: com.amazonaws.services.s3.model.AmazonS3Exception: The bucket is in this region: us-west-2. Please use this region to retry the request (Service: Amazon S3; Status Code: 301; Error Code: 301 Moved Permanently; Request ID: Z09V8PMEEN5PHDRZ; S3 Extended Request ID: B7KDQntCuVmLJAyXvuY4UNXjdUrgn3xd26n8u7ThueNNxvKas6g3RsXo7oxBcvHrpcous2L+Lbk=; Proxy: null), S3 Extended Request ID: B7KDQntCuVmLJAyXvuY4UNXjdUrgn3xd26n8u7ThueNNxvKas6g3RsXo7oxBcvHrpcous2L+Lbk=:301 Moved Permanently: The bucket is in this region: us-west-2. Please use this region to retry the request (Service: Amazon S3; Status Code: 301; Error Code: 301 Moved Permanently; Request ID: Z09V8PMEEN5PHDRZ; S3 Extended Request ID: B7KDQntCuVmLJAyXvuY4UNXjdUrgn3xd26n8u7ThueNNxvKas6g3RsXo7oxBcvHrpcous2L+Lbk=; Proxy: null) at org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:233) at org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:171) at org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:3440) at org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:3346) at org.apache.hadoop.fs.s3a.S3AFileSystem.extractOrFetchSimpleFileStatus(S3AFileSystem.java:4890) at org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$executeOpen$6(S3AFileSystem.java:1437) at org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.invokeTrackingDuration(IOStatisticsBinding.java:543) at org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.lambda$trackDurationOfOperation$5(IOStatisticsBinding.java:524) at org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.trackDuration(IOStatisticsBinding.java:445) at org.apache.hadoop.fs.s3a.S3AFileSystem.executeOpen(S3AFileSystem.java:1435) at org.apache.hadoop.fs.s3a.S3AFileSystem.open(S3AFileSystem.java:1409) <......>{code} CC: [~ste...@apache.org] [~mthakur] [~dannycjones] -- This message was sent by Atlassian Jira (v8.20.1#820001) --------------------------------------------------------------------- To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org