Mehakmeet Singh created HADOOP-18201:
----------------------------------------
Summary: Remove base and bucket overrides for endpoint in
ITestS3ARequesterPays.java
Key: HADOOP-18201
URL: https://issues.apache.org/jira/browse/HADOOP-18201
Project: Hadoop Common
Issue Type: Bug
Components: fs/s3
Affects Versions: 3.3.3
Reporter: Mehakmeet Singh
Assignee: Daniel Carl Jones
If a user has set the endpoint for their bucket in a test environment, we
should ignore that in a test that won't use the bucket that the endpoint is set
for. In this case, we are using "s3a://usgs-landsat/" which is in the region
us-west-2, and would fail if the user has explicitly set the endpoint to
something else.
Example (I have set the endpoint to ap-south-1):
{code:java}
[ERROR]
testRequesterPaysDisabledFails(org.apache.hadoop.fs.s3a.ITestS3ARequesterPays)
Time elapsed: 9.323 s <<< ERROR!
org.apache.hadoop.fs.s3a.AWSRedirectException: getFileStatus on
s3a://usgs-landsat/collection02/catalog.json:
com.amazonaws.services.s3.model.AmazonS3Exception: The bucket is in this
region: us-west-2. Please use this region to retry the request (Service: Amazon
S3; Status Code: 301; Error Code: 301 Moved Permanently; Request ID:
Z09V8PMEEN5PHDRZ; S3 Extended Request ID:
B7KDQntCuVmLJAyXvuY4UNXjdUrgn3xd26n8u7ThueNNxvKas6g3RsXo7oxBcvHrpcous2L+Lbk=;
Proxy: null), S3 Extended Request ID:
B7KDQntCuVmLJAyXvuY4UNXjdUrgn3xd26n8u7ThueNNxvKas6g3RsXo7oxBcvHrpcous2L+Lbk=:301
Moved Permanently: The bucket is in this region: us-west-2. Please use this
region to retry the request (Service: Amazon S3; Status Code: 301; Error Code:
301 Moved Permanently; Request ID: Z09V8PMEEN5PHDRZ; S3 Extended Request ID:
B7KDQntCuVmLJAyXvuY4UNXjdUrgn3xd26n8u7ThueNNxvKas6g3RsXo7oxBcvHrpcous2L+Lbk=;
Proxy: null)
at org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:233)
at org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:171)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:3440)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:3346)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.extractOrFetchSimpleFileStatus(S3AFileSystem.java:4890)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$executeOpen$6(S3AFileSystem.java:1437)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.invokeTrackingDuration(IOStatisticsBinding.java:543)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.lambda$trackDurationOfOperation$5(IOStatisticsBinding.java:524)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.trackDuration(IOStatisticsBinding.java:445)
at org.apache.hadoop.fs.s3a.S3AFileSystem.executeOpen(S3AFileSystem.java:1435)
at org.apache.hadoop.fs.s3a.S3AFileSystem.open(S3AFileSystem.java:1409)
<......>{code}
CC: [[email protected]] [~mthakur] [~dannycjones]
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
