Adam Budde created HADOOP-11670:
-----------------------------------
Summary: Fix IAM instance profile auth for s3a (broken in
HADOOP-11446)
Key: HADOOP-11670
URL: https://issues.apache.org/jira/browse/HADOOP-11670
Project: Hadoop Common
Issue Type: Sub-task
Components: fs/s3
Affects Versions: 2.6.0
Reporter: Adam Budde
Fix For: 2.7.0
One big advantage provided by the s3a filesystem is the ability to use an IAM
instance profile in order to authenticate when attempting to access an S3
bucket from an EC2 instance. This eliminates the need to deploy AWS account
credentials to the instance or to provide them to Hadoop via the
fs.s3a.awsAccessKeyId and fs.s3a.awsSecretAccessKey params.
The patch submitted to resolve HADOOP-11446 breaks this behavior by using the
S3Credentials class to read the value of these two params (this change is
unrelated to resolving HADOOP-11446).
S3AFileSystem.java, lines 161-170:
{code}
// Try to get our credentials or just connect anonymously
S3Credentials s3Credentials = new S3Credentials();
s3Credentials.initialize(name, conf);
AWSCredentialsProviderChain credentials = new AWSCredentialsProviderChain(
new BasicAWSCredentialsProvider(s3Credentials.getAccessKey(),
s3Credentials.getSecretAccessKey()),
new InstanceProfileCredentialsProvider(),
new AnonymousAWSCredentialsProvider()
);
{code}
As you can see, the getAccessKey() and getSecretAccessKey() methods from the
S3Credentials class are now used to provide constructor arguments to
BasicAWSCredentialsProvider. These methods will raise an exception if the
fs.s3a.awsAccessKeyId or fs.s3a.awsSecretAccessKey params are missing,
respectively. If a user is relying on an IAM instance profile to authenticate
to an S3 bucket and therefore doesn't supply values for these params, they will
receive an exception and won't be able to access the bucket.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
