[
https://issues.apache.org/jira/browse/HADOOP-9293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13575552#comment-13575552
]
Andy Sautins commented on HADOOP-9293:
--------------------------------------
Steve,
Stepping back a bit what I'd like to achieve is to access S3 from an Amazon
Elastic Map Reduce cluster client without having to add S3 credentials to
another configuration file. This should be possible to achieve specifically
when connecting to the cluster using a SOCKS proxy.
What would be ideal is if setting
hadoop.rpc.socket.factory.class.ClientProtocol to
org.apache.hadoop.net.SocksSocketFactory would use the S3 configuration
settings from the proxy host since the S3 credentials are already in the
configuration files on the EMR cluster.
Not sure if that makes more sense or not, but you are probably right my
approach may be suspect, but I do think there is still something here around
not having to add the S3 credential configuration to another file on the client.
> For S3 use credentials file
> ---------------------------
>
> Key: HADOOP-9293
> URL: https://issues.apache.org/jira/browse/HADOOP-9293
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs/s3
> Affects Versions: 1.0.2
> Environment: Linux
> Reporter: Andy Sautins
> Priority: Minor
> Labels: features, newbie
> Attachments: HADOOP-9293.patch
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> The following document describes the current way that S3 credentials can be
> specified ( http://wiki.apache.org/hadoop/AmazonS3 ). In summary they are:
> * in the S3 URI.
> * in the hadoop-site.xml file as
> ** fs.s3.awsAccessKeyId
> ** fs.s3.awsSecretAccessKey
> ** fs.s3n.awsAccessKeyId
> ** fs.s3n.aswSecretAccessKey
> The amazon EMR tool elastic-mapreduce already provide the ability to use a
> credentials file ( see
> http://s3.amazonaws.com/awsdocs/ElasticMapReduce/latest/emr-qrc.pdf ).
> I would propose that we allow roughly the same access to credentials through
> a credentials file that is currently provided by elastic-mapreduce. This
> should allow for centralized administration of credentials which should be
> positive for security.
> I propose the following properties:
> {quote}
>
> <property><name>f3.s3.awsCredentialsFile</name><value>/path/to/file</value></property>
>
> <property><name>fs.s3n.awsCredentialsFile</name><value>/path/to/file</value></property>
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
