Read from AWS s3 with out having to hard-code sensitive keys

[Krishna Rao]

Hi all,

Is there a method for reading from s3 without having to hard-code keys? The
only 2 ways I've found both require this:

1. Set conf in code e.g.:
sc.hadoopConfiguration().set("fs.s3.awsAccessKeyId", "<aws_key>")
sc.hadoopConfiguration().set("fs.s3.awsSecretAccessKey",
"<aws_secret_key>")

2. Set keys in URL, e.g.:
sc.textFile("s3n://<aws_key>@<aws_secret_key>/bucket/test/testdata")


Both if which I'm reluctant to do within production code!


Cheers