The current version of Hadoop in EMR (both Apache and MapR) does not support the IAM authentication to S3 without the credentials in core-site. I believe the support has been integrated into Hadoop 2.6 … so when the EMR distributions upgrade to that level, the access you request should be supported.
Did you successfully configure the drill-bit to use the full EMRFS jars, or did you default to the older jets3t support ? If you have the classpath settings for full emrfs support, please share them with the group (and I will integrate that support into the MapR EMR bootstrap action under development for Drill). Regards, David On May 11, 2015, at 6:55 PM, Alonzo Barnett <abarn...@lexmark.com> wrote: > I am looking into using Drill with AWS EMR. My organization uses IAM roles > with EMR in order to rotate security credentials. > > In a first round of testing I successfully connected Drill to S3 on a > cluster without IAM role based credential rotation which required placing > access and secret keys in core-site.xml. > > It is possible to still use Drill with S3 without hardcoding credentials > into core-site? With emrfs there is some work by AWS to rotate > credentials, and I would like to exploit the modification to core-site if > possible. > > -- > v/r, > Alonzo Barnett
