[
https://issues.apache.org/jira/browse/HADOOP-16567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-16567:
------------------------------------
Parent: HADOOP-19353 (was: HADOOP-18477)
> S3A Secret access to fall back to XML if credential provider raises IOE.
> ------------------------------------------------------------------------
>
> Key: HADOOP-16567
> URL: https://issues.apache.org/jira/browse/HADOOP-16567
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.1.2
> Reporter: Steve Loughran
> Priority: Major
>
> This is hive related. Hive can put secrets into a JCEKS file which only hive
> may read.
> S3A file systems created on behalf of a user do not have access to this file.
> Yet it is listed as the credential provider in the
> hadoop.credential.providers option in core-site -and that is marked as final.
> When the S3 a initializre() method looks up passwords and encryption keys,
> the failure to open the file raises an IOE -and the FS cannot be instantiated.
> Proposed: {{S3AUtils.lookupPassword()}} to catch such exceptions, and fall
> back to using {{Configuration.get()}} and so retrieve any property in the
> XML. If there is one failing here, it is if the user did want to read from a
> credential provider, the failure to read the credential will be lost, and the
> filesystem will simply get the default value.
> There is a side issue, that permission exceptions can surface as found not
> found exceptions, which are then wrapped as generic IOEs in Configuration. It
> will be hard and brittle to attempt to only respond to permission
> restrictions. We could look at improving {{Configuration.getPassword()}} but
> that class is so widely used, I am not in a rush to break things.
> I think this means we have to add another option. Trying to be clever about
> when to fall back versus when to rethrow the exception is doomed.
> If this works for S3A, we will need to consider replicating it for ABFS.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
