Steve Loughran created HADOOP-16607:
---------------------------------------
Summary: s3a attempts to look up password/encryption fail if JCEKS
file unreadable
Key: HADOOP-16607
URL: https://issues.apache.org/jira/browse/HADOOP-16607
Project: Hadoop Common
Issue Type: Bug
Reporter: Steve Loughran
Hive deployments can use a JCEKs file to store secrets, which it sets up
To be readable only by the Hive user, listing it under
hadoop.credential.providers.
When it tries to create an S3A FS instance as another user, via a doAs{}
clause, the S3A FS getPassword() call fails on the subsequent
AccessDeniedException -even if the secret it is looking for is in the XML file
or, as in the case of encryption settings, or session key undefined.
I can you point the blame at hive for this -it's the one with a forbidden JCEKS
file on the provider path, but I think it is easiest to fix in S3AUtils than
in hive, and safer then changing Configuration.
ABFS is likely to see the same problem.
I propose an option to set the fallback policy.
I initially thought about always handling this:
Catching the exception, attempting to downgrade to Reading XML and if that
fails rethrowing the caught exception.
However, this will do the wrong thing if the option is completely undefined,
As is common with the encryption settings.
I don't want to simply default to log and continue here though, as it may be a
legitimate failure -such as when you really do want to read secrets from such a
source.
Issue: what fallback policies?
* fail: fail fast. today's policy; the default.
* ignore: log and continue
We could try and be clever in future. To get away with that, we would have to
declare which options were considered compulsory and re-throw the caught
Exception if no value was found in the XML file.
That can be a future enhancement -but it is why I want the policy to be an
enumeration, rather than a simple boolean.
Tests: should be straightforward; set hadoop.credential.providers to a
non-existent file and expected to be processed according to the settings.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
