Fang-Yu Rao created IMPALA-14699:
------------------------------------
Summary: Impala would encounter the error of "Recursive load of
credential provider" when a remote jceks path is put in the core-site
Key: IMPALA-14699
URL: https://issues.apache.org/jira/browse/IMPALA-14699
Project: IMPALA
Issue Type: Bug
Reporter: Fang-Yu Rao
Impala would encounter the error of "Recursive load of credential provider"
when a remote jceks path is put in the core-site.
The following is part of the stack trace we'd get if we put the following
key-value pair of ("hadoop.security.credential.provider.path",
"jceks://[email protected]:8020/secret.jceks")
in core-site.xml.
{code}
26/01/24 19:22:53 WARN security.LdapGroupsMapping: Exception while trying to
get password for alias hadoop.security.group.mapping.ldap.ssl.keystore.password:
java.io.IOException: Configuration problem with provider path.
at
org.apache.hadoop.conf.Configuration.getPasswordFromCredentialProviders(Configuration.java:2469)
at
org.apache.hadoop.conf.Configuration.getPassword(Configuration.java:2388)
at
org.apache.hadoop.security.LdapGroupsMapping.getPassword(LdapGroupsMapping.java:879)
at
org.apache.hadoop.security.LdapGroupsMapping.loadSslConf(LdapGroupsMapping.java:837)
at
org.apache.hadoop.security.LdapGroupsMapping.setConf(LdapGroupsMapping.java:733)
at
org.apache.hadoop.security.RuleBasedLdapGroupsMapping.setConf(RuleBasedLdapGroupsMapping.java:58)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:77)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:137)
at org.apache.hadoop.security.Groups.<init>(Groups.java:105)
at org.apache.hadoop.security.Groups.<init>(Groups.java:101)
at
org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:476)
at
org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:352)
at
org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:314)
at
org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:2008)
at
org.apache.hadoop.security.UserGroupInformation.createLoginUser(UserGroupInformation.java:743)
at
org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:693)
at
org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:604)
...
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3509)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:524)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:364)
at
org.apache.hadoop.security.alias.KeyStoreProvider.initFileSystem(KeyStoreProvider.java:84)
at
org.apache.hadoop.security.alias.AbstractJavaKeyStoreProvider.<init>(AbstractJavaKeyStoreProvider.java:85)
at
org.apache.hadoop.security.alias.KeyStoreProvider.<init>(KeyStoreProvider.java:49)
at
org.apache.hadoop.security.alias.JavaKeyStoreProvider.<init>(JavaKeyStoreProvider.java:42)
at
org.apache.hadoop.security.alias.JavaKeyStoreProvider.<init>(JavaKeyStoreProvider.java:35)
at
org.apache.hadoop.security.alias.JavaKeyStoreProvider$Factory.createProvider(JavaKeyStoreProvider.java:68)
at
org.apache.hadoop.security.alias.CredentialProviderFactory.getProviders(CredentialProviderFactory.java:91)
at
org.apache.hadoop.conf.Configuration.getPasswordFromCredentialProviders(Configuration.java:2450)
at
org.apache.hadoop.security.LdapGroupsMapping.getPasswordFromCredentialProviders(LdapGroupsMapping.java:858)
at
org.apache.hadoop.security.LdapGroupsMapping.loadSslConf(LdapGroupsMapping.java:846)
at
org.apache.hadoop.security.LdapGroupsMapping.setConf(LdapGroupsMapping.java:733)
at
org.apache.hadoop.security.RuleBasedLdapGroupsMapping.setConf(RuleBasedLdapGroupsMapping.java:58)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:77)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:137)
at org.apache.hadoop.security.Groups.<init>(Groups.java:105)
at org.apache.hadoop.security.Groups.<init>(Groups.java:101)
at
org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:476)
at
org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:352)
at
org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:314)
at
org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:2008)
at
org.apache.hadoop.security.UserGroupInformation.createLoginUser(UserGroupInformation.java:743)
at
org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:693)
Caused by: org.apache.hadoop.fs.PathIOException:
`jceks://[email protected]:8020/secret.jceks':
Recursive load of credential provider; if loading a JCEKS file, this means that
the filesystem connector is trying to load the same file
{code}
Currently we instantiate Configuration instance at
https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/service/JniFrontend.java#L719.
We need to figure out where else to instantiate Configuration so that we won't
hit this issue described above.
{code}
// Caching this saves ~50ms per call to getHadoopConfig
private static final Configuration CONF = new Configuration();
private static final Groups GROUPS =
Groups.getUserToGroupsMappingService(CONF);
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
