[ https://issues.apache.org/jira/browse/HADOOP-11934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14532989#comment-14532989 ]
Mike Yoder commented on HADOOP-11934: ------------------------------------- Yeah, if I remove the credential provider from the LdapGroupsMapping everything is fine. A local keystore provider that's basically identical to the JavaKeyStoreProvider but only looks on the local file system fits my use case just fine. Might there be a way, inside JavaKeyStoreProvider, to look at the URI before calling path.getFilesystem() and do something different if it's not in HDFS? > Use of JavaKeyStoreProvider in LdapGroupsMapping causes infinite loop > --------------------------------------------------------------------- > > Key: HADOOP-11934 > URL: https://issues.apache.org/jira/browse/HADOOP-11934 > Project: Hadoop Common > Issue Type: Bug > Components: security > Affects Versions: 2.6.0 > Reporter: Mike Yoder > Assignee: Larry McCay > > I was attempting to use the LdapGroupsMapping code and the > JavaKeyStoreProvider at the same time, and hit a really interesting, yet > fatal, issue. The code goes into what ought to have been an infinite loop, > were it not for it overflowing the stack and Java ending the loop. Here is a > snippet of the stack; my annotations are at the bottom. > {noformat} > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370) > at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296) > at > org.apache.hadoop.security.alias.JavaKeyStoreProvider.<init>(JavaKeyStoreProvider.java:88) > at > org.apache.hadoop.security.alias.JavaKeyStoreProvider.<init>(JavaKeyStoreProvider.java:65) > at > org.apache.hadoop.security.alias.JavaKeyStoreProvider$Factory.createProvider(JavaKeyStoreProvider.java:291) > at > org.apache.hadoop.security.alias.CredentialProviderFactory.getProviders(CredentialProviderFactory.java:58) > at > org.apache.hadoop.conf.Configuration.getPasswordFromCredentialProviders(Configuration.java:1863) > at > org.apache.hadoop.conf.Configuration.getPassword(Configuration.java:1843) > at > org.apache.hadoop.security.LdapGroupsMapping.getPassword(LdapGroupsMapping.java:386) > at > org.apache.hadoop.security.LdapGroupsMapping.setConf(LdapGroupsMapping.java:349) > at > org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73) > at > org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133) > at org.apache.hadoop.security.Groups.<init>(Groups.java:70) > at org.apache.hadoop.security.Groups.<init>(Groups.java:66) > at > org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:280) > at > org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:283) > at > org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:260) > at > org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:804) > at > org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:774) > at > org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:647) > at > org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:2753) > at > org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:2745) > at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2611) > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370) > at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296) > at > org.apache.hadoop.security.alias.JavaKeyStoreProvider.<init>(JavaKeyStoreProvider.java:88) > at > org.apache.hadoop.security.alias.JavaKeyStoreProvider.<init>(JavaKeyStoreProvider.java:65) > at > org.apache.hadoop.security.alias.JavaKeyStoreProvider$Factory.createProvider(JavaKeyStoreProvider.java:291) > at > org.apache.hadoop.security.alias.CredentialProviderFactory.getProviders(CredentialProviderFactory.java:58) > at > org.apache.hadoop.conf.Configuration.getPasswordFromCredentialProviders(Configuration.java:1863) > at > org.apache.hadoop.conf.Configuration.getPassword(Configuration.java:1843) > at > org.apache.hadoop.security.LdapGroupsMapping.getPassword(LdapGroupsMapping.java:386) > at > org.apache.hadoop.security.LdapGroupsMapping.setConf(LdapGroupsMapping.java:349) > at > org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73) > at > org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133) > at org.apache.hadoop.security.Groups.<init>(Groups.java:70) > at org.apache.hadoop.security.Groups.<init>(Groups.java:66) > at > org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:280) > at > org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:283) > at > org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:260) > at > org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:804) > at > org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:774) > at > org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:647) > at > org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:2753) > at > org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:2745) > at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2611) > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370) > at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296){noformat} > Here's my annotation, going from bottom to top. > * Somehow we enter Path.getFileSystem() > * This goes to FileSystem cache stuff, and then it wants the current user > * So we get to UserGroupInformation.getCurrentUser(), which as you can > imagine gets to > * getUserToGroupsMappingService and thence to LdapGroupsMapping.setConf(). > * That code gets the needed passwords, and we're using the > CredentialProvider, so unsurprisingly we get to > * getPasswordFromCredentialProviders() - which chooses the > JavaKeyStoreProvider like I told it to. > * The JavaKeyStoreProvider, in its constructor, does "fs = > path.getFileSystem(conf);" > * And guess what, we're back in Path.getFileSystem, where we started at the > beginning. > Please let me know if I've somehow configured something incorrectly, but if I > have I can't figure out what it is... -- This message was sent by Atlassian JIRA (v6.3.4#6332)