Jonathan Maron created SLIDER-888: ------------------------------------- Summary: intermittent errors when accessing key store password during localization of cert stores Key: SLIDER-888 URL: https://issues.apache.org/jira/browse/SLIDER-888 Project: Slider Issue Type: Bug Components: appmaster, security Reporter: Jonathan Maron Assignee: Jonathan Maron
Under certain conditions exceptions are generated during localization of certificate stores to containers: 2015-05-18 20:35:38,533 [RoleLaunchService-010] ERROR appmaster.RoleLaunchService - Exception thrown while trying to start ACCUMULO_MONITOR: java.util.NoSuchElementException java.util.NoSuchElementException at sun.misc.CompoundEnumeration.nextElement(CompoundEnumeration.java:59) at java.util.ServiceLoader$LazyIterator.hasNext(ServiceLoader.java:349) at java.util.ServiceLoader$1.hasNext(ServiceLoader.java:439) at org.apache.hadoop.security.alias.CredentialProviderFactory.getProviders(CredentialProviderFactory.java:57) at org.apache.hadoop.conf.Configuration.getPasswordFromCredentialProviders(Configuration.java:1946) at org.apache.hadoop.conf.Configuration.getPassword(Configuration.java:1926) at org.apache.slider.server.services.security.AbstractSecurityStoreGenerator.getStorePassword(AbstractSecurityStoreGenerator.java:64) at org.apache.slider.server.services.security.KeystoreGenerator.generate(KeystoreGenerator.java:43) at org.apache.slider.server.services.security.StoresGenerator.generateSecurityStores(StoresGenerator.java:48) at org.apache.slider.providers.agent.AgentProviderService.generateSecurityStores(AgentProviderService.java:561) at org.apache.slider.providers.agent.AgentProviderService.localizeContainerSecurityStores(AgentProviderService.java:545) at org.apache.slider.providers.agent.AgentProviderService.buildContainerLaunchContext(AgentProviderService.java:453) at org.apache.slider.server.appmaster.RoleLaunchService$RoleLauncher.run(RoleLaunchService.java:201) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) and 2015-05-18 20:28:42,393 [RoleLaunchService-008] ERROR appmaster.RoleLaunchService - Exception thrown while trying to start ACCUMULO_TSERVER: java.io.IOException: Configuration problem with provider path. java.io.IOException: Configuration problem with provider path. at org.apache.hadoop.conf.Configuration.getPasswordFromCredentialProviders(Configuration.java:1966) at org.apache.hadoop.conf.Configuration.getPassword(Configuration.java:1926) at org.apache.slider.server.services.security.AbstractSecurityStoreGenerator.getStorePassword(AbstractSecurityStoreGenerator.java:64) at org.apache.slider.server.services.security.KeystoreGenerator.generate(KeystoreGenerator.java:43) at org.apache.slider.server.services.security.StoresGenerator.generateSecurityStores(StoresGenerator.java:48) at org.apache.slider.providers.agent.AgentProviderService.generateSecurityStores(AgentProviderService.java:561) at org.apache.slider.providers.agent.AgentProviderService.localizeContainerSecurityStores(AgentProviderService.java:545) at org.apache.slider.providers.agent.AgentProviderService.buildContainerLaunchContext(AgentProviderService.java:453) at org.apache.slider.server.appmaster.RoleLaunchService$RoleLauncher.run(RoleLaunchService.java:201) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.io.IOException: No CredentialProviderFactory for jceks://hdfs/user/hrt_qa/accumulo-test_read_write_ssl.jceks in hadoop.security.credential.provider.path at org.apache.hadoop.security.alias.CredentialProviderFactory.getProviders(CredentialProviderFactory.java:66) at org.apache.hadoop.conf.Configuration.getPasswordFromCredentialProviders(Configuration.java:1946) ... 11 more -- This message was sent by Atlassian JIRA (v6.3.4#6332)