[ https://issues.apache.org/jira/browse/HADOOP-14195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15965712#comment-15965712 ]
Steve Loughran commented on HADOOP-14195: ----------------------------------------- Seems serious. Should this go back to branch 2.7? > CredentialProviderFactory$getProviders is not thread-safe > --------------------------------------------------------- > > Key: HADOOP-14195 > URL: https://issues.apache.org/jira/browse/HADOOP-14195 > Project: Hadoop Common > Issue Type: Bug > Components: security > Affects Versions: 2.7.0 > Reporter: Vihang Karajgaonkar > Assignee: Vihang Karajgaonkar > Fix For: 2.9.0, 2.8.1, 3.0.0-alpha3 > > Attachments: HADOOP-14195.001.patch, HADOOP-14195.002.patch, > HADOOP-14195.003.patch, TestCredentialProvider.java > > > Multi-threaded access to CredentialProviderFactory is not thread-safe because > {{java.util.ServiceLoader}} is not thread-safe (as noted in its Java doc). > Thanks to [~jzhuge] I was able to reproduce this issue but creating a simple > multi-threaded application which executes the following code in parallel. > {code:java} > for (int i = 0; i < ITEMS; i++) { > futures.add(executor.submit(new Callable<Void>() { > @Override > public Void call() throws Exception { > boolean found = false; > for (CredentialProviderFactory factory : serviceLoader) { > CredentialProvider kp = factory.createProvider(uri, > conf); > if (kp != null) { > result.add(kp); > found = true; > break; > } > } > if (!found) { > throw new IOException(Thread.currentThread() + "No > CredentialProviderFactory for " + uri); > } else { > System.out.println(Thread.currentThread().getName() + " > found credentialProvider for " + path); > } > return null; > } > })); > } > {code} > I see the following exception trace when I execute the above code. > {code:java} > java.util.concurrent.ExecutionException: java.util.NoSuchElementException > at java.util.concurrent.FutureTask.report(FutureTask.java:122) > at java.util.concurrent.FutureTask.get(FutureTask.java:192) > at TestCredentialProvider.main(TestCredentialProvider.java:58) > Caused by: java.util.NoSuchElementException > at java.net.URLClassLoader$3.nextElement(URLClassLoader.java:615) > at java.net.URLClassLoader$3.nextElement(URLClassLoader.java:590) > at sun.misc.CompoundEnumeration.nextElement(CompoundEnumeration.java:61) > at > java.util.ServiceLoader$LazyIterator.hasNextService(ServiceLoader.java:357) > at java.util.ServiceLoader$LazyIterator.hasNext(ServiceLoader.java:393) > at java.util.ServiceLoader$1.hasNext(ServiceLoader.java:474) > at TestCredentialProvider$1.call(TestCredentialProvider.java:38) > at TestCredentialProvider$1.call(TestCredentialProvider.java:1) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > {code} > I also see a NPE sometimes > {code:java} > java.util.concurrent.ExecutionException: java.lang.NullPointerException > at java.util.concurrent.FutureTask.report(FutureTask.java:122) > at java.util.concurrent.FutureTask.get(FutureTask.java:192) > at TestCredentialProvider.main(TestCredentialProvider.java:58) > Caused by: java.lang.NullPointerException > at java.util.ServiceLoader.parse(ServiceLoader.java:304) > at java.util.ServiceLoader.access$200(ServiceLoader.java:185) > at > java.util.ServiceLoader$LazyIterator.hasNextService(ServiceLoader.java:357) > at java.util.ServiceLoader$LazyIterator.hasNext(ServiceLoader.java:393) > at java.util.ServiceLoader$1.hasNext(ServiceLoader.java:474) > at TestCredentialProvider$1.call(TestCredentialProvider.java:38) > at TestCredentialProvider$1.call(TestCredentialProvider.java:1) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org