[ https://issues.apache.org/jira/browse/HADOOP-12611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15042024#comment-15042024 ]
Robert Kanter commented on HADOOP-12611: ---------------------------------------- {quote} it can inspect the initial state of secret (currentSecret\[1] == null) right after the three ZKSignerSecretProvider are initialized.{quote} Ok, I think the problem here is that HADOOP-12181 changed the rollover frequency to only 2 seconds (it was originally 15). So, it's very likely that one or more of the {{ZKSignerSecretProviders}} has already rolled by the time we check that {{currentSecret\[1] == null}}, so it's no longer {{null}}. With a rollover time of 15 seconds, this was much less likely to occur, though I suppose it was still possible. Trying to keep the timing/randomness that we'd like to test but still making the test not flakey, I think the best approach would be to do something similar to the 001 patch where you'd have a subclass of {{ZKSignerSecretProvider}} that keeps track of {{secrets}}, let it run for two iterations (which you can check based on the size of the list keeping the {{secrets}} so no timing is involved), and then verify that each iteration of {{secrets}} is correct. For example: # {{\[s1]\[null]}} # {{\[s2]\[s1]}} # {{\[s3]\[s2]}} And the test itself doesn't have any timing components because you'd be waiting for the list of {{secrets}} from each {{ZKSignerSecretProvider}} to contain 3 snapshots. You can then just check that all sets of snapshots agree. Does that sound good? > TestZKSignerSecretProvider#testMultipleInit occasionally fail > ------------------------------------------------------------- > > Key: HADOOP-12611 > URL: https://issues.apache.org/jira/browse/HADOOP-12611 > Project: Hadoop Common > Issue Type: Bug > Reporter: Wei-Chiu Chuang > Assignee: Wei-Chiu Chuang > Attachments: HADOOP-12611.001.patch, HADOOP-12611.002.patch > > > https://builds.apache.org/job/Hadoop-Common-trunk/2053/testReport/junit/org.apache.hadoop.security.authentication.util/TestZKSignerSecretProvider/testMultipleInit/ > Error Message > expected null, but was:<[B@142bad79> > Stacktrace > java.lang.AssertionError: expected null, but was:<[B@142bad79> > at org.junit.Assert.fail(Assert.java:88) > at org.junit.Assert.failNotNull(Assert.java:664) > at org.junit.Assert.assertNull(Assert.java:646) > at org.junit.Assert.assertNull(Assert.java:656) > at > org.apache.hadoop.security.authentication.util.TestZKSignerSecretProvider.testMultipleInit(TestZKSignerSecretProvider.java:149) > I think the failure was introduced after HADOOP-12181 > This is likely where the root cause is: > 2015-11-29 00:24:33,325 ERROR ZKSignerSecretProvider - An unexpected > exception occurred while pulling data fromZooKeeper > java.lang.IllegalStateException: instance must be started before calling this > method > at > com.google.common.base.Preconditions.checkState(Preconditions.java:145) > at > org.apache.curator.framework.imps.CuratorFrameworkImpl.getData(CuratorFrameworkImpl.java:363) > at > org.apache.hadoop.security.authentication.util.ZKSignerSecretProvider.pullFromZK(ZKSignerSecretProvider.java:341) > at > org.apache.hadoop.security.authentication.util.ZKSignerSecretProvider.rollSecret(ZKSignerSecretProvider.java:264) > at > org.apache.hadoop.security.authentication.util.ZKSignerSecretProvider$$EnhancerByMockitoWithCGLIB$$575f06d8.CGLIB$rollSecret$2(<generated>) > at > org.apache.hadoop.security.authentication.util.ZKSignerSecretProvider$$EnhancerByMockitoWithCGLIB$$575f06d8$$FastClassByMockitoWithCGLIB$$6f94a716.invoke(<generated>) > at org.mockito.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:216) > at > org.mockito.internal.creation.AbstractMockitoMethodProxy.invokeSuper(AbstractMockitoMethodProxy.java:10) > at > org.mockito.internal.invocation.realmethod.CGLIBProxyRealMethod.invoke(CGLIBProxyRealMethod.java:22) > at > org.mockito.internal.invocation.realmethod.FilteredCGLIBProxyRealMethod.invoke(FilteredCGLIBProxyRealMethod.java:27) > at > org.mockito.internal.invocation.Invocation.callRealMethod(Invocation.java:211) > at > org.mockito.internal.stubbing.answers.CallsRealMethods.answer(CallsRealMethods.java:36) > at org.mockito.internal.MockHandler.handle(MockHandler.java:99) > at > org.mockito.internal.creation.MethodInterceptorFilter.intercept(MethodInterceptorFilter.java:47) > at > org.apache.hadoop.security.authentication.util.ZKSignerSecretProvider$$EnhancerByMockitoWithCGLIB$$575f06d8.rollSecret(<generated>) > at > org.apache.hadoop.security.authentication.util.RolloverSignerSecretProvider$1.run(RolloverSignerSecretProvider.java:97) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) -- This message was sent by Atlassian JIRA (v6.3.4#6332)