[ https://issues.apache.org/jira/browse/GEODE-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15321701#comment-15321701 ]
Dan Smith commented on GEODE-1422: ---------------------------------- I did a little digging into this failure. When I look at the state with a debugger, it looks like this extra event is sitting in AbstractGatewaySender.tmpQueuedEvents data structure. The size of that structure is added on to the size of the queue. It looks like the tmpQueuedEvents is only added to while the queue is stopped - once it is started, this map is drained. There is some synchronization between adding events a draining the queue. But I see this code in ParallelAsyncEventQueueImpl.start, which appears to check the tmpQueuedEvents size outside of the synchronization: {code} if (!tmpQueuedEvents.isEmpty()) { enqueueTempEvents(); } {code} > CI Failure: > ParallelGatewaySenderOperationsOffHeapDUnitTest.testParallelPropagationSenderStartAfterStop_Scenario2 > ----------------------------------------------------------------------------------------------------------------- > > Key: GEODE-1422 > URL: https://issues.apache.org/jira/browse/GEODE-1422 > Project: Geode > Issue Type: Bug > Components: wan > Reporter: Sai Boorlagadda > Labels: CI > > {noformat} > Error Message > com.gemstone.gemfire.test.dunit.RMIException: While invoking > com.gemstone.gemfire.internal.cache.wan.parallel.ParallelGatewaySenderOperationsDUnitTest$$Lambda$998/238210599.run > in VM 4 running on Host kuwait.gemstone.com with 8 VMs > Stacktrace > com.gemstone.gemfire.test.dunit.RMIException: While invoking > com.gemstone.gemfire.internal.cache.wan.parallel.ParallelGatewaySenderOperationsDUnitTest$$Lambda$998/238210599.run > in VM 4 running on Host kuwait.gemstone.com with 8 VMs > at com.gemstone.gemfire.test.dunit.VM.invoke(VM.java:389) > at com.gemstone.gemfire.test.dunit.VM.invoke(VM.java:355) > at com.gemstone.gemfire.test.dunit.VM.invoke(VM.java:293) > at > com.gemstone.gemfire.internal.cache.wan.parallel.ParallelGatewaySenderOperationsDUnitTest.testParallelPropagationSenderStartAfterStop_Scenario2(ParallelGatewaySenderOperationsDUnitTest.java:358) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:497) > at junit.framework.TestCase.runTest(TestCase.java:176) > at junit.framework.TestCase.runBare(TestCase.java:141) > at junit.framework.TestResult$1.protect(TestResult.java:122) > at junit.framework.TestResult.runProtected(TestResult.java:142) > at junit.framework.TestResult.run(TestResult.java:125) > at junit.framework.TestCase.run(TestCase.java:129) > at junit.framework.TestSuite.runTest(TestSuite.java:252) > at junit.framework.TestSuite.run(TestSuite.java:247) > at > org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:86) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.runTestClass(JUnitTestClassExecuter.java:112) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.execute(JUnitTestClassExecuter.java:56) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassProcessor.processTestClass(JUnitTestClassProcessor.java:66) > at > org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51) > at sun.reflect.GeneratedMethodAccessor197.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:497) > at > org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35) > at > org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24) > at > org.gradle.messaging.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32) > at > org.gradle.messaging.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93) > at com.sun.proxy.$Proxy2.processTestClass(Unknown Source) > at > org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:109) > at sun.reflect.GeneratedMethodAccessor196.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:497) > at > org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35) > at > org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24) > at > org.gradle.messaging.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:360) > at > org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:54) > at > org.gradle.internal.concurrent.StoppableExecutorImpl$1.run(StoppableExecutorImpl.java:40) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: com.jayway.awaitility.core.ConditionTimeoutException: Condition > defined as a lambda expression in > com.gemstone.gemfire.internal.cache.wan.WANTestBase that uses int, > intcom.gemstone.gemfire.internal.cache.RegionQueue Expected queue entries: 0 > but actual entries: 1 expected:<0> but was:<1> within 120 seconds. > at > com.jayway.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:122) > at > com.jayway.awaitility.core.AssertionCondition.await(AssertionCondition.java:117) > at > com.jayway.awaitility.core.AssertionCondition.await(AssertionCondition.java:32) > at > com.jayway.awaitility.core.ConditionFactory.until(ConditionFactory.java:764) > at > com.jayway.awaitility.core.ConditionFactory.until(ConditionFactory.java:603) > at > com.gemstone.gemfire.internal.cache.wan.WANTestBase.validateQueueContents(WANTestBase.java:3089) > at > com.gemstone.gemfire.internal.cache.wan.parallel.ParallelGatewaySenderOperationsDUnitTest.lambda$testParallelPropagationSenderStartAfterStop_Scenario2$84651174$4(ParallelGatewaySenderOperationsDUnitTest.java:358) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:497) > at hydra.MethExecutor.executeObject(MethExecutor.java:268) > at > com.gemstone.gemfire.test.dunit.standalone.RemoteDUnitVM.executeMethodOnObject(RemoteDUnitVM.java:82) > at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:497) > at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:323) > at sun.rmi.transport.Transport$1.run(Transport.java:200) > at sun.rmi.transport.Transport$1.run(Transport.java:197) > at java.security.AccessController.doPrivileged(Native Method) > at sun.rmi.transport.Transport.serviceCall(Transport.java:196) > at > sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568) > at > sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826) > at > sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$95(TCPTransport.java:683) > at java.security.AccessController.doPrivileged(Native Method) > at > sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682) > ... 3 more > Standard Output > Previously run tests: [WANManagementDUnitTest, ClusterConfigurationDUnitTest, > TestRemoteClusterDUnitTest, CacheXml70GatewayDUnitTest, > CacheXml80GatewayDUnitTest, WanAutoDiscoveryDUnitTest, WANSSLDUnitTest, > WanValidationsDUnitTest, NewWANConcurrencyCheckForDestroyDUnitTest, > WANLocatorServerDUnitTest, ShutdownAllPersistentGatewaySenderDUnitTest, > PDXNewWanDUnitTest, CommonParallelGatewaySenderOffHeapDUnitTest, > NewWanAuthenticationDUnitTest, SenderWithTransportFilterDUnitTest, > ReplicatedRegion_ParallelWANPropogationDUnitTest, > ReplicatedRegion_ParallelWANPersistenceDUnitTest, > CommonParallelGatewaySenderDUnitTest, > SerialWANPropogation_PartitionedRegionOffHeapDUnitTest, > ParallelGatewaySenderOperationsOffHeapDUnitTest] > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)