[ https://issues.apache.org/jira/browse/GEODE-9383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17363089#comment-17363089 ]
Geode Integration commented on GEODE-9383: ------------------------------------------ Seen in [distributed-test-openjdk8 #38|https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-main/jobs/distributed-test-openjdk8/builds/38] ... see [test results|http://files.apachegeode-ci.info/builds/apache-develop-main/1.15.0-build.0315/test-results/distributedTest/1623439337/] or download [artifacts|http://files.apachegeode-ci.info/builds/apache-develop-main/1.15.0-build.0315/test-artifacts/1623439337/distributedtestfiles-openjdk8-1.15.0-build.0315.tgz]. > CI Failure: ReconnectDUnitTest.testReconnectWithRequiredRoleRegained FAILED > --------------------------------------------------------------------------- > > Key: GEODE-9383 > URL: https://issues.apache.org/jira/browse/GEODE-9383 > Project: Geode > Issue Type: Bug > Components: tests > Affects Versions: 1.15.0 > Reporter: Mark Hanson > Priority: Major > > The CI's distributed-test-openjdk8 failed with the error below. > > {noformat} > 10:52:10org.apache.geode.cache30.ReconnectDUnitTest > > testReconnectWithRequiredRoleRegained FAILED > 10:52:10 java.lang.AssertionError: Thread did not terminate after 120000 > ms: Thread[run invoked on an instance of > org.apache.geode.test.dunit.internal.IdentifiableRunnable,5,Async Invocations] > 10:52:10 at org.junit.Assert.fail(Assert.java:89) > 10:52:10 at > org.apache.geode.test.dunit.ThreadUtils.join(ThreadUtils.java:147) > 10:52:10 at > org.apache.geode.test.dunit.ThreadUtils.join(ThreadUtils.java:110) > 10:52:10 at > org.apache.geode.cache30.ReconnectDUnitTest.testReconnectWithRequiredRoleRegained(ReconnectDUnitTest.java:905) > 10:52:10 > 10:52:10 java.lang.AssertionError: Suspicious strings were written to the > log during this run. > 10:52:10 Fix the strings or use IgnoredException.addIgnoredException to > ignore. > 10:52:10 > ----------------------------------------------------------------------- > 10:52:10 Found suspect string in 'dunit_suspect-vm3.log' at line 837 > 10:52:10 > 10:52:10 [fatal 2021/06/11 17:50:10.326 GMT <Geode Membership View > Creator> tid=253] Membership service failure: Exiting due to possible network > partition event due to loss of 1 cache processes: > [3c0ddb2b78f9(251)<v5>:41004] > 10:52:10 > org.apache.geode.distributed.internal.membership.api.MemberDisconnectedException: > Exiting due to possible network partition event due to loss of 1 cache > processes: [3c0ddb2b78f9(251)<v5>:41004] > 10:52:10 at > org.apache.geode.distributed.internal.membership.gms.GMSMembership$ManagerImpl.forceDisconnect(GMSMembership.java:1787) > 10:52:10 at > org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave.forceDisconnect(GMSJoinLeave.java:1122) > 10:52:10 at > org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave.access$1300(GMSJoinLeave.java:80) > 10:52:10 at > org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave$ViewCreator.prepareAndSendView(GMSJoinLeave.java:2588) > 10:52:10 at > org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave$ViewCreator.createAndSendView(GMSJoinLeave.java:2565) > 10:52:10 at > org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave$ViewCreator.run(GMSJoinLeave.java:2340) > 10:52:10 > 10:52:10 > ----------------------------------------------------------------------- > 10:52:10 Found suspect string in 'dunit_suspect-vm1.log' at line 692 > 10:52:10 > 10:52:10 [fatal 2021/06/11 17:50:09.327 GMT <unicast > receiver,3c0ddb2b78f9-23916> tid=392] Membership service failure: Member > isn't responding to heartbeat requests > 10:52:10 > org.apache.geode.distributed.internal.membership.api.MemberDisconnectedException: > Member isn't responding to heartbeat requests > 10:52:10 at > org.apache.geode.distributed.internal.membership.gms.GMSMembership$ManagerImpl.forceDisconnect(GMSMembership.java:1787) > 10:52:10 at > org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave.forceDisconnect(GMSJoinLeave.java:1122) > 10:52:10 at > org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave.processRemoveMemberMessage(GMSJoinLeave.java:725) > 10:52:10 at > org.apache.geode.distributed.internal.membership.gms.messenger.JGroupsMessenger$JGroupsReceiver.receive(JGroupsMessenger.java:1367) > 10:52:10 at > org.apache.geode.distributed.internal.membership.gms.messenger.JGroupsMessenger$JGroupsReceiver.receive(JGroupsMessenger.java:1303) > 10:52:10 at org.jgroups.JChannel.invokeCallback(JChannel.java:816) > 10:52:10 at org.jgroups.JChannel.up(JChannel.java:741) > 10:52:10 at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1030) > 10:52:10 at org.jgroups.protocols.FRAG2.up(FRAG2.java:165) > 10:52:10 at org.jgroups.protocols.FlowControl.up(FlowControl.java:390) > 10:52:10 at > org.jgroups.protocols.UNICAST3.deliverMessage(UNICAST3.java:1077) > 10:52:10 at > org.jgroups.protocols.UNICAST3.handleDataReceived(UNICAST3.java:792) > 10:52:10 at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:433) > 10:52:10 at > org.apache.geode.distributed.internal.membership.gms.messenger.StatRecorder.up(StatRecorder.java:72) > 10:52:10 at > org.apache.geode.distributed.internal.membership.gms.messenger.AddressManager.up(AddressManager.java:70) > 10:52:10 at org.jgroups.protocols.TP.passMessageUp(TP.java:1658) > 10:52:10 at > org.jgroups.protocols.TP$SingleMessageHandler.run(TP.java:1876) > 10:52:10 at > org.jgroups.util.DirectExecutor.execute(DirectExecutor.java:10) > 10:52:10 at org.jgroups.protocols.TP.handleSingleMessage(TP.java:1789) > 10:52:10 at org.jgroups.protocols.TP.receive(TP.java:1714) > 10:52:10 at > org.apache.geode.distributed.internal.membership.gms.messenger.Transport.receive(Transport.java:160) > 10:52:10 at org.jgroups.protocols.UDP$PacketReceiver.run(UDP.java:701) > 10:52:10 at java.lang.Thread.run(Thread.java:748) > 10:52:10 > 10:52:10 > ----------------------------------------------------------------------- > 10:52:10 Found suspect string in 'dunit_suspect-vm2.log' at line 555 > 10:52:10 > 10:52:10 [fatal 2021/06/11 17:50:09.376 GMT <unicast > receiver,3c0ddb2b78f9-55561> tid=51] Membership service failure: Membership > coordinator 3c0ddb2b78f9(291:locator)<ec><v0>:41001 has declared that a > network partition has occurred > 10:52:10 > org.apache.geode.distributed.internal.membership.api.MemberDisconnectedException: > Membership coordinator 3c0ddb2b78f9(291:locator)<ec><v0>:41001 has declared > that a network partition has occurred > 10:52:10 at > org.apache.geode.distributed.internal.membership.gms.GMSMembership$ManagerImpl.forceDisconnect(GMSMembership.java:1787) > 10:52:10 at > org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave.forceDisconnect(GMSJoinLeave.java:1122) > 10:52:10 at > org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave.processNetworkPartitionMessage(GMSJoinLeave.java:1466) > 10:52:10 at > org.apache.geode.distributed.internal.membership.gms.messenger.JGroupsMessenger$JGroupsReceiver.receive(JGroupsMessenger.java:1367) > 10:52:10 at > org.apache.geode.distributed.internal.membership.gms.messenger.JGroupsMessenger$JGroupsReceiver.receive(JGroupsMessenger.java:1303) > 10:52:10 at org.jgroups.JChannel.invokeCallback(JChannel.java:816) > 10:52:10 at org.jgroups.JChannel.up(JChannel.java:741) > 10:52:10 at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1030) > 10:52:10 at org.jgroups.protocols.FRAG2.up(FRAG2.java:165) > 10:52:10 at org.jgroups.protocols.FlowControl.up(FlowControl.java:390) > 10:52:10 at > org.jgroups.protocols.UNICAST3.deliverMessage(UNICAST3.java:1077) > 10:52:10 at > org.jgroups.protocols.UNICAST3.handleDataReceived(UNICAST3.java:792) > 10:52:10 at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:433) > 10:52:10 at > org.apache.geode.distributed.internal.membership.gms.messenger.StatRecorder.up(StatRecorder.java:72) > 10:52:10 at > org.apache.geode.distributed.internal.membership.gms.messenger.AddressManager.up(AddressManager.java:70) > 10:52:10 at org.jgroups.protocols.TP.passMessageUp(TP.java:1658) > 10:52:10 at > org.jgroups.protocols.TP$SingleMessageHandler.run(TP.java:1876) > 10:52:10 at > org.jgroups.util.DirectExecutor.execute(DirectExecutor.java:10) > 10:52:10 at org.jgroups.protocols.TP.handleSingleMessage(TP.java:1789) > 10:52:10 at org.jgroups.protocols.TP.receive(TP.java:1714) > 10:52:10 at > org.apache.geode.distributed.internal.membership.gms.messenger.Transport.receive(Transport.java:160) > 10:52:10 at org.jgroups.protocols.UDP$PacketReceiver.run(UDP.java:701) > 10:52:10 at java.lang.Thread.run(Thread.java:748) > 10:52:10 at org.junit.Assert.fail(Assert.java:89) > 10:52:10 at > org.apache.geode.test.dunit.internal.DUnitLauncher.closeAndCheckForSuspects(DUnitLauncher.java:409) > 10:52:10 at > org.apache.geode.test.dunit.internal.DUnitLauncher.closeAndCheckForSuspects(DUnitLauncher.java:425) > 10:52:10 at > org.apache.geode.test.dunit.internal.JUnit4DistributedTestCase.cleanupAllVms(JUnit4DistributedTestCase.java:550) > 10:52:10 at > org.apache.geode.test.dunit.internal.JUnit4DistributedTestCase.doTearDownDistributedTestCase(JUnit4DistributedTestCase.java:497) > 10:52:10 at > org.apache.geode.test.dunit.internal.JUnit4DistributedTestCase.tearDownDistributedTestCase(JUnit4DistributedTestCase.java:480) > 10:52:10 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > 10:52:10 at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > 10:52:10 at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > 10:52:10 at java.lang.reflect.Method.invoke(Method.java:498) > 10:52:10 at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) > 10:52:10 at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > 10:52:10 at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) > 10:52:10 at > org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) > 10:52:10 at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) > 10:52:10 at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) > 10:52:10 at > org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) > 10:52:10 at > org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) > 10:52:10 at > org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) > 10:52:10 at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) > 10:52:10 at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) > 10:52:10 at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) > 10:52:10 at > org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) > 10:52:10 at > org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) > 10:52:10 at > org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) > 10:52:10 at > org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) > 10:52:10 at > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) > 10:52:10 at > org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) > 10:52:10 at org.junit.runners.ParentRunner.run(ParentRunner.java:413) > 10:52:10 at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:110) > 10:52:10 at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58) > 10:52:10 at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38) > 10:52:10 at > org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:62) > 10:52:10 at > org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51) > 10:52:10 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > 10:52:10 at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > 10:52:10 at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > 10:52:10 at java.lang.reflect.Method.invoke(Method.java:498) > 10:52:10 at > org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36) > 10:52:10 at > org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24) > 10:52:10 at > org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33) > 10:52:10 at > org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94) > 10:52:10 at com.sun.proxy.$Proxy2.processTestClass(Unknown Source) > 10:52:10 at > org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:119) > 10:52:10 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > 10:52:10 at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > 10:52:10 at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > 10:52:10 at java.lang.reflect.Method.invoke(Method.java:498) > 10:52:10 at > org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36) > 10:52:10 at > org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24) > 10:52:10 at > org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:182) > 10:52:10 at > org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:164) > 10:52:10 at > org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:414) > 10:52:10 at > org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64) > 10:52:10 at > org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48) > 10:52:10 at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > 10:52:10 at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > 10:52:10 at > org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56) > 10:52:10 at java.lang.Thread.run(Thread.java:748) {noformat} > > {noformat} > =-=-=-=-=-=-=-=-=-=-=-=-=-=-= Test Results URI > =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= > http://files.apachegeode-ci.info/builds/apache-develop-main/1.15.0-build.0315/test-results/distributedTest/1623439337/ > =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= > Test report artifacts from this job are available at: > http://files.apachegeode-ci.info/builds/apache-develop-main/1.15.0-build.0315/test-artifacts/1623439337/distributedtestfiles-openjdk8-1.15.0-build.0315.tgz > {noformat} > > > -- This message was sent by Atlassian Jira (v8.3.4#803005)