[ https://issues.apache.org/jira/browse/GEODE-9528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17404068#comment-17404068 ]
Barrett Oglesby commented on GEODE-9528: ---------------------------------------- This is a test issue. The test currently does: 1. force disconnect 2. assert the MembershipListener for the region is removed The finally block of GMSMembership.ManagerImpl.forceDisconnect spins off the DisconnectThread to do the actual disconnect. The MembershipListener is removed by that thread. {noformat} } finally { new LoggingThread("DisconnectThread", false, () -> { lifecycleListener.forcedDisconnect(); uncleanShutdown(reason, shutdownCause); }).start(); } {noformat} If there is a delay in the DisconnectThread processing, the test could fail. Here is the normal flow: 1. The Test worker thread invokes forceDisconnectMember 2. The DisconnectThread removes the MembershipListener 3. The Test worker thread asserts the listener is removed Here is logging that shows this behavior: {noformat} [warn 2021/08/24 13:38:47.445 PDT server <Test worker> tid=0xb] XXX DistributionAdvisorIntegrationTest.verifyMembershipListenerIsRemovedAfterForceDisconnect before forceDisconnectMember [warn 2021/08/24 13:38:47.452 PDT server <DisconnectThread> tid=0x32] XXX ManagerImpl.forceDisconnect about to uncleanShutdown [warn 2021/08/24 13:38:47.472 PDT server <ReconnectThread> tid=0x32] XXX DistributionAdvisor.close removeMembershipListener advisee=verifyMembershipListenerIsRemovedAfterForceDisconnect [warn 2021/08/24 13:38:47.566 PDT server <Test worker> tid=0xb] XXX DistributionAdvisorIntegrationTest.verifyMembershipListenerIsRemovedAfterForceDisconnect after forceDisconnectMember [warn 2021/08/24 13:38:47.567 PDT server <Test worker> tid=0xb] XXX DistributionAdvisorIntegrationTest.verifyMembershipListenerIsRemovedAfterForceDisconnect assert {noformat} If the DisconnectThread has any kind of delay in running, the order changes, and the test fails: {noformat} [warn 2021/08/24 14:05:29.270 PDT server <Test worker> tid=0xb] XXX DistributionAdvisorIntegrationTest.verifyMembershipListenerIsRemovedAfterForceDisconnect before forceDisconnectMember [warn 2021/08/24 14:05:29.382 PDT server <DisconnectThread> tid=0x32] XXX ManagerImpl.forceDisconnect about to uncleanShutdown [warn 2021/08/24 14:05:29.392 PDT server <Test worker> tid=0xb] XXX DistributionAdvisorIntegrationTest.verifyMembershipListenerIsRemovedAfterForceDisconnect after forceDisconnectMember [warn 2021/08/24 14:05:29.393 PDT server <Test worker> tid=0xb] XXX DistributionAdvisorIntegrationTest.verifyMembershipListenerIsRemovedAfterForceDisconnect assert [warn 2021/08/24 14:05:29.403 PDT server <DisconnectThread> tid=0x32] XXX DistributionAdvisor.close removeMembershipListener advisee=verifyMembershipListenerIsRemovedAfterForceDisconnect org.junit.ComparisonFailure: Expecting value to be false but was true expected:<[fals]e> but was:<[tru]e> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at org.apache.geode.distributed.internal.DistributionAdvisorIntegrationTest.verifyMembershipListenerIsRemovedAfterForceDisconnect(DistributionAdvisorIntegrationTest.java:62) {noformat} The fix is to add await().untilAsserted like: {noformat} await().untilAsserted( () -> assertThat(manager.getMembershipListeners().contains(listener)).isFalse()); {noformat} With the introduced delay, the test failed 80/100 times. With the await change, the test passed 100/100 times. > CI Failure: DistributionAdvisorIntegrationTest > > verifyMembershipListenerIsRemovedAfterForceDisconnect > ------------------------------------------------------------------------------------------------------ > > Key: GEODE-9528 > URL: https://issues.apache.org/jira/browse/GEODE-9528 > Project: Geode > Issue Type: Bug > Components: membership > Affects Versions: 1.12.5, 1.13.5, 1.14.0 > Reporter: Owen Nichols > Assignee: Barrett Oglesby > Priority: Major > > {noformat} > org.apache.geode.distributed.internal.DistributionAdvisorIntegrationTest > > verifyMembershipListenerIsRemovedAfterForceDisconnect FAILED > org.junit.ComparisonFailure: expected:<[fals]e> but was:<[tru]e> > at > jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) > at > jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > at > jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at > org.apache.geode.distributed.internal.DistributionAdvisorIntegrationTest.verifyMembershipListenerIsRemovedAfterForceDisconnect(DistributionAdvisorIntegrationTest.java:57) > {noformat} -- This message was sent by Atlassian Jira (v8.3.4#803005)