[
https://issues.apache.org/jira/browse/GEODE-538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15527484#comment-15527484
]
Kenneth Howe commented on GEODE-538:
------------------------------------
With the async*.getResult(MAX_WAIT) commented out, then checkData() is called
while persistent data recovery is still in progress following the child region
creation on all the members. The test now (usually) fails because a
PartitionedRegion.get() call targets a bucket that hasn't yet been recovered.
PR.getNodeForBucketReadOrLoad() ends up creating a new bucket, and the get()
ends up returning null.
Examining full logs there will also be one or more members that throw
ConflictingPersistentDataException when the recovery thread(s) tries to recover
the bucket that was erroneously created.
put() operations on PRs with incomplete persistent recovery can also cause this
failure.
> PersistentColocatedPartitionedRegionDUnitTest.testRebalanceWithOfflineChildRegion
> failed if do not wait on getResult
> --------------------------------------------------------------------------------------------------------------------
>
> Key: GEODE-538
> URL: https://issues.apache.org/jira/browse/GEODE-538
> Project: Geode
> Issue Type: Bug
> Components: persistence
> Reporter: Jinmei Liao
> Assignee: Kenneth Howe
> Attachments: logs.zip
>
>
> While attempting to work on GEODE-506, I commented out line s1538-1540 like
> the following
> async1 = vm1.invokeAsync(createChildPR);
> async0 = vm0.invokeAsync(createChildPR);
> AsyncInvocation async2 = vm2.invokeAsync(createChildPR);
> //async0.getResult(MAX_WAIT); -- line 1538
> //async1.getResult(MAX_WAIT); -- line 1539
> //async2.getResult(MAX_WAIT); -- line 1540
> This will reproduce the error similar to the one reported in GEODE506, but it
> failed on checking the data on parent region instead of the child region. Log
> files attached.
> dunit.RMIException: While invoking
> com.gemstone.gemfire.internal.cache.partitioned.PersistentPartitionedRegionTestBase$5.run
> in VM 0 running on Host localhost with 4 VMs
> at dunit.VM.invoke(VM.java:368)
> at dunit.VM.invoke(VM.java:312)
> at dunit.VM.invoke(VM.java:266)
> at
> com.gemstone.gemfire.internal.cache.partitioned.PersistentPartitionedRegionTestBase.checkData(PersistentPartitionedRegionTestBase.java:182)
> at
> com.gemstone.gemfire.internal.cache.partitioned.PersistentPartitionedRegionTestBase.checkData(PersistentPartitionedRegionTestBase.java:165)
> at
> com.gemstone.gemfire.internal.cache.partitioned.PersistentColocatedPartitionedRegionDUnitTest.rebalanceWithOfflineChildRegion(PersistentColocatedPartitionedRegionDUnitTest.java:1543)
> at
> com.gemstone.gemfire.internal.cache.partitioned.PersistentColocatedPartitionedRegionDUnitTest.testRebalanceWithOfflineChildRegion(PersistentColocatedPartitionedRegionDUnitTest.java:1223)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at junit.framework.TestCase.runTest(TestCase.java:176)
> at junit.framework.TestCase.runBare(TestCase.java:141)
> at junit.framework.TestResult$1.protect(TestResult.java:122)
> at junit.framework.TestResult.runProtected(TestResult.java:142)
> at junit.framework.TestResult.run(TestResult.java:125)
> at junit.framework.TestCase.run(TestCase.java:129)
> at junit.framework.TestSuite.runTest(TestSuite.java:252)
> at junit.framework.TestSuite.run(TestSuite.java:247)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:86)
> at
> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86)
> at
> org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)
> Caused by: junit.framework.AssertionFailedError: For key 1 expected:<a> but
> was:<null>
> at junit.framework.Assert.fail(Assert.java:57)
> at junit.framework.Assert.failNotEquals(Assert.java:329)
> at junit.framework.Assert.assertEquals(Assert.java:78)
> at junit.framework.TestCase.assertEquals(TestCase.java:244)
> at
> com.gemstone.gemfire.internal.cache.partitioned.PersistentPartitionedRegionTestBase$5.run(PersistentPartitionedRegionTestBase.java:177)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at hydra.MethExecutor.executeObject(MethExecutor.java:267)
> at
> dunit.standalone.RemoteDUnitVM.executeMethodOnObject(RemoteDUnitVM.java:77)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:323)
> at sun.rmi.transport.Transport$1.run(Transport.java:200)
> at sun.rmi.transport.Transport$1.run(Transport.java:197)
> at java.security.AccessController.doPrivileged(Native Method)
> at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
> at
> sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$256(TCPTransport.java:683)
> at java.security.AccessController.doPrivileged(Native Method)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
