Barry Oglesby created GEODE-3030:
------------------------------------
Summary: The possibleDuplicate boolean may not be set to true in
previously processed AEQ events
Key: GEODE-3030
URL: https://issues.apache.org/jira/browse/GEODE-3030
Project: Geode
Issue Type: Bug
Components: wan
Reporter: Barry Oglesby
When a secondary bucket becomes primary, it sets possibleDuplicate=true for
batchSize events in AbstractBucketRegionQueue.markEventsAsDuplicate:
{noformat}
java.lang.Exception: Stack trace
at java.lang.Thread.dumpStack(Thread.java:1329)
at
com.gemstone.gemfire.internal.cache.AbstractBucketRegionQueue.markEventsAsDuplicate(AbstractBucketRegionQueue.java:329)
at
com.gemstone.gemfire.internal.cache.BucketRegionQueue.beforeAcquiringPrimaryState(BucketRegionQueue.java:203)
at
com.gemstone.gemfire.internal.cache.BucketAdvisor.acquiredPrimaryLock(BucketAdvisor.java:1257)
at
com.gemstone.gemfire.internal.cache.BucketAdvisor.acquirePrimaryRecursivelyForColocated(BucketAdvisor.java:1397)
at
com.gemstone.gemfire.internal.cache.BucketAdvisor$VolunteeringDelegate.doVolunteerForPrimary(BucketAdvisor.java:2695)
at
com.gemstone.gemfire.internal.cache.BucketAdvisor$VolunteeringDelegate$1.run(BucketAdvisor.java:2575)
at
com.gemstone.gemfire.internal.cache.BucketAdvisor$VolunteeringDelegate$2.run(BucketAdvisor.java:2908)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at
com.gemstone.gemfire.distributed.internal.DistributionManager.runUntilShutdown(DistributionManager.java:692)
at
com.gemstone.gemfire.distributed.internal.DistributionManager$6$1.run(DistributionManager.java:1029)
at java.lang.Thread.run(Thread.java:745)
{noformat}
In my test case, the batch size is 1 so possibleDuplicate is set to true for
only 1 event in each bucket. It is not set for the remaining events in the
bucket.
The ParallelQueueRemovalMessage is sent asynchronously from remote members so
more than 1 batch of events could have been processed between message sends.
So, possibleDuplicate should be set to true for more than batchSize events
(possibly all of them).
Here is an example from my test.
Server 1 is primary for bucket 5 and is stopped. Server 2 takes over primary
for bucket 5.
{noformat}
Server 1
--------
Server 1 processed 3 events from bucket 5 right before it was stopped:
TestGatewayEventListener processed 51427367-8d36-4425-aa02-e44c54774543
TestGatewayEventListener processed a1af3501-9030-460d-86cc-fe5b88bd5b0a
TestGatewayEventListener processed c1db3ec2-4dad-4444-9ea7-11bd34e492ec
No ParallelQueueRemovalMessage was sent to the remote nodes before the member
was stopped.
Server 2
--------
Server 2 took over primary for bucket 5 and processed those same 3 events - one
with possibleDuplicate=true, the others with possibleDuplicate=false. In all
three cases a SQLIntegrityConstraintViolationException was thrown since the
event had already been processed by the previous primary server.
TestGatewayEventListener caught EXPECTED exception
eventKey=51427367-8d36-4425-aa02-e44c54774543; operation=CREATE;
possibleDuplicate=true;
exception=java.sql.SQLIntegrityConstraintViolationException: The statement was
aborted because it would have caused a duplicate key value in a unique or
primary key constraint or unique index identified by 'SQL170601145521130'
defined on 'TRADES'.
TestGatewayEventListener caught UNEXPECTED exception
eventKey=a1af3501-9030-460d-86cc-fe5b88bd5b0a; operation=CREATE;
possibleDuplicate=false;
exception=java.sql.SQLIntegrityConstraintViolationException: The statement was
aborted because it would have caused a duplicate key value in a unique or
primary key constraint or unique index identified by 'SQL170601145521130'
defined on 'TRADES'.
TestGatewayEventListener caught UNEXPECTED exception
eventKey=c1db3ec2-4dad-4444-9ea7-11bd34e492ec; operation=CREATE;
possibleDuplicate=false;
exception=java.sql.SQLIntegrityConstraintViolationException: The statement was
aborted because it would have caused a duplicate key value in a unique or
primary key constraint or unique index identified by 'SQL170601145521130'
defined on 'TRADES'.
AbstractBucketRegionQueue.markEventsAsDuplicate set possibleDuplicate=true for
51427367-8d36-4425-aa02-e44c54774543, but not for the other events:
AbstractBucketRegionQueue.markEventsAsDuplicate marking posDup
eventKey=51427367-8d36-4425-aa02-e44c54774543
AbstractBucketRegionQueue.markEventsAsDuplicate not marking posDup
eventKey=a1af3501-9030-460d-86cc-fe5b88bd5b0a
AbstractBucketRegionQueue.markEventsAsDuplicate not marking posDup
eventKey=c1db3ec2-4dad-4444-9ea7-11bd34e492ec
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
