[ https://issues.apache.org/activemq/browse/AMQ-2102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=50261#action_50261 ]
Gary Tully commented on AMQ-2102: --------------------------------- ying, could you possibly codify your test in Junit and attach it. The broker setup from this test may be useful: http://svn.apache.org/viewvc/activemq/trunk/activemq-core/src/test/java/org/apache/activemq/bugs/AMQ2102Test.java?view=markup then I can run your test case in tandem against my changes. thanks. > Master/slave out of sync with multiple consumers > ------------------------------------------------ > > Key: AMQ-2102 > URL: https://issues.apache.org/activemq/browse/AMQ-2102 > Project: ActiveMQ > Issue Type: Bug > Components: Broker > Affects Versions: 5.2.0 > Reporter: Dan James > Assignee: Gary Tully > Attachments: master.xml, MasterSlaveBug.java, MasterSlavePatch.patch, > slave.xml, slaveDispatchOnNotification.patch > > > I'm seeing exceptions like this in a simple master/slave setup: > ERROR Service - Async error occurred: > javax.jms.JMSException: Slave broker out of sync with master: Dispatched > message (ID:DUL1SJAMES-L2-1231-1233929569359-0:4:1:1:207) was not in the > pending list for MasterSlaveBug > javax.jms.JMSException: Slave broker out of sync with master: Dispatched > message (ID:DUL1SJAMES-L2-1231-1233929569359-0:4:1:1:207) was not in the > pending list for MasterSlaveBug > The problem only happens when there are multiple consumers listening to the > queue, and is more likely to occur as there are more consumers listening. > I've written a test program that demonstrates the problem. > I start the master and slave with an empty data directory and let them both > startup and settle. Then start the test program. The test program creates a > specified number of consumers, and then starts queuing 256 messages. The > consumers process the message by sending a reply. The producer counts the > replies. Both consumers and the producer see all the messages, but with > multiple consumers it is very likely that the error above will occur and > several of the messages will still be queued on the slave. > While debugging through the activemq code, I noticed that both the master and > the slave dispatch the message to a consumer's pending list independently. > In other words, it is possible that the master will add the message to > consumer A's pending list and the slave will add the message to consumer B's > pending list. Once the message has been processed by consumer A, the master > sends a message to the slaving which specifies consumer A so that the slave > can remove the message. The slave looks on its copy of consumer A's pending > list and cannot find the message. As a result, it throws this exception and > the message stays stuck on consumer B's pending list on the slave. > Master and slave configurations along with MasterSlaveBug.java are attached > to this issue. > Start master and slave brokers: > activemq xbean:master.xml > activemq xbean:slave.xml > Run with (only one consumer, the bug does not appear): > java -classpath .:activemq-all-5.2.0.jar MasterSlaveBug 1 > Run with (sixteen consumers, the bug does appear): > java -classpath .:activemq-all-5.2.0.jar MasterSlaveBug 16 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.