Rob, Can I hijack this issue and bring back an issue I have filed in Jira and is assigned to you (AMQ-1916) and that is the issue of a new ack mode (I called it WINDOW_ACK) that is a mix between CLIENT_ACK and INDIV_ACK in that the acknowledgement is related to the message but unlike in CLIENT_ACK, does not ack all messages in a session, but rather all messages delivered before the acked message (requires ordering of messages and tracking them by the broker but I think the building blocks are there)
any idea if/when this will be implemented? Cheers, Bonny rajdavies wrote: > > Bonny - agreed - looks like that bit of code has never been > synchronized - will add a fix ... > > cheers, > > Rob > > Rob Davies > http://fusesource.com > http://rajdavies.blogspot.com/ > > > On 9 Feb 2009, at 23:14, bonnyr wrote: > >> >> Gary, >> >> Why would the collection be used in CLIENT_ACK mode then? >> >> In our application we've got to work in this mode since there's a >> processing >> chain >> that may take a while between delivery of the message and its ack, so >> chaging this >> mode is not possible for our use case. >> >> We're aware of the fact that CLIENT_ACK actually acks all the >> messages in a >> session and >> it's not really related to the message itself, but that's a story for >> another day and another >> version of AMQ... >> >> However, wouldn't static analysis of the code be sufficient in this >> case? >> The code segment >> above is the only thing that is not protected by synchronization, >> whilst >> every other use of >> this collection is. >> >> Cheers, >> >> Bonny >> >> >> Gary Tully wrote: >>> >>>> I'm not quite able to cause this to happen in a simple test case. >>>> Perhaps >>>> this is to do with some >>>> thread timing/scheduling issues in the simple test case vs our >>>> application. >>>> >>> Yea, that is fairly typical. >>> >>>> However, are you in a >>>> position to explain what the deliveredMessage collection is used >>>> for? Is >>>> it >>>> not used in all connection/ >>>> ack modes? >>>> >>> Sure, the deliveredMessage list is used to generate acks in the Auto >>> ack mode. In addition, >>> with the consumer optimizeAcknowledge option, there is more likely to >>> be some outstanding >>> delivered messages that require acks. On close, any outstanding acks >>> will require delivery so >>> you may run into contention with concurrent delivery and close >>> execution as you suggest. >>> >>> May I suggest using AUTO_ACK and additionally using >>> optimizeAcknowledge if needed. >>> >>>> Perhaps if I knew what the behaviour should be like, I'd be able >>>> to construct a test >>>> case that simulate our environment. >>>> >>> Hopefully :-) >>> >>> Gary. >>> >>> >>>> Cheers, >>>> >>>> Bonny >>>> >>>> >>>> Gary Tully wrote: >>>>> >>>>> Hi Bonny, >>>>> that looks like a bug indeed, should be easy to replicate in a >>>>> Junit >>>>> tests case I think. Could you raise a jira issue for this and if >>>>> you >>>>> have some tests code that demonstrates, please include it. >>>>> >>>>> for more info see: http://activemq.apache.org/contributing.html >>>>> >>>>> Thanks, >>>>> Gary. >>>>> >>>>> 2009/2/6 bonnyr <bon...@optusnet.com.au>: >>>>>> >>>>>> AMQ 5.1 (but problem exists in the sources of AMQ-5.2 as of today) >>>>>> >>>>>> My setup: >>>>>> * Broker is configured with a single queue, full with messages, >>>>>> on a >>>>>> host >>>>>> accessible via the network. >>>>>> * Application configured with a single consumer, connected to a >>>>>> single >>>>>> sesssion, running in its own thread. >>>>>> * ActiveMQ delivers lots of messages using one of the >>>>>> ActiveMQSessionTask >>>>>> threads. >>>>>> * Session configured as CLIENT_ACKNOLEDGE >>>>>> >>>>>> >>>>>> Since the queue is full of messages, delivery of these messages >>>>>> happen >>>>>> as >>>>>> fast as the network >>>>>> can deliver and the AMQ thread is invoking the onMessage with >>>>>> each new >>>>>> message. These messages >>>>>> are then processed by the consumer thread. The consumer thread >>>>>> then >>>>>> decides >>>>>> to close the connection >>>>>> and the following ensues: >>>>>> [code] >>>>>> java.util.ConcurrentModificationException >>>>>> at >>>>>> java.util.LinkedList >>>>>> $ListItr.checkForComodification(LinkedList.java:617) >>>>>> at java.util.LinkedList$ListItr.next(LinkedList.java:552) >>>>>> at >>>>>> org >>>>>> .apache >>>>>> .activemq >>>>>> .ActiveMQMessageConsumer.dispose(ActiveMQMessageConsumer.java:663) >>>>>> at >>>>>> org >>>>>> .apache >>>>>> .activemq >>>>>> .ActiveMQMessageConsumer.close(ActiveMQMessageConsumer.java:583) >>>>>> at >>>>>> com.xxx.app..AMsgQueueConsumer.doClose(AMsgQueueConsumer.java:351) >>>>>> at com.xxx.app.AMsgQueueConsumer.suspend >>>>>> >>>>>> ... snip ... >>>>>> >>>>>> [/code] >>>>>> >>>>>> This happens because AMQ is busy delivering messages and there >>>>>> is a >>>>>> collection [b]deliveredMessages[/b] that is >>>>>> not protected by synchronisation in exactly one place >>>>>> (everywhere else >>>>>> it >>>>>> is...) which is executing the >>>>>> following code (in the dispose method): >>>>>> [code] >>>>>> ... snip ... >>>>>> if (session.isClientAcknowledge()) { >>>>>> if (!this.info.isBrowser()) { >>>>>> // rollback duplicates that aren't acknowledged >>>>>> for (MessageDispatch old : deliveredMessages) { >>>>>> session.connection.rollbackDuplicate(this, >>>>>> old.getMessage()); >>>>>> } >>>>>> } >>>>>> } >>>>>> ... snip ... >>>>>> [/code] >>>>>> >>>>>> Since the code iterates over the collection, and the iterator >>>>>> checks >>>>>> for >>>>>> modifications to the underlying collection, and since the AMQ >>>>>> message >>>>>> delivery thread has managed to sneak in a couple more messages >>>>>> while the consumer thread attempted to close the connection, the >>>>>> problem >>>>>> occurs (it could be that >>>>>> the for syntax hides the explicit iterator calls, but...) >>>>>> >>>>>> >>>>>> Is this an ommission or is there a reason for not synchronising >>>>>> access >>>>>> to >>>>>> this collection? If it's not a bug >>>>>> then how should the consumer be disconnected? >>>>>> >>>>>> Cheers, >>>>>> >>>>>> Bonny >>>>>> -- >>>>>> View this message in context: >>>>>> http://www.nabble.com/ConcurrentModificationException-while-closing-consumer-tp21867250p21867250.html >>>>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com. >>>>>> >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> http://blog.garytully.com >>>>> >>>>> Open Source SOA >>>>> http://FUSESource.com >>>>> >>>>> >>>> >>>> -- >>>> View this message in context: >>>> http://www.nabble.com/ConcurrentModificationException-while-closing-consumer-tp21867250p21907669.html >>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com. >>>> >>>> >>> >>> >>> >>> -- >>> http://blog.garytully.com >>> >>> Open Source SOA >>> http://FUSESource.com >>> >>> >> >> -- >> View this message in context: >> http://www.nabble.com/ConcurrentModificationException-while-closing-consumer-tp21867250p21924323.html >> Sent from the ActiveMQ - User mailing list archive at Nabble.com. >> > > > > > > > > > > > -- View this message in context: http://www.nabble.com/ConcurrentModificationException-while-closing-consumer-tp21867250p21945368.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
