Thanks Gary, that is extremely helpful. Are there plans to address this potential deficiency? You mentioned some improvement ideas, but does a ticket exist yet to address them?
I'll probably take your ideas and try to at least put something together that works for us, but since I'm very new to the activemq code base I'm not so sure that what I end up with will be upstream worthy. On Fri, Oct 09, 2009 at 02:10:23PM +0100, Gary Tully wrote: > Eric, > just in case you have not seen it, you may find some of the detail in > the the following old thread relevant to your investigation. > > http://www.nabble.com/Serious-dispatch-issue-tt23990060.html#a23996544 > > > 2009/10/8 Eric Van Dewoestine <ervan...@gmail.com>: > > Thanks for the link. Did you try applying that patch? If so did it > > resolve your issue? > > > > I'll give the patch a shot, but the note in the ticket regarding > > failures for large message counts and Joe's warnings make me a little > > apprehensive. I'll update my findings as soon as I get a chance to > > test it out. > > > > On Thu, Oct 8, 2009 at 10:17 AM, farshad <teez_ho...@hotmail.com> wrote: > >> > >> Hi Eric, > >> > >> I think you and I have run into the same problem. Please check below. > >> > >> Farshad > >> > >> http://www.nabble.com/Question-about-Queue-destinations-in-network-of-brokers-td25776018.html > >> > >> > >> Eric Van wrote: > >>> > >>> ActiveMQ 5.3.0_SNAPSHOT (Sep 8th according to the snapshots listing) > >>> > >>> I'm running into an issue with the store and forward feature of > >>> activemq, which I using in an attempt to create a highly available > >>> distributed queue. I'm trying to figure out if the issue is a > >>> misconfiguration on my part, expected behavior of activemq, or a bug. > >>> > >>> The summary of the problem is that given 2 brokers, B1 and B2, which > >>> each have one consumer, C1 and C2, which are subscribed to the same > >>> queue. If I stop a consumer on one of the brokers, the pending > >>> messages from that broker are not always forwarded to the other broker > >>> which still has a consumer, leading to those messages getting > >>> indefinitely stuck. > >>> > >>> The steps I use to reproduce this scenario are as follows (Note: all > >>> producing and consuming is performed over the stomp transport): > >>> > >>> Couple notes about the consumers: > >>> - they have a prefetchSize of 40 > >>> - the processing of messages can take some time, so for the purposes > >>> of this exercise, I've created a simple consumer that sleeps for 10 > >>> seconds before sending the message ack (using client-individual ack > >>> mode) > >>> > >>> 1. start both brokers (B1 and B2). The consumers (C1 and C2) are not > >>> yet running. > >>> 2. produce a few thousand messages to B1 > >>> Note: B1 now has a few thousand pending messages and B2 has 0. > >>> 3. start consumer C2 (listing for messages from B2) > >>> Note: messages are are successfully received and begin processing > >>> (monitoring the brokers shows pending messages decreasing). Now B2 > >>> has all the pending messages and B1 has 0. > >>> 4. start consumer C1 (listing for messages from B1) > >>> Note: no messages are received, which is another issue I have since > >>> B2 now has thousands of pending messages which C1 could help > >>> process, but instead sits idle while C2 is forced to handle all the > >>> messages. > >>> 5. stop consumer C2 > >>> Note: now I have thousands of messages sitting on B2 and 0 on B1 > >>> where a C1 is alive and ready to handle them. So at this point, > >>> despite having a consumer running, thousands of messages are stuck > >>> in the queue. > >>> 6. stop consumer C1 > >>> Note: now I have no consumers. Stopping and restarting C1 has no > >>> effect on the pending messages sitting on B1's queue. > >>> 7. stop both brokers > >>> 8. start B1, then start B2 > >>> 9. start C1 > >>> Note: now all messages have migrated from B2 to B1 and C1 is again > >>> processing messages. > >>> > >>> So after step 5, the only way to recover from the stuck messages is to > >>> restart the brokers. > >>> > >>> Below is my current connector config which I have on both brokers. > >>> I've tried playing with the various properties of the connector, but > >>> it seems as though no matter what I try the above scenario continues > >>> to occur. > >>> > >>> <networkConnector > >>> name="default-nc" > >>> uri="multicast://default?group=${broker.group}" > >>> dynamicOnly="true" > >>> networkTTL="25" > >>> suppressDuplicateQueueSubscriptions="true"/> > >>> > >>> > >>> So, is this an activemq bug? Am I mis-using activemq? Is there some > >>> other way to achieve a highly available distributed queue? > >>> > >>> Any help in this regard is greatly appreciated. > >>> > >>> -- > >>> eric > >>> > >>> > >> > >> -- > >> View this message in context: > >> http://www.nabble.com/Issue-creating-a-distributed-queue-using-store-and-forward.-tp25790672p25803310.html > >> Sent from the ActiveMQ - User mailing list archive at Nabble.com. > >> > >> > > > > -- > > eric > > > > > > -- > http://blog.garytully.com > > Open Source Integration > http://fusesource.com -- eric
