On 2 August 2010 18:49, Don Santillan <donzym...@gmail.com> wrote: > Hello, > > I am trying to create an application that would use a message queue that is > consumed by multiple consumers running concurrently but would still maintain > the order of message processing by a certain ID. > > For example, I have a set of messages added to a queue: > ID1-M1 > ID1-M2 > ID2-M1 > ID3-M1 > ID2-M2 > ID3-M2 > > I need the Messages coming from ID1 get consumed by Consumer1, ID2 to > Consumer2, and ID3 to Consumer3, that maintains order such as Consumer1 will > process ID1-M1 and then ID1-M2, Consumer 2 process ID2-M1 and then ID2-M2, > and Consumer3 process ID3-M1 and then ID3-M2. > > I also like to have the Consumers come from a ConsumerPool so that if a > consumer is done processing with the messages, it will go back to the pool > and become available for other Messages owned by another ID. I also like to > have the pool a behavior that would create a new consumer in the pool if a > consumer dies so that a fix number of consumer is available from the pool. > > I saw in the documentation that the Message Group, I think, fits into the > requirement perfectly. Process messages asynchronous, maintain message > order, and failover feature. > > Unfortunately, I was not able to understand completely how to do it. I > understand that the producer needs to set a property "JMSXGroupID" to > identify which group the message will get associated and consumed by a > single consumer. What I don't understand is how the broker will select which > consumer to pass the message to? Even more, where will the consumers come > from? Do I need to create, for example, 20 consumers and subscribe them all > to the broker? What if the consumer one by one dies, can the broker > automatically recreate all 20 consumers? > > It would be greatly appreciated if anybody can write a simple code that > would show how consumers get subscribed to the broker and have an > explanation that my requirement would really fit into Message Group. If this > is not possible through Message Group, any suggestions on how to implement > this?
The broker chooses a consumer on demand to fill a hash bucket. As consumers die new consumers get chosen. So all you need to do is add message group IDs to your message (as specific as possible to use the biggest possible number of hash buckets) and create a number of consumers. There's no need to manually assign consumers to individual message group IDs or anything like that. Sometimes prefetch can be set high which can reduce the amount of concurrent consumers used; if you are only sending a small number of messages you might find that only a small number of consumers are actually used. http://activemq.apache.org/i-do-not-receive-messages-in-my-second-consumer.html http://activemq.apache.org/what-is-the-prefetch-limit-for.html so you might want to set the prefetch to say 1 if you've only a relatively small number of messages to ensure a good spread across your consumers. -- James ------- http://macstrac.blogspot.com/ Open Source Integration http://fusesource.com/
