Hi, We have Apache Camel routes consuming messages and updating a database. Due to the nature of the business, there will be multiple messages in the queue that need to update the same row in the db tables. This results in a lot of db transaction rollbacks as the number of consumers increases.
We have proved to ourselves that using Message Groups with part of the db table primary key as the JMSXGroupID resolves the rollback. There will also be considerably more groups than consumers, so I have no concerns about consumers being idle because all groups are already serviced. We have also proved that we can horizontally scale the Camel deployment based on queue depth. What we can't work out is how to put the two solutions together, as whilst Message Group will rebalance when a consumer disconnects, it does not rebalance on new consumers, according to the documentation. The documentation suggests that sending in a message with a -1 SeqNr will force the group to be treated as new, however at the point where we are scaling up there will already be thousands of messages on the queue. Does the reset happen when the broker receives the message or when the message gets to the head of the queue? If the latter, could I use message priority to force it to the head? If those conditions are met then I thinking that the Camel routes could process 1k msgs then send in a -1 reset message for their current GroupID to cause it to be rebalanced. Would also need to set prefetch=1 to ensure that the client does not have messages buffered, although even if we didn't do that, this pattern should reduce the number of rollbacks significantly. Is there a better way for groups to be moved to new consumers? Any suggestions gratefully received. Mark Johnson
