Apologies as of course I should have included the version. AMQ Classic 5.15.1.
We have done some internal benchmarking comparing with AMQ Artemis 2.24.0, specifically on total sustainable throughput, but so far have found no compelling reason to switch away from Classic. On Sat, 26 Nov 2022 at 18:57, Justin Bertram <jbert...@apache.org> wrote: > What version of ActiveMQ are you using? > > > Justin > > On Fri, Nov 25, 2022 at 4:20 AM Mark Johnson <spike.john...@gmail.com> > wrote: > > > Hi, > > > > We have Apache Camel routes consuming messages and updating a database. > Due > > to the nature of the business, there will be multiple messages in the > queue > > that need to update the same row in the db tables. This results in a lot > of > > db transaction rollbacks as the number of consumers increases. > > > > We have proved to ourselves that using Message Groups with part of the db > > table primary key as the JMSXGroupID resolves the rollback. There will > also > > be considerably more groups than consumers, so I have no concerns about > > consumers being idle because all groups are already serviced. > > > > We have also proved that we can horizontally scale the Camel deployment > > based on queue depth. > > > > What we can't work out is how to put the two solutions together, as > whilst > > Message Group will rebalance when a consumer disconnects, it does not > > rebalance on new consumers, according to the documentation. > > > > The documentation suggests that sending in a message with a -1 SeqNr will > > force the group to be treated as new, however at the point where we are > > scaling up there will already be thousands of messages on the queue. > > > > Does the reset happen when the broker receives the message or when the > > message gets to the head of the queue? If the latter, could I use message > > priority to force it to the head? > > > > If those conditions are met then I thinking that the Camel routes could > > process 1k msgs then send in a -1 reset message for their current GroupID > > to cause it to be rebalanced. Would also need to set prefetch=1 to ensure > > that the client does not have messages buffered, although even if we > didn't > > do that, this pattern should reduce the number of rollbacks > significantly. > > > > Is there a better way for groups to be moved to new consumers? > > > > Any suggestions gratefully received. > > > > Mark Johnson > > >
