We are currently using activemq for our messaging infrastructure and we are finding that on random intervals we will see messages getting dispatched to consumer connections/sessions that no longer exist (consumer has been disconnected). We were running activemq 5.11.1 and upgrade to 5.14.1 when we started seeing this issue. We have a pretty stock configuration for our broker. I originally had thought our consumers were just taking a very long time, but I started investigating through jconsole and found that connections and consumers seemed to be showing up from servers that were no longer running. I am just looking for some ideas on how to get to the bottom of this issue. I have tried recreating this issue outside of our production systems without any avail.
Some details about our stack: We are using activemq 5.14.1 The activemq broker is running on a dedicated machine as a stand alone broker. The broker is running on OpenJDK 1.7.0_79 We are using java 8, camel and spring to send and receive messages from activemq. The consumers are running a set machines that auto scale up and down Currently all consumers are configured to prefetch 1 message Consumers will close and restart after consuming 10 messages right now. We use nio transport for all client connections. We have ~ 150 queues being served by a activemq instance. Each queue has very different amounts of load, but the busiest are seeing 200-300K messages a day. To combat this issue we have enabled abortSlowAckConsumerStrategy with a timeout of 60000. This has reduced the recurrence of this issue, but it still is happening multiple times a day. The issue does not seem to correlate in a spike of activity with the consumers or on the server running activemq. All of our sessions are transacted We are using kahadb persistence adapter -- View this message in context: http://activemq.2283324.n4.nabble.com/Messages-dispatched-to-consumers-but-never-get-processed-tp4720876.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
