Hi, It's been quite some time now and I just wanted to follow up for archiving purposes. Since we updated everything to 5.13, we don't see the issue any more. I am convinced it had been caused by the 5.7 client library.
Best regards, Martin On Sat, Jan 9, 2016 at 6:24 AM, Tim Bain <tb...@alumni.duke.edu> wrote: > Great, thanks for reporting back. > > Tim > > On Fri, Jan 8, 2016 at 1:04 PM, Martin C. <mart...@gmx.at> wrote: > >> Hello, >> >> sorry for the late response, missed the message. We are currently very >> confident that the issue was triggered because one of the client >> applications was still using a very old 5.7 library to connect to the >> 5.12 broker. Once we updated the client library in that particular >> application, things started to improve and we are currently testing if >> upgrading everything across all clients and also the broker to 5.13 >> fixes this for good. >> >> Once done, I'll post our test results. >> >> Best regards, >> Martin >> >> >> On Sun, Dec 20, 2015 at 4:50 PM, Tim Bain <tb...@alumni.duke.edu> wrote: >> > Martin, did you ever resolve this issue? >> > >> > If not, I'd recommend looking at the messages that expire to see if there >> > is a pattern to them. >> > >> > Also, do you have a single broker, or a network of brokers? If the >> latter, >> > what is your networkTTL set to? >> > >> > Tim >> > On Dec 2, 2015 9:50 AM, "Martin Carpella" <martin.carpe...@gmail.com> >> wrote: >> > >> >> Hi, >> >> >> >> We've ran into some problems since we updated to Activemq 5.12.1. Our >> >> most busy queue has stuck messages which also do NOT expire. >> >> The queue has around 200 producers (each producer has it's own message >> >> group, making sure messages of a producer do not overtake each other) >> >> which send non-persistent messages with a timeout of 40 seconds. They >> >> produce around 20-30 msgs / second. 5 cached consumers exist. >> >> >> >> Our problem is that all 5 consumers are consuming messages but some of >> >> those messages are apparently not delivered. They get stuck in the >> >> queue and stay there. They do not expire. >> >> The only solution to "clear" the queue is to use a QueueBrowser and >> >> inspect it. Once I connect with the QueueBrowser, all messages are >> >> apparently moved to expiration. After that the processing works for a >> >> couple of minutes until the messages start clogging up again. >> >> >> >> The consumers do not use any form of selector other than the JMS >> >> message group. The operation on the server side is very lightweight >> >> and the load on the server is low so i do not think that it's the >> >> fault of the server for not processing the messages fast enough (and >> >> they should at least time out after their expiration deadline is >> >> reached). >> >> The problem scales apparently with the amount of the producers / >> >> produced messages. Systems with ~100 producers have much fewer stuck >> >> messages. >> >> >> >> All our other queues use message groups as well but work as intended. >> >> A maybe noticable difference is that the messages that get stuck are >> >> non-persistent and have a TTL. We have some high-throughput queues >> >> with non-expiring, non-persistent messages, which do not show those >> >> symptoms. >> >> >> >> Good ideas on what could be the issues are very welcome! Thanks in >> advance! >> >> >> >> Best regards, >> >> Martin >> >> >>
