I am seeing some strange behavior from activemq 5.2.0 related to delivery of messages with a redelivered flag.
We have configured the broker to use jdbc persistence (no journal) talking to mysql and are using the STOMP protocol. For testing, I have a consumer that for each message will BEGIN a transaction, ACK message on the first queue , and SEND a copy of the message (being careful to clear out the appropriate headers like message-id) to a different queue. It then waits for a RECEIPT message and that triggers a COMMIT. If I put persistent messages in the queue, start my consumer but kill it before exhausting the messages, and restart I always seem to end up with a few messages that are “stuck” I can browse the messages (they show “redelivered=true”) in the admin console, but nothing short of restarting the broker will allow them to be redelivered. Upon restarting the consumer, it appears that activemq attempts to push the message (see log output below) but the stomp client never sees them, and that activity never gets logged in the stomp trace. Any ideas what could be happening and if there is a workaround? I have thread dumps as well, but jconsole does not detect any deadlocks. On last data point, if I put new messages in the queue they get processed as I would expect but the stuck messages remain. I really want to avoid regular restarts of activemq in production, so any help would be greatly appreciated! David Snyder -------------------- QUEUE STATS -------------------- Queue->orders->Attributes: DequeueCount: 996 DispatchCount: 996 EnqueueCount: 1000 InFlightCount: 114 QueueSize: 4 ProducerFlowControl: false UseCache: true Note: I had a pretty high prefetch for this test, and I think the in flight count reflects the number of messages that were delivered, but never ack’d due to killing the consumer. I see the same behavior with a prefetch of 1. -------------------- ACTIVEMQ LOG -------------------- DEBUG Queue - queue://orders - Recovery - Message pushed '12446788 - Message ID:dsynder-1761-1247148825394-1:0:-1:1:1 dropped=false locked=true' to subscription: 'QueueSubscription: consumer=ID:dsynder-1761-1247148825394-1:3:-1:4, destinations=1, dispatched=0, delivered=0, pending=0' DEBUG Queue - queue://orders - Recovery - Message pushed '17435985 - Message ID:dsynder-1761-1247148825394-1:0:-1:1:2 dropped=false locked=true' to subscription: 'QueueSubscription: consumer=ID:dsynder-1761-1247148825394-1:3:-1:4, destinations=1, dispatched=0, delivered=0, pending=0' DEBUG Queue - queue://orders - Recovery - Message pushed '7846644 - Message ID:dsynder-1761-1247148825394-1:0:-1:1:3 dropped=false locked=true' to subscription: 'QueueSubscription: consumer=ID:dsynder-1761-1247148825394-1:3:-1:4, destinations=1, dispatched=0, delivered=0, pending=0' DEBUG Queue - queue://orders - Recovery - Message pushed '27310413 - Message ID:dsynder-1761-1247148825394-1:0:-1:1:4 dropped=false locked=true' to subscription: 'QueueSubscription: consumer=ID:dsynder-1761-1247148825394-1:3:-1:4, destinations=1, dispatched=0, delivered=0, pending=0' DEBUG JDBCPersistenceAdapter - Cleaning up old messages.DEBUG DefaultJDBCAdapter - Executing SQL: DELETE FROM ACTIVEMQ_MSGSWHERE ( EXPIRATION<>0 AND EXPIRATION<?) OR ID <= ( SELECT min(ACTIVEMQ_ACKS.LAST_ACKED_ID) FROM ACTIVEMQ_ACKS WHERE ACTIVEMQ_ACKS.CONTAINER=ACTIVEMQ_MSGS.CONTAI NER) DEBUG DefaultJDBCAdapter - Deleted 0 old message(s). DEBUG JDBCPersistenceAdapter - Cleanup done. -------------------- STOMP TRACE -------------------- 2009-07-09 10:42:07,579 [0.240.145:50282] TRACE StompTransportFilter - Received: SUBSCRIBE ack:client activemq.prefetchSize:1 activemq.dispatchAsync:true destination:/queue/orders id:34b52eb2560982879bd8febb4590c7b2 x-client:127.0.0.1 x-digest:MD5:d41d8cd98f00b204e9800998ecf8427e -- View this message in context: http://www.nabble.com/ActiveMQ-stuck-messages-when-%27redelivered-%3D-true%27-tp24413676p24413676.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
