[jira] [Updated] (AMQ-7066) Using dynamic scaling and message grouping feature together makes messages stuck
[ https://issues.apache.org/jira/browse/AMQ-7066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] subhajit maiti updated AMQ-7066: Description: Recently we observed a strange problem in one of our flows. We could see some messages being stuck in the queue when we are using the message grouping and dynamic scaling feature together. *Usecase:* We want to process the messages belonging to a user (application user) in a sequence. Multiple users can send messages to the same queue, but messages originating from transactions made by a user should be processed sequentially. In order to do so we are using the "JMSXGroupID" header to group the messages and the "username" serves as the group key. *Our configuration and versions:* # We are using ActiveMQ 5.15.5 # We are using camel routing to connect to ActiveMQ, both on the producer and consumer side. Camel version : 2.20.1 # We are using the following properties in the consumer URI to enable dynamic scaling : maxConcurrentConsumers, idleTaskExecutionLimit (defaulted to 10), maxMessagesPerTask (defaulted to 250) # JDK version : 1.8.0_91 *Observations:* # The messages are stuck for some of the groups. Messages belonging to other groups are processing fine. There is no identifiable pattern for the groups that are stuck. During each execution some random groups got stuck. # The messages get processed when the consumer container is restarted and works fine for sometime before the issue recurs. # *When we verified from the admin console, we could see that the number of consumers are going beyond the max value set for it.* # We verified from the thread dumps that all the consumers are healthy and are in TIMED_WAITING state. Even though there are some messages stuck in the queue, they are not being delivered to the waiting consumers. # We have entry-exit logs for the producers and consumers. We could see the messages are being sent successfully by the producer and are never delivered to the consumer. *Expected behaviour* : All messages should be delivered to the consumers. *Actual behaviour* : Some of the messages belonging to random groups get stuck in the queue until the consumer container is restarted. We found a similar issue https://issues.apache.org/jira/browse/AMQ-4613 on your board reported long back without any activity. Could these be due to the same root cause? was: Recently we observed a strange problem in one of our flows. We could see some messages being stuck in the queue when we are using the message grouping and dynamic scaling feature together. *Usecase:* We want to process the messages belonging to a user (application user) in a sequence. Multiple users can send messages to the same queue, but messages originating from transactions made by a user should be processed sequentially. In order to do so we are using the "JMSXGroupID" header to group the messages and the "username" serves as the group key. *Our configuration and versions:* # We are using ActiveMQ 5.15.5 # We are using camel routing to connect to ActiveMQ, both on the producer and consumer side. Camel version : 2.20.1 # We are using the following properties in the consumer URI to enable dynamic scaling : maxConcurrentConsumers, idleTaskExecutionLimit (defaulted to 10), maxMessagesPerTask (defaulted to 250) # JDK version : 1.8.0_91 *Observations:* # The messages are stuck for some of the groups. Messages belonging to other groups are processing fine. There is no identifiable pattern for the groups that are stuck. During each execution some random groups got stuck. # The messages get processed when the consumer container is restarted and works fine for sometime before the issue recurs. # When we verified from the admin console, we could see that the number of consumers are going beyond the max value set for it. # We verified from the thread dumps that all the consumers are healthy and are in TIMED_WAITING state. Even though there are some messages stuck in the queue, they are not being delivered to the waiting consumers. # We have entry-exit logs for the producers and consumers. We could see the messages are being sent successfully by the producer and are never delivered to the consumer. *Expected behaviour* : All messages should be delivered to the consumers. *Actual behaviour* : Some of the messages belonging to random groups get stuck in the queue until the consumer container is restarted. > Using dynamic scaling and message grouping feature together makes messages > stuck > > > Key: AMQ-7066 > URL: https://issues.apache.org/jira/browse/AMQ-7066 > Project: ActiveMQ > Issue Type: Bug > Components: activemq-camel >Affects Versions: 5.15.5 >Reporter: subhajit maiti >Priority:
[jira] [Updated] (AMQ-7066) Using dynamic scaling and message grouping feature together makes messages stuck
[ https://issues.apache.org/jira/browse/AMQ-7066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] subhajit maiti updated AMQ-7066: Description: Recently we observed a strange problem in one of our flows. We could see some messages being stuck in the queue when we are using the message grouping and dynamic scaling feature together. *Usecase:* We want to process the messages belonging to a user (application user) in a sequence. Multiple users can send messages to the same queue, but messages originating from transactions made by a user should be processed sequentially. In order to do so we are using the "JMSXGroupID" header to group the messages and the "username" serves as the group key. *Our configuration and versions:* # We are using ActiveMQ 5.15.5 # We are using camel routing to connect to ActiveMQ, both on the producer and consumer side. Camel version : 2.20.1 # We are using the following properties in the consumer URI to enable dynamic scaling : maxConcurrentConsumers, idleTaskExecutionLimit (defaulted to 10), maxMessagesPerTask (defaulted to 250) # JDK version : 1.8.0_91 *Observations:* # The messages are stuck for some of the groups. Messages belonging to other groups are processing fine. There is no identifiable pattern for the groups that are stuck. During each execution some random groups got stuck. # The messages get processed when the consumer container is restarted and works fine for sometime before the issue recurs. # When we verified from the admin console, we could see that the number of consumers are going beyond the max value set for it. # We verified from the thread dumps that all the consumers are healthy and are in TIMED_WAITING state. Even though there are some messages stuck in the queue, they are not being delivered to the waiting consumers. # We have entry-exit logs for the producers and consumers. We could see the messages are being sent successfully by the producer and are never delivered to the consumer. *Expected behaviour* : All messages should be delivered to the consumers. *Actual behaviour* : Some of the messages belonging to random groups get stuck in the queue until the consumer container is restarted. was: Recently we observed a strange problem in one of our flows. We could see some messages being stuck in the queue when we are using the message grouping and dynamic scaling feature together. *Usecase:* We want to process the messages belonging to a user (application user) in a sequence. Multiple users can send messages to the same queue, but messages originating from transactions made by a user should be processed sequentially. In order to do so we are using the "JMSXGroupID" header to group the messages and the "username" serves as the group key. *Our configuration and versions:* # We are using ActiveMQ 5.15.5 # We are using camel routing to connect to ActiveMQ, both on the producer and consumer side. Camel version : 2.20.1 # We are using the following properties in the consumer URI to enable dynamic scaling : maxConcurrentConsumers, idleTaskExecutionLimit (defaulted to 10) maxMessagesPerTask (defaulted to 250) # JDK version : 1.8.0_91 *Observations:* # The messages are stuck for some of the groups. Messages belonging to other groups are processing fine. There is no identifiable pattern for the groups that are stuck. During each execution some random groups got stuck. # The messages get processed when the consumer container is restarted and works fine for sometime before the issue recurs. # When we verified from the admin console, we could see that the number of consumers are going beyond the max value set for it. # We verified from the thread dumps that all the consumers are healthy and are in TIMED_WAITING state. Even though there are some messages stuck in the queue, they are not being delivered to the waiting consumers. # We have entry-exit logs for the producers and consumers. We could see the messages are being sent successfully by the producer and are never delivered to the consumer. Expected behaviour : All messages should be delivered to the consumers. Actual behaviour : Some of the messages belonging to random groups get stuck in the queue until the consumer container is restarted. > Using dynamic scaling and message grouping feature together makes messages > stuck > > > Key: AMQ-7066 > URL: https://issues.apache.org/jira/browse/AMQ-7066 > Project: ActiveMQ > Issue Type: Bug > Components: activemq-camel >Affects Versions: 5.15.5 >Reporter: subhajit maiti >Priority: Critical > > Recently we observed a strange problem in one of our flows. We could see some > messages being stuck in the queue when we are using the message grouping and > dynamic
[jira] [Created] (AMQ-7066) Using dynamic scaling and message grouping feature together makes messages stuck
subhajit maiti created AMQ-7066: --- Summary: Using dynamic scaling and message grouping feature together makes messages stuck Key: AMQ-7066 URL: https://issues.apache.org/jira/browse/AMQ-7066 Project: ActiveMQ Issue Type: Bug Components: activemq-camel Affects Versions: 5.15.5 Reporter: subhajit maiti Recently we observed a strange problem in one of our flows. We could see some messages being stuck in the queue when we are using the message grouping and dynamic scaling feature together. *Usecase:* We want to process the messages belonging to a user (application user) in a sequence. Multiple users can send messages to the same queue, but messages originating from transactions made by a user should be processed sequentially. In order to do so we are using the "JMSXGroupID" header to group the messages and the "username" serves as the group key. *Our configuration and versions:* # We are using ActiveMQ 5.15.5 # We are using camel routing to connect to ActiveMQ, both on the producer and consumer side. Camel version : 2.20.1 # We are using the following properties in the consumer URI to enable dynamic scaling : maxConcurrentConsumers, idleTaskExecutionLimit (defaulted to 10) maxMessagesPerTask (defaulted to 250) # JDK version : 1.8.0_91 *Observations:* # The messages are stuck for some of the groups. Messages belonging to other groups are processing fine. There is no identifiable pattern for the groups that are stuck. During each execution some random groups got stuck. # The messages get processed when the consumer container is restarted and works fine for sometime before the issue recurs. # When we verified from the admin console, we could see that the number of consumers are going beyond the max value set for it. # We verified from the thread dumps that all the consumers are healthy and are in TIMED_WAITING state. Even though there are some messages stuck in the queue, they are not being delivered to the waiting consumers. # We have entry-exit logs for the producers and consumers. We could see the messages are being sent successfully by the producer and are never delivered to the consumer. Expected behaviour : All messages should be delivered to the consumers. Actual behaviour : Some of the messages belonging to random groups get stuck in the queue until the consumer container is restarted. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
