Re: Active MQ Performance
Out of two issues, I am able to resolve first one (Queue taking considerable time on persistent messages). Topic was fast because by default, concurrentStoreAndDispatchTopics is false for Topic. For Queue, it's enabled. But main difference was enableJournalDiskSyncs on each message. >From SAR command, I found that IO wait time was quite high. Sometime 30%. I switched off enableJournalDiskSyncs and concurrentStoreAndDispatchQueues and it worked. I think it's not ideal and makes it less reliable when broker kills but it works until I find an alternative to wait io. Still looking into for seconding issue. Thanks Ajay Kumar -- View this message in context: http://activemq.2283324.n4.nabble.com/Active-MQ-Performance-tp4712952p4713145.html Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
Re: Active MQ Performance
Hello Matt, Thanks for your message. I will do these settings and see what happens. I have attached links to code and results. My test show these two problems: - ActiveMQ Queue takes more time in Persistent Mode (Q28,Q31 rows in excel sheet) as compared to Sonic. - ActiveMQ Non Persistent Per Message Max time is very high. (Example: U59, U56). It looks consumer is not fast enough to process them. Overall, Maximum Time taken by any message in flight between producer and sender in ActiveMQ is quote high. I have copied the code to IDEONE editor (it's just a copy and IDEONE don't have necessary libs) to share if it helps: Producer: http://ideone.com/4nmARP Consumer: http://ideone.com/QnFHYk Results: https://docs.google.com/spreadsheets/d/1hHt7EsgwGSxHR11edunWSuzzTbc88Wch8lFkBXmPxUI/edit?usp=sharing If you can't access code on IDEONE: Producer - https://docs.google.com/document/d/1RbGUKYUbWUk8LcxqbCwf53_lNEfhnxfDz3TPqSNT2Dg/edit?usp=sharing Consumer - https://docs.google.com/document/d/1jW5qPG40yUTBaHezEpjKrwBFZriL2bpxHZcM0U_2Gfk/edit?usp=sharing Sorry if above link passage is not as per forum protocol. I am new to the forum. Thanks AJ -- View this message in context: http://activemq.2283324.n4.nabble.com/Active-MQ-Performance-tp4712952p4713058.html Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
Re: Active MQ Performance
Hi- How big are the messages? Can you post a link to the C++ code. There may be some gotchas in how its sent or recv'd. It sounds like you have confirmed that you do not have any network or disk performance issues, since SonicMQ is getting reasonable numbers on the same server. Out-of-the-box ActiveMQ is configured to ensure you don't run out of memory or lose messages, so it is setup to work for very low memory footprints. As a starting point adjust the memoryUsage setting here: Flip the openwire to nio as well: On 6/14/16 7:56 AM, ActiveMQ Investigation wrote: Hello Celebert, I am using apache-activemq-5.12.1. I found out the issue. My subscribers were on same machine as ActiveMQ install. While 5 subscribers, consuming messages on same machine as install, it was quite intensive. I moved subscribers to a separate machine than install and it took 9 seconds to send messages and almost similar to receive. This is about persistent messages. For non persistent, in new setup, it takes 1.7 seconds which is acceptable. However, consumer (gets all msgs in 6 seconds) in is showing messages waiting for 1+ seconds. For SonicMQ, time taken is consistent irrespective of consumer on machine where sonic install is there. I feel that this 9 second is still high for sending 25000 msgs. It can do better. Also, can this CPU and memory intensiveness improved. I will go through Artemis. Sorry for naive query but how different Artemis is to ActiveMQ. I haven't explored that front. Thanks AJ Thanks AJ -- View this message in context: http://activemq.2283324.n4.nabble.com/Active-MQ-Performance-tp4712952p4712960.html Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
Re: Active MQ Performance
Artemis is an initiative to make a new broker with similar features to ActiveMQ5 which is getting closer and closer each day. I would say It's worth giving it a try. 1.3.0 just passed the voting, you should see the announcement soon for the release. On Tue, Jun 14, 2016 at 8:56 AM, ActiveMQ Investigation <ajay.ku...@winterflood.com> wrote: > Hello Celebert, > > I am using apache-activemq-5.12.1. > > I found out the issue. My subscribers were on same machine as ActiveMQ > install. While 5 subscribers, consuming messages on same machine as install, > it was quite intensive. I moved subscribers to a separate machine than > install and it took 9 seconds to send messages and almost similar to > receive. This is about persistent messages. > > For non persistent, in new setup, it takes 1.7 seconds which is acceptable. > However, consumer (gets all msgs in 6 seconds) in is showing messages > waiting for 1+ seconds. > > > For SonicMQ, time taken is consistent irrespective of consumer on machine > where sonic install is there. > > > I feel that this 9 second is still high for sending 25000 msgs. It can do > better. Also, can this CPU and memory intensiveness improved. > > I will go through Artemis. Sorry for naive query but how different Artemis > is to ActiveMQ. I haven't explored that front. > > Thanks > AJ > > > > > > > > > > Thanks > AJ > > > > -- > View this message in context: > http://activemq.2283324.n4.nabble.com/Active-MQ-Performance-tp4712952p4712960.html > Sent from the ActiveMQ - Dev mailing list archive at Nabble.com. -- Clebert Suconic
Re: Active MQ Performance
Hello Celebert, I am using apache-activemq-5.12.1. I found out the issue. My subscribers were on same machine as ActiveMQ install. While 5 subscribers, consuming messages on same machine as install, it was quite intensive. I moved subscribers to a separate machine than install and it took 9 seconds to send messages and almost similar to receive. This is about persistent messages. For non persistent, in new setup, it takes 1.7 seconds which is acceptable. However, consumer (gets all msgs in 6 seconds) in is showing messages waiting for 1+ seconds. For SonicMQ, time taken is consistent irrespective of consumer on machine where sonic install is there. I feel that this 9 second is still high for sending 25000 msgs. It can do better. Also, can this CPU and memory intensiveness improved. I will go through Artemis. Sorry for naive query but how different Artemis is to ActiveMQ. I haven't explored that front. Thanks AJ Thanks AJ -- View this message in context: http://activemq.2283324.n4.nabble.com/Active-MQ-Performance-tp4712952p4712960.html Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
Re: Active MQ Performance
What version ? Can you try Artemis ? On Tuesday, June 14, 2016, ActiveMQ Investigation < ajay.ku...@winterflood.com> wrote: > Hello, > > I am doing some raw performance tests on ActiveMQ to replace SonicMQ (JMS > based) on C++ side. We have CMS using OpenWire. > My test has got five Non Durable subscribers on a Topic. When I send 25000 > messages, following happens: > > 1. Persistent - It takes 3 times (27sec) more than what it takes on SonicMQ > (9sec). I need to find out what's causing this delay and how to improve it. > I understand that publisher will wait until broker comes back to ack > messages. I am using Auto_ACK. I have looked at SonicMQ settings for IO > Buffer and socket buffer and do the same for ActiveMQ but no success. > > > 2. Non Persistent - This one is faster as fire and forget but messages are > sometime 2 seconds in flight on subscriber which is unacceptable to our use > case. I tried setting prefetch etc thinking it's slow consumer but no > success. > > Hardware is same for both. ActiveMQ install is default installation. I am > going through performance tuning guide mentioned in the forum but any quick > clues / areas to look at will be appreciated. > > Sorry if my query is too subjective but I want to find out where ActiveMQ > is > taking time and why. I have done raw tests just using stomp on ActiveMQ (No > CPP involved) and it was way faster as compared to Sonic. So I suspect, I > need to tweak some settings which may be causing it. > > > Thanks > AJ > > > > > -- > View this message in context: > http://activemq.2283324.n4.nabble.com/Active-MQ-Performance-tp4712952.html > Sent from the ActiveMQ - Dev mailing list archive at Nabble.com. > -- Clebert Suconic
Active MQ Performance
Hello, I am doing some raw performance tests on ActiveMQ to replace SonicMQ (JMS based) on C++ side. We have CMS using OpenWire. My test has got five Non Durable subscribers on a Topic. When I send 25000 messages, following happens: 1. Persistent - It takes 3 times (27sec) more than what it takes on SonicMQ (9sec). I need to find out what's causing this delay and how to improve it. I understand that publisher will wait until broker comes back to ack messages. I am using Auto_ACK. I have looked at SonicMQ settings for IO Buffer and socket buffer and do the same for ActiveMQ but no success. 2. Non Persistent - This one is faster as fire and forget but messages are sometime 2 seconds in flight on subscriber which is unacceptable to our use case. I tried setting prefetch etc thinking it's slow consumer but no success. Hardware is same for both. ActiveMQ install is default installation. I am going through performance tuning guide mentioned in the forum but any quick clues / areas to look at will be appreciated. Sorry if my query is too subjective but I want to find out where ActiveMQ is taking time and why. I have done raw tests just using stomp on ActiveMQ (No CPP involved) and it was way faster as compared to Sonic. So I suspect, I need to tweak some settings which may be causing it. Thanks AJ -- View this message in context: http://activemq.2283324.n4.nabble.com/Active-MQ-Performance-tp4712952.html Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
[jira] Commented: (AMQ-2333) Active MQ performance issues when there are more than 100 queue'd up messages
[ https://issues.apache.org/activemq/browse/AMQ-2333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=53031#action_53031 ] Gary Tully commented on AMQ-2333: - investigate turning off producer flow control and increasing the memory available to the broker through the systemUsage config. Active MQ performance issues when there are more than 100 queue'd up messages - Key: AMQ-2333 URL: https://issues.apache.org/activemq/browse/AMQ-2333 Project: ActiveMQ Issue Type: Bug Components: Broker, JMS client Affects Versions: 5.2.0 Reporter: Marcus Malcom Priority: Critical Over the past couple of days some of our queues get rather full because of downstream problems. The messages start numbering in the 1000's. When that happens ActiveMQ slows way down. I believe is slows down because we are trying to produce a message to the overloaded queue and it's taking a long time (minutes instead of seconds). Once the overloaded queue is emptied the problems go away. Our system pretty much has all the defaults. Note: this was not a problem before upgrading to 5.2.0 Any ideas on what should be done? -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (AMQ-2333) Active MQ performance issues when there are more than 100 queue'd up messages
[
https://issues.apache.org/activemq/browse/AMQ-2333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=53047#action_53047
]
Marcus Malcom edited comment on AMQ-2333 at 7/30/09 8:23 AM:
-
OK, I'm going to try the systemUsage first. Here's my current config:
{panel}
!-- The maximum about of space the broker will use before slowing
down producers --
systemUsage
systemUsage
memoryUsage
memoryUsage limit=20 mb/
/memoryUsage
storeUsage
storeUsage limit=1 gb name=foo/
/storeUsage
tempUsage
tempUsage limit=100 mb/
/tempUsage
/systemUsage
/systemUsage
{panel}
Switching it to:
{panel}
memoryUsage limit=50 mb/
tempUsage limit=500 mb/
{panel}
Hopefully that will fix the problem - will let you know shortly.
was (Author: malc...@ochin.org):
OK, I'm going to try the systemUsage first. Here my current config:
{panel}
!-- The maximum about of space the broker will use before slowing
down producers --
systemUsage
systemUsage
memoryUsage
memoryUsage limit=20 mb/
/memoryUsage
storeUsage
storeUsage limit=1 gb name=foo/
/storeUsage
tempUsage
tempUsage limit=100 mb/
/tempUsage
/systemUsage
/systemUsage
{panel}
Switching it to:
{panel}
memoryUsage limit=50 mb/
tempUsage limit=500 mb/
{panel}
Hopefully that will fix the problem - will let you know shortly.
Active MQ performance issues when there are more than 100 queue'd up messages
-
Key: AMQ-2333
URL: https://issues.apache.org/activemq/browse/AMQ-2333
Project: ActiveMQ
Issue Type: Bug
Components: Broker, JMS client
Affects Versions: 5.2.0
Reporter: Marcus Malcom
Priority: Critical
Over the past couple of days some of our queues get rather full because of
downstream problems. The messages start numbering in the 1000's. When that
happens ActiveMQ slows way down. I believe is slows down because we are
trying to produce a message to the overloaded queue and it's taking a long
time (minutes instead of seconds). Once the overloaded queue is emptied the
problems go away.
Our system pretty much has all the defaults.
Note: this was not a problem before upgrading to 5.2.0
Any ideas on what should be done?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AMQ-2333) Active MQ performance issues when there are more than 100 queue'd up messages
[
https://issues.apache.org/activemq/browse/AMQ-2333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=53047#action_53047
]
Marcus Malcom commented on AMQ-2333:
OK, I'm going to try the systemUsage first. Here my current config:
{panel}
!-- The maximum about of space the broker will use before slowing
down producers --
systemUsage
systemUsage
memoryUsage
memoryUsage limit=20 mb/
/memoryUsage
storeUsage
storeUsage limit=1 gb name=foo/
/storeUsage
tempUsage
tempUsage limit=100 mb/
/tempUsage
/systemUsage
/systemUsage
{panel}
Switching it to:
{panel}
memoryUsage limit=50 mb/
tempUsage limit=500 mb/
{panel}
Hopefully that will fix the problem - will let you know shortly.
Active MQ performance issues when there are more than 100 queue'd up messages
-
Key: AMQ-2333
URL: https://issues.apache.org/activemq/browse/AMQ-2333
Project: ActiveMQ
Issue Type: Bug
Components: Broker, JMS client
Affects Versions: 5.2.0
Reporter: Marcus Malcom
Priority: Critical
Over the past couple of days some of our queues get rather full because of
downstream problems. The messages start numbering in the 1000's. When that
happens ActiveMQ slows way down. I believe is slows down because we are
trying to produce a message to the overloaded queue and it's taking a long
time (minutes instead of seconds). Once the overloaded queue is emptied the
problems go away.
Our system pretty much has all the defaults.
Note: this was not a problem before upgrading to 5.2.0
Any ideas on what should be done?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (AMQ-2333) Active MQ performance issues when there are more than 100 queue'd up messages
[ https://issues.apache.org/activemq/browse/AMQ-2333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Malcom resolved AMQ-2333. Resolution: Fixed The configuration changes that I made fixed our problem. Thank you for the help! Active MQ performance issues when there are more than 100 queue'd up messages - Key: AMQ-2333 URL: https://issues.apache.org/activemq/browse/AMQ-2333 Project: ActiveMQ Issue Type: Bug Components: Broker, JMS client Affects Versions: 5.2.0 Reporter: Marcus Malcom Priority: Critical Over the past couple of days some of our queues get rather full because of downstream problems. The messages start numbering in the 1000's. When that happens ActiveMQ slows way down. I believe is slows down because we are trying to produce a message to the overloaded queue and it's taking a long time (minutes instead of seconds). Once the overloaded queue is emptied the problems go away. Our system pretty much has all the defaults. Note: this was not a problem before upgrading to 5.2.0 Any ideas on what should be done? -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (AMQ-2333) Active MQ performance issues when there are more than a 1000 queue'd up messages
[ https://issues.apache.org/activemq/browse/AMQ-2333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=53024#action_53024 ] Marcus Malcom commented on AMQ-2333: This just happened w/ only 142 messages in a queue Active MQ performance issues when there are more than a 1000 queue'd up messages Key: AMQ-2333 URL: https://issues.apache.org/activemq/browse/AMQ-2333 Project: ActiveMQ Issue Type: Bug Components: Broker, JMS client Affects Versions: 5.2.0 Reporter: Marcus Malcom Over the past couple of days some of our queues get rather full because of downstream problems. The messages start numbering in the 1000's. When that happens ActiveMQ slows way down. I believe is slows down because we are trying to produce a message to the overloaded queue and it's taking a long time (minutes instead of seconds). Once the overloaded queue is emptied the problems go away. Our system pretty much has all the defaults. Note: this was not a problem before upgrading to 5.2.0 Any ideas on what should be done? -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (AMQ-2333) Active MQ performance issues when there are more than a 100 queue'd up messages
[ https://issues.apache.org/activemq/browse/AMQ-2333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Malcom updated AMQ-2333: --- Summary: Active MQ performance issues when there are more than a 100 queue'd up messages (was: Active MQ performance issues when there are more than a 1000 queue'd up messages) Active MQ performance issues when there are more than a 100 queue'd up messages --- Key: AMQ-2333 URL: https://issues.apache.org/activemq/browse/AMQ-2333 Project: ActiveMQ Issue Type: Bug Components: Broker, JMS client Affects Versions: 5.2.0 Reporter: Marcus Malcom Over the past couple of days some of our queues get rather full because of downstream problems. The messages start numbering in the 1000's. When that happens ActiveMQ slows way down. I believe is slows down because we are trying to produce a message to the overloaded queue and it's taking a long time (minutes instead of seconds). Once the overloaded queue is emptied the problems go away. Our system pretty much has all the defaults. Note: this was not a problem before upgrading to 5.2.0 Any ideas on what should be done? -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (AMQ-2333) Active MQ performance issues when there are more than 100 queue'd up messages
[ https://issues.apache.org/activemq/browse/AMQ-2333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Malcom updated AMQ-2333: --- Summary: Active MQ performance issues when there are more than 100 queue'd up messages (was: Active MQ performance issues when there are more than a 100 queue'd up messages) Active MQ performance issues when there are more than 100 queue'd up messages - Key: AMQ-2333 URL: https://issues.apache.org/activemq/browse/AMQ-2333 Project: ActiveMQ Issue Type: Bug Components: Broker, JMS client Affects Versions: 5.2.0 Reporter: Marcus Malcom Over the past couple of days some of our queues get rather full because of downstream problems. The messages start numbering in the 1000's. When that happens ActiveMQ slows way down. I believe is slows down because we are trying to produce a message to the overloaded queue and it's taking a long time (minutes instead of seconds). Once the overloaded queue is emptied the problems go away. Our system pretty much has all the defaults. Note: this was not a problem before upgrading to 5.2.0 Any ideas on what should be done? -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (AMQ-2333) Active MQ performance issues when there are more than 100 queue'd up messages
[ https://issues.apache.org/activemq/browse/AMQ-2333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Malcom updated AMQ-2333: --- Priority: Critical (was: Major) Active MQ performance issues when there are more than 100 queue'd up messages - Key: AMQ-2333 URL: https://issues.apache.org/activemq/browse/AMQ-2333 Project: ActiveMQ Issue Type: Bug Components: Broker, JMS client Affects Versions: 5.2.0 Reporter: Marcus Malcom Priority: Critical Over the past couple of days some of our queues get rather full because of downstream problems. The messages start numbering in the 1000's. When that happens ActiveMQ slows way down. I believe is slows down because we are trying to produce a message to the overloaded queue and it's taking a long time (minutes instead of seconds). Once the overloaded queue is emptied the problems go away. Our system pretty much has all the defaults. Note: this was not a problem before upgrading to 5.2.0 Any ideas on what should be done? -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (AMQ-2333) Active MQ performance issues when there are more than a 1000 queue'd up messages
Active MQ performance issues when there are more than a 1000 queue'd up messages Key: AMQ-2333 URL: https://issues.apache.org/activemq/browse/AMQ-2333 Project: ActiveMQ Issue Type: Bug Components: Broker, JMS client Affects Versions: 5.2.0 Reporter: Marcus Malcom Over the past couple of days some of our queues get rather full because of downstream problems. The messages start numbering in the 1000's. When that happens ActiveMQ slows way down. I believe is slows down because we are trying to produce a message to the overloaded queue and it's taking a long time (minutes instead of seconds). Once the overloaded queue is emptied the problems go away. Our system pretty much has all the defaults. Note: this was not a problem before upgrading to 5.2.0 Any ideas on what should be done? -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
