That sounds like pretty terrible performance, and although I've never looked at that code, I can't imagine a reason why the performance couldn't be improved substantially. Are you willing to do some of that digging to at least identify which code is slow (and ideally submit recommended changes to fix that code)?
If so, I'd first identify the resource that is the bottleneck (CPU? Disk I/O? Memory thrashing?), since that'll drive the rest of your investigation. Then figure out which code the time is being spent in, using whatever tools you have available (VisualVM is a good option if you don't have access to a better one, and it's in every Oracle JDK), and go from there. And don't be afraid to ask for feedback on how to approach a particular change, either here or on the dev mailing list. BTW, thanks for doing a really good job with your problem description; this was a great example of how to ask for help. Tim On May 21, 2015 2:32 AM, "contezero74" <eros.pedr...@gmail.com> wrote: > Hi Everyone, > in one of our project we are suffering for ActiveMQ performance degrade > when > we use the schedule facility (enabling schedulerSupport in ActiveMQ > configuration) with high amount of message (1M or more). We are, also, > using > Camel for route management. > > Our routes are configured as follow: > - a producer P produces 800 messages/sec and send them through a non > persistent ActiveMQ queue Q > - a consumer C1 processes messages from Q according to the following rule: > for each message it verifies if the message can be process immediately > (every message has a time-window attribute to define that). If it can, it > forwards the message to the Throttling queue T; if it cannot, it forwards > the messages into a support queue S > - a consumer Cs processes messages from S and it schedules the message > according to its time-window (setting the ActiveMQ property > ScheduledMessage.AMQ_SCHEDULED_DELAY with the following statement: > "message.setLongProperty(ScheduledMessage.AMQ_SCHEDULED_DELAY, <delay>);") > - when a scheduled message wakes up from the ActiveMQ scheduled area, it is > forwarded to the Throttling queue T > - the consumer Ct processes the message from T using a third parties > component (TCP connection) > > When P produces messages for immediate processing (i.e., without > time-window) the system performs well even with an high amount of messages > (1M or more): monitoring the queue we can see them almost empty (no message > stays pending in any queue). > > When P produces messages for delayed processing (i.e., with time-window) we > see the following behavior: > - we have messages pending in all the queues (Q and S), while we expect to > have it only on the S queue > - when the message scheduled in S wakes up they moves to the T queue: this > messages are processed according to the throttling configuration (1K > messages/sec) > - the message still pending in Q and S are moved very slowly to T (around > 50 > messages/sec) > - in case we try a second round of messages (without time-window) produced > by P, just after all the precious messages have been processed, we still > sending out messages to 50 messages/sec > > Summary of the issue: > * ActiveMQ version: 5.9.0 (configured with 3GB of max memory limit) > * Camel version: 2.15.1 > * Spring version: 4.1.3.RELEASE > * Critical route: > * Without using scheduler: within 1M messages, we process easily 1K > messages/sec (the limit depends from a Camel Throttling queue) > * Using scheduler: within 1M messages, we process 50 messages/sec > * Monitoring CPUs and memory usage, we cannot spot-out any issue: from 2% > to > 5% of CPUs usage and we never go out-of-memory (the Java GC seems works > fine > too= > * Hardware: virtualized based on VMWare: > - CPUs: 24 vCPUs mapped on 'Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz' > - Ram: 31744MB > > > cheers and thanks in advance for your help, > Eros > > > > -- > View this message in context: > http://activemq.2283324.n4.nabble.com/Performance-degrade-issue-using-ActiveMQ-scheduler-tp4696754.html > Sent from the ActiveMQ - User mailing list archive at Nabble.com. >
