Want to post the code for your consumer? Is each consumer in its own JVM or are all consumers sharing the same JVM? Sharing same Session/Connection?
On Thu, Feb 13, 2014 at 2:31 PM, jlindwall <jlindw...@yahoo.com> wrote: > I am stress testing activemq 5.9.0 by sending a flood of non-persistent > messages to a topic using jmeter. > > My producers are 50 threads attempting to deliver 1000 msgs/sec; each of > size 1K. There is merely a 1 millisecond delay in between message sends. > There are 100 consumers on the topic. It runs fine for 10-20 seconds or so, > then I see the consumer stop working and "GC overhead limit exceeded" errors > appear in the consumer's console (Weirdly, monitoring the gc via visualvm > it does not seem like a massive gc event...). Looking at the heap dump from > the consumer client I see a large number of ActiveMQObjectMessage instances, > owned by the GC root ActiveMDSessionExecutor.messageQueue(). > > Is the consumer message dispatching simply falling badly behind incoming > message flow? Can I tune the consumer somehow to compensate? Or is this > just the expected failure mode for this scenario? > > > Thanks for any insights. > > Sample output from the consumers: > > Created the tree successfully using JMS_Subscribers_Forever.jmx > Starting the test @ Thu Feb 13 11:33:51 PST 2014 (1392320031599) > Waiting for possible shutdown message on port 4445 > summary + 0 in 5s = 0.0/s Avg: 0 Min: 5001 Max: 5001 Err: > 1 (0.00%) Active: 100 Started: 100 Finished: 0 > summary + 0 in 35s = 0.0/s Avg: 0 Min: 4999 Max: 5006 Err: > 600 (0.00%) Active: 100 Started: 100 Finished: 0 > java.lang.OutOfMemoryError: GC overhead limit exceeded > Dumping heap to java_pid3941.hprof ... > Heap dump file created [654947324 bytes in 8.677 secs] > > Exception: java.lang.OutOfMemoryError thrown from the > UncaughtExceptionHandler in thread "ActiveMQ Session Task-1" > Uncaught Exception java.lang.OutOfMemoryError: GC overhead limit exceeded. > See log file for details. > Logging Error: Unknown error writing event. > Uncaught Exception java.lang.OutOfMemoryError: GC overhead limit exceeded. > See log file for details. > java.lang.OutOfMemoryError: GC overhead limit exceeded > Uncaught Exception java.lang.OutOfMemoryError: GC overhead limit exceeded. > See log file for details. > Uncaught Exception java.lang.OutOfMemoryError: GC overhead limit exceeded. > See log file for details. > Logging Error: Unknown error writing event. > java.lang.OutOfMemoryError: GC overhead limit exceeded > Logging Error: Unknown error writing event. > java.lang.OutOfMemoryError: GC overhead limit exceeded > Uncaught Exception java.lang.OutOfMemoryError: GC overhead limit exceeded. > See log file for details. > > Exception: java.lang.OutOfMemoryError thrown from the > UncaughtExceptionHandler in thread "ActiveMQ Transport: > tcp://server/172.55.33.120:61616@41330" > Uncaught Exception java.lang.OutOfMemoryError: GC overhead limit exceeded. > See log file for details. > Uncaught Exception java.lang.OutOfMemoryError: GC overhead limit exceeded. > See log file for details. > Uncaught Exception java.lang.OutOfMemoryError: GC overhead limit exceeded. > See log file for details. > Uncaught Exception java.lang.OutOfMemoryError: GC overhead limit exceeded. > See log file for details. > Uncaught Exception java.lang.OutOfMemoryError: GC overhead limit exceeded. > See log file for details. > Uncaught Exception java.lang.OutOfMemoryError: GC overhead limit exceeded. > See log file for details. > Uncaught Exception java.lang.OutOfMemoryError: GC overhead limit exceeded. > See log file for details. > > > > > -- > View this message in context: > http://activemq.2283324.n4.nabble.com/Stress-testing-OOME-tp4677959.html > Sent from the ActiveMQ - User mailing list archive at Nabble.com. -- Christian Posta http://www.christianposta.com/blog twitter: @christianposta
