Quoting from an email I posted in another thread a few days ago: When a system is slow, there's always something (or several somethings, rotating between them) that is the limiting resource. I'd start by trying to determine what that resource is, using standard OS monitoring tools (top and the like). If you can tell that CPU is a primary bottleneck, use a profiler or sampler like JVisualVM to determine where the time is being spent, and share what you find here (maybe there's a bug we need to fix). If it's disk IO, make sure your Log4J logging is at the right level (i.e. not DEBUG or TRACE), and if it's still the bottleneck, consider options to upgrade your storage hardware to something more performant. If the JVM is spending a lot of time garbage collecting (JVisualVM and JConsole will tell you that), increase the broker's heap size. If your OS is heavily into swap, buy more RAM. And the list goes on, but the starting point is you characterizing what's actually slow, and we can help you figure out why it's slow.
Tim On Apr 24, 2017 2:15 AM, "anoppe" <a...@noppe.nl> wrote: > Hi everybody, > > I've setup 2 brokers with different responsibilities: > mq1 is an instance the webapplications consumes messages from and is > receiving messages also > mq2 is an instance that received messages only. > > mq2 has some composite topics configured: > <destinationInterceptors> > <virtualDestinationInterceptor> > <virtualDestinations> > <compositeTopic name="company.*.topic" forwardOnly="false"> > <forwardTo> > <topic physicalName="event.stream.topic" /> > </forwardTo> > </compositeTopic> > <compositeTopic name="data.stream.topic" forwardOnly="false"> > <forwardTo> > <topic physicalName="event.stream.topic" /> > </forwardTo> > </compositeTopic> > <compositeTopic name="alert.triggers.forwarder.topic" > forwardOnly="true"> > <forwardTo> > <topic physicalName="alert.triggers.topic" /> > </forwardTo> > </compositeTopic> > </virtualDestinations> > </virtualDestinationInterceptor> > </destinationInterceptors> > > - All topics that match the name 'company.*.topic' are duplicated to > 'event.stream.topic' > - The topic 'data.stream.topic' is duplicated to 'event.stream.topic' > - The topic 'alert.triggers.forwarder.topic' is duplicated to > 'alert.triggers.topic'. > > mq1 and mq2 are in a network of brokers, where mq2 is forwarding the > messages in 'event.stream.topic' and 'alert.triggers.topic' to mq1, so the > consumers on that instance receive those messages also. To prevent > duplication, I've excluded 'data.stream.topic' form the network-of-brokers: > <networkConnectors> > <networkConnector name="eventStreamAndAlertTriggersForwarder" > uri="static:(tcp://ip-of-mq1:61616)" duplex="false"> > <excludedDestinations> > <topic physicalName="data.stream.topic" /> > </excludedDestinations> > <staticallyIncludedDestinations> > <topic physicalName="event.stream.topic"/> > <topic physicalName="alert.triggers.topic"/> > </staticallyIncludedDestinations> > </networkConnector> > </networkConnectors> > > On a functional level this works just fine. But, we tried to loadtest our > infrastructure and ran into unexpected behaviour: When we send messages at > a > rate of ~200m/s to 'company.*.topic'@mq2, consuming from mq2 becomes very > slow: When we stop sending new messages, we are able to consume at a rate > of > ~30m/s. If we start sending messages again, the rate of consuming drops to > ~5m/s and a queue builds up. > The consumer we measured runs on the same machine. > When we send and consume messages on mq1, we get the expected rate of about > 30m/s. > > Does anyone has an idea how we can improve performance? Did I misconfigure > something? > > Regards, > Auke Noppe > > > > > > -- > View this message in context: http://activemq.2283324.n4. > nabble.com/Network-of-brokers-and-forwarding-performance-tp4725187.html > Sent from the ActiveMQ - User mailing list archive at Nabble.com. >
