BTW, I thought from your description that mq2 was a publish-only node, but you said you're testing performance by consuming from it. Can you please describe again how you're using the nodes, using the words "publish" and "consume" instead of the ambiguous word "receive"?
Tim On Apr 25, 2017 7:14 AM, "Tim Bain" <tb...@alumni.duke.edu> wrote: > Quoting from an email I posted in another thread a few days ago: > > When a system is slow, there's always something (or several somethings, > rotating between them) that is the limiting resource. I'd start by trying > to determine what that resource is, using standard OS monitoring tools (top > and the like). If you can tell that CPU is a primary bottleneck, use a > profiler or sampler like JVisualVM to determine where the time is being > spent, and share what you find here (maybe there's a bug we need to fix). > If it's disk IO, make sure your Log4J logging is at the right level (i.e. > not DEBUG or TRACE), and if it's still the bottleneck, consider options to > upgrade your storage hardware to something more performant. If the JVM is > spending a lot of time garbage collecting (JVisualVM and JConsole will tell > you that), increase the broker's heap size. If your OS is heavily into > swap, buy more RAM. And the list goes on, but the starting point is you > characterizing what's actually slow, and we can help you figure out why > it's slow. > > Tim > > On Apr 24, 2017 2:15 AM, "anoppe" <a...@noppe.nl> wrote: > >> Hi everybody, >> >> I've setup 2 brokers with different responsibilities: >> mq1 is an instance the webapplications consumes messages from and is >> receiving messages also >> mq2 is an instance that received messages only. >> >> mq2 has some composite topics configured: >> <destinationInterceptors> >> <virtualDestinationInterceptor> >> <virtualDestinations> >> <compositeTopic name="company.*.topic" forwardOnly="false"> >> <forwardTo> >> <topic physicalName="event.stream.topic" /> >> </forwardTo> >> </compositeTopic> >> <compositeTopic name="data.stream.topic" >> forwardOnly="false"> >> <forwardTo> >> <topic physicalName="event.stream.topic" /> >> </forwardTo> >> </compositeTopic> >> <compositeTopic name="alert.triggers.forwarder.topic" >> forwardOnly="true"> >> <forwardTo> >> <topic physicalName="alert.triggers.topic" /> >> </forwardTo> >> </compositeTopic> >> </virtualDestinations> >> </virtualDestinationInterceptor> >> </destinationInterceptors> >> >> - All topics that match the name 'company.*.topic' are duplicated to >> 'event.stream.topic' >> - The topic 'data.stream.topic' is duplicated to 'event.stream.topic' >> - The topic 'alert.triggers.forwarder.topic' is duplicated to >> 'alert.triggers.topic'. >> >> mq1 and mq2 are in a network of brokers, where mq2 is forwarding the >> messages in 'event.stream.topic' and 'alert.triggers.topic' to mq1, so the >> consumers on that instance receive those messages also. To prevent >> duplication, I've excluded 'data.stream.topic' form the >> network-of-brokers: >> <networkConnectors> >> <networkConnector name="eventStreamAndAlertTriggersForwarder" >> uri="static:(tcp://ip-of-mq1:61616)" duplex="false"> >> <excludedDestinations> >> <topic physicalName="data.stream.topic" /> >> </excludedDestinations> >> <staticallyIncludedDestinations> >> <topic physicalName="event.stream.topic"/> >> <topic physicalName="alert.triggers.topic"/> >> </staticallyIncludedDestinations> >> </networkConnector> >> </networkConnectors> >> >> On a functional level this works just fine. But, we tried to loadtest our >> infrastructure and ran into unexpected behaviour: When we send messages >> at a >> rate of ~200m/s to 'company.*.topic'@mq2, consuming from mq2 becomes very >> slow: When we stop sending new messages, we are able to consume at a rate >> of >> ~30m/s. If we start sending messages again, the rate of consuming drops to >> ~5m/s and a queue builds up. >> The consumer we measured runs on the same machine. >> When we send and consume messages on mq1, we get the expected rate of >> about >> 30m/s. >> >> Does anyone has an idea how we can improve performance? Did I misconfigure >> something? >> >> Regards, >> Auke Noppe >> >> >> >> >> >> -- >> View this message in context: http://activemq.2283324.n4.nab >> ble.com/Network-of-brokers-and-forwarding-performance-tp4725187.html >> Sent from the ActiveMQ - User mailing list archive at Nabble.com. >> >
