Will do Justin and many thanks for all the additional details which I will certainly bring forward internally, much appreciated
On 27 Mar 2023, at 18:58, Justin Bertram <jbert...@apache.org> wrote: I recently added a new section to the clustering documentation regarding things to keep in mind regarding performance [1]. Also, it's worth noting that often the bottleneck in messaging is not the broker itself but rather the consumer(s). It might be worth ensuring that the bottleneck really is the broker. As noted in the new documentation [1], adding brokers to a cluster can actually *reduce* throughput in certain circumstances. Let me know if using group-name works for you. Justin [1] https://github.com/apache/activemq-artemis/blob/main/docs/user-manual/en/clusters.md#performance-considerations On Mon, Mar 27, 2023 at 12:44 PM Roy Cohen <roy_co...@hotmail.com> wrote: > I haven’t tried he group-name yet. > > With regards to the third broker: The architects believe it’ll improve > performance given the amount of messages the brokers need to process (in > other words “throw more resources at it…”) > > >> On 27 Mar 2023, at 18:28, Justin Bertram <jbert...@apache.org> wrote: >> >>> What would you suggest is to do ? >> >> Did you try my previous suggestion already (i.e. using the "group-name" >> element in the "master" or "slave" element of "colocated")? >> >> Aside from that, do you know why you were asked to add another broker? >> Depending on the reason it may not be a good solution. >> >> >> Justin >> >> On Mon, Mar 27, 2023 at 12:07 PM Roy Cohen <roy_co...@hotmail.com> > wrote: >> >>> Hi Justin >>> >>> It is a good question I honestly don’t have the answer for. I inherited >>> this configuration and was asked to add a third broker and to ensure > the co >>> located backups are being done in such a way that each broker points on >>> another. Perhaps those who asked for it don’t fully understand Artemis > ! :) >>> >>> That said, those co located backup on the existing setup with two > brokers >>> do work as we have been enabled to recover lost messages in the past. So >>> even not optimal, technically it does work ? >>> >>> I can only imagine that those who initially designed it about 5 years > ago >>> did not use a shared storage to avoid latency. >>> >>> What would you suggest is to do ? >>> >>> On 27 Mar 2023, at 18:00, Justin Bertram <jbert...@apache.org> wrote: >>> >>> Your screenshot didn't come through the list. >>> >>> In any case, I'm pretty confused at this point. You're clearly using a >>> colocated configuration that will request a backup from another broker > in >>> the cluster, but you say you're not running multiple brokers in the same >>> JVM. If you aren't running multiple brokers in the same JVM then what > are >>> you using the colocated configuration for? The whole point of the > colocated >>> configuration is to run multiple brokers in the same JVM (i.e. a primary >>> broker and also a backup broker for another primary in the cluster). >>> >>> >>> Justin >>> >>> On Mon, Mar 27, 2023 at 11:42 AM Roy Cohen <roy_co...@hotmail.com> > wrote: >>> >>>> I don’t believe we are. >>>> >>>> So assume three Virtual Machines on Azure. >>>> >>>> Each VM runs one Artemis broker >>>> >>>> >>>> All of their ha policy section on all three brokers look like that: >>>> >>>> <ha-policy> >>>> <replication> >>>> <colocated> >>>> <max-backups>1</max-backups> >>>> <request-backup>true</request-backup> >>>> >>>> <backup-request-retry-interval>1000</backup-request-retry-interval> >>>> <excludes> >>>> <connector-ref>my-connector</connector-ref> >>>> <connector-ref>thishostname.mydomain</connector-ref> >>>> </excludes> >>>> <master> >>>> <check-for-live-server>true</check-for-live-server> >>>> </master> >>>> <slave> >>>> <allow-failback>true</allow-failback> >>>> <restart-backup>true</restart-backup> >>>> <scale-down/> >>>> </slave> >>>> </colocated> >>>> </replication> >>>> </ha-policy> >>>> >>>> >>>> >>>> >>>> On 27 Mar 2023, at 17:26, Justin Bertram <jbert...@apache.org> wrote: >>>> >>>> We are not running multiple brokers on the same JVM but a single > instance >>>> >>>> per VM, so each one has a dedicated JVM and VM >>>> >>>> Based on your previous message I was under the impression you were > using >>>> the "colocated" feature. *If* you're using this then you definitely are >>>> running multiple brokers in the same JVM because that's precisely what >>> that >>>> feature does. It runs a primary and a backup broker in the *same JVM*. > If >>>> you aren't using a "colocated" configuration then I'm not sure what the >>>> original question is about. Can you clarify? >>>> >>>> >>>> Justin >>>> >>>> On Mon, Mar 27, 2023 at 11:07 AM Roy Cohen <roy_co...@hotmail.com> >>> wrote: >>>> >>>> Hi Justin >>>> >>>> Thank you for your input. >>>> >>>> Sorry, should have been clearer on our setup - We are not running >>> multiple >>>> brokers on the same JVM but a single instance per VM, so each one has a >>>> dedicated JVM and VM >>>> >>>> Thanks >>>> Roy >>>> >>>> >>>> On 27 Mar 2023, at 16:59, Justin Bertram <jbert...@apache.org> wrote: >>>> >>>> I'm not entirely sure if the configuration you want is possible. You >>>> >>>> might >>>> >>>> try using the "group-name" element in the "master" or "slave" element > of >>>> "colocated." Only servers with the same group-name will pair together. >>>> >>>> Aside from that I would actually recommend against using colocated >>>> >>>> brokers. >>>> >>>> The original use-case for this functionality was very early cloud >>>> infrastructure where durable, attached storage was not readily > available. >>>> However, since then most (if not all) cloud environments support > durable >>>> storage separate from the broker so that if the broker goes down a new, >>>> identical broker can be spun-up relatively quickly and attached to the >>>> >>>> same >>>> >>>> storage. This provides functional high availability without the need > for >>>> any idle backups or replication of any kind which functionally > nullifies >>>> this feature. >>>> >>>> Additionally, it turns out that (surprise!) configuring & running >>>> >>>> multiple >>>> >>>> brokers in the same JVM is difficult and error-prone not to mention the >>>> complication of dynamically coordinating the acquisition of backups in > a >>>> running cluster and protecting against split-brain. >>>> >>>> >>>> Justin >>>> >>>> On Thu, Mar 23, 2023 at 7:37 AM Roy Cohen <roy_co...@hotmail.com> > wrote: >>>> >>>> Hello everyone >>>> >>>> We have a setup of three Artemis brokers (very old version don’t ask > :)) >>>> >>>> We would like to configure the co located backups such that the backups >>>> are sent in this order: >>>> >>>> Broker01 -> Broker02 >>>> Broker02 -> Broker03 >>>> Broker03 -> Broker01 >>>> >>>> >>>> I was reading on co located backups here: >>>> >>>> >>> > https://activemq.apache.org/components/artemis/documentation/1.0.0/ha.html >>>> >>>> however not sure I fully understand how to configure the xml section to >>>> achieve that. >>>> >>>> Shall I add excludes in each broker, i.e. >>>> >>>> <colocated> >>>> <excludes> >>>> <connector-ref>...</connector-ref> >>>> </excludes> >>>> >>>> Any help would be appreciated. >>>> >>>> Many thanks in advance ! >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>> > >
