Hi Justin Would it be able to reach out to you directly to discuss a couple of points around the discussion below privately ?
Thanks Roy > On 28 Mar 2023, at 10:26, Roy Cohen <roy_co...@hotmail.com> wrote: > > You have indeed ! :) > > On 27 Mar 2023, at 19:35, Justin Bertram <jbert...@apache.org> wrote: > > I wrote that on a completely different thread [1] related to MQTT retained > messages in a cluster. It is not related to this thread or your issue > generally. > > > Justin > > [1] https://lists.apache.org/thread/oq41shfpv108m739km3rhs4tfj76c1zf > > On Mon, Mar 27, 2023 at 1:28 PM Roy Cohen <roy_co...@hotmail.com> wrote: > >> To quote: >> >> “This functionality isn't supported, and while it may be technically >> feasible to implement I'm not sure how much sense it makes overall.” >> >> On 27 Mar 2023, at 19:16, Justin Bertram <jbert...@apache.org> wrote: >> >> I'm not sure where I may have indicated that either one of those things >> isn't supported. >> >> In any case, you can do either. >> >> >> Justin >> >> On Mon, Mar 27, 2023 at 1:07 PM Roy Cohen <roy_co...@hotmail.com> wrote: >> >>> Just to be clear: When you say “isn’t supported” do you mean a third >>> broker or co located backups when running each broker on its own VM ? >>> >>>> On 27 Mar 2023, at 19:04, Roy Cohen <roy_co...@hotmail.com> wrote: >>>> >>>> Will do Justin and many thanks for all the additional details which I >>> will certainly bring forward internally, much appreciated >>>> >>>> On 27 Mar 2023, at 18:58, Justin Bertram <jbert...@apache.org> wrote: >>>> >>>> I recently added a new section to the clustering documentation >> regarding >>>> things to keep in mind regarding performance [1]. >>>> >>>> Also, it's worth noting that often the bottleneck in messaging is not >> the >>>> broker itself but rather the consumer(s). It might be worth ensuring >> that >>>> the bottleneck really is the broker. As noted in the new documentation >>> [1], >>>> adding brokers to a cluster can actually *reduce* throughput in certain >>>> circumstances. >>>> >>>> Let me know if using group-name works for you. >>>> >>>> >>>> Justin >>>> >>>> [1] >>>> >>> >> https://github.com/apache/activemq-artemis/blob/main/docs/user-manual/en/clusters.md#performance-considerations >>>> >>>> On Mon, Mar 27, 2023 at 12:44 PM Roy Cohen <roy_co...@hotmail.com> >>> wrote: >>>> >>>>> I haven’t tried he group-name yet. >>>>> >>>>> With regards to the third broker: The architects believe it’ll improve >>>>> performance given the amount of messages the brokers need to process >> (in >>>>> other words “throw more resources at it…”) >>>>> >>>>> >>>>>> On 27 Mar 2023, at 18:28, Justin Bertram <jbert...@apache.org> wrote: >>>>>> >>>>>>> What would you suggest is to do ? >>>>>> >>>>>> Did you try my previous suggestion already (i.e. using the >> "group-name" >>>>>> element in the "master" or "slave" element of "colocated")? >>>>>> >>>>>> Aside from that, do you know why you were asked to add another broker? >>>>>> Depending on the reason it may not be a good solution. >>>>>> >>>>>> >>>>>> Justin >>>>>> >>>>>> On Mon, Mar 27, 2023 at 12:07 PM Roy Cohen <roy_co...@hotmail.com> >>>>> wrote: >>>>>> >>>>>>> Hi Justin >>>>>>> >>>>>>> It is a good question I honestly don’t have the answer for. I >>> inherited >>>>>>> this configuration and was asked to add a third broker and to ensure >>>>> the co >>>>>>> located backups are being done in such a way that each broker points >>> on >>>>>>> another. Perhaps those who asked for it don’t fully understand >> Artemis >>>>> ! :) >>>>>>> >>>>>>> That said, those co located backup on the existing setup with two >>>>> brokers >>>>>>> do work as we have been enabled to recover lost messages in the past. >>> So >>>>>>> even not optimal, technically it does work ? >>>>>>> >>>>>>> I can only imagine that those who initially designed it about 5 years >>>>> ago >>>>>>> did not use a shared storage to avoid latency. >>>>>>> >>>>>>> What would you suggest is to do ? >>>>>>> >>>>>>> On 27 Mar 2023, at 18:00, Justin Bertram <jbert...@apache.org> >> wrote: >>>>>>> >>>>>>> Your screenshot didn't come through the list. >>>>>>> >>>>>>> In any case, I'm pretty confused at this point. You're clearly using >> a >>>>>>> colocated configuration that will request a backup from another >> broker >>>>> in >>>>>>> the cluster, but you say you're not running multiple brokers in the >>> same >>>>>>> JVM. If you aren't running multiple brokers in the same JVM then what >>>>> are >>>>>>> you using the colocated configuration for? The whole point of the >>>>> colocated >>>>>>> configuration is to run multiple brokers in the same JVM (i.e. a >>> primary >>>>>>> broker and also a backup broker for another primary in the cluster). >>>>>>> >>>>>>> >>>>>>> Justin >>>>>>> >>>>>>> On Mon, Mar 27, 2023 at 11:42 AM Roy Cohen <roy_co...@hotmail.com> >>>>> wrote: >>>>>>> >>>>>>>> I don’t believe we are. >>>>>>>> >>>>>>>> So assume three Virtual Machines on Azure. >>>>>>>> >>>>>>>> Each VM runs one Artemis broker >>>>>>>> >>>>>>>> >>>>>>>> All of their ha policy section on all three brokers look like that: >>>>>>>> >>>>>>>> <ha-policy> >>>>>>>> <replication> >>>>>>>> <colocated> >>>>>>>> <max-backups>1</max-backups> >>>>>>>> <request-backup>true</request-backup> >>>>>>>> >>>>>>>> <backup-request-retry-interval>1000</backup-request-retry-interval> >>>>>>>> <excludes> >>>>>>>> <connector-ref>my-connector</connector-ref> >>>>>>>> <connector-ref>thishostname.mydomain</connector-ref> >>>>>>>> </excludes> >>>>>>>> <master> >>>>>>>> <check-for-live-server>true</check-for-live-server> >>>>>>>> </master> >>>>>>>> <slave> >>>>>>>> <allow-failback>true</allow-failback> >>>>>>>> <restart-backup>true</restart-backup> >>>>>>>> <scale-down/> >>>>>>>> </slave> >>>>>>>> </colocated> >>>>>>>> </replication> >>>>>>>> </ha-policy> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On 27 Mar 2023, at 17:26, Justin Bertram <jbert...@apache.org> >>> wrote: >>>>>>>> >>>>>>>> We are not running multiple brokers on the same JVM but a single >>>>> instance >>>>>>>> >>>>>>>> per VM, so each one has a dedicated JVM and VM >>>>>>>> >>>>>>>> Based on your previous message I was under the impression you were >>>>> using >>>>>>>> the "colocated" feature. *If* you're using this then you definitely >>> are >>>>>>>> running multiple brokers in the same JVM because that's precisely >>> what >>>>>>> that >>>>>>>> feature does. It runs a primary and a backup broker in the *same >>> JVM*. >>>>> If >>>>>>>> you aren't using a "colocated" configuration then I'm not sure what >>> the >>>>>>>> original question is about. Can you clarify? >>>>>>>> >>>>>>>> >>>>>>>> Justin >>>>>>>> >>>>>>>> On Mon, Mar 27, 2023 at 11:07 AM Roy Cohen <roy_co...@hotmail.com> >>>>>>> wrote: >>>>>>>> >>>>>>>> Hi Justin >>>>>>>> >>>>>>>> Thank you for your input. >>>>>>>> >>>>>>>> Sorry, should have been clearer on our setup - We are not running >>>>>>> multiple >>>>>>>> brokers on the same JVM but a single instance per VM, so each one >>> has a >>>>>>>> dedicated JVM and VM >>>>>>>> >>>>>>>> Thanks >>>>>>>> Roy >>>>>>>> >>>>>>>> >>>>>>>> On 27 Mar 2023, at 16:59, Justin Bertram <jbert...@apache.org> >>> wrote: >>>>>>>> >>>>>>>> I'm not entirely sure if the configuration you want is possible. You >>>>>>>> >>>>>>>> might >>>>>>>> >>>>>>>> try using the "group-name" element in the "master" or "slave" >> element >>>>> of >>>>>>>> "colocated." Only servers with the same group-name will pair >>> together. >>>>>>>> >>>>>>>> Aside from that I would actually recommend against using colocated >>>>>>>> >>>>>>>> brokers. >>>>>>>> >>>>>>>> The original use-case for this functionality was very early cloud >>>>>>>> infrastructure where durable, attached storage was not readily >>>>> available. >>>>>>>> However, since then most (if not all) cloud environments support >>>>> durable >>>>>>>> storage separate from the broker so that if the broker goes down a >>> new, >>>>>>>> identical broker can be spun-up relatively quickly and attached to >>> the >>>>>>>> >>>>>>>> same >>>>>>>> >>>>>>>> storage. This provides functional high availability without the need >>>>> for >>>>>>>> any idle backups or replication of any kind which functionally >>>>> nullifies >>>>>>>> this feature. >>>>>>>> >>>>>>>> Additionally, it turns out that (surprise!) configuring & running >>>>>>>> >>>>>>>> multiple >>>>>>>> >>>>>>>> brokers in the same JVM is difficult and error-prone not to mention >>> the >>>>>>>> complication of dynamically coordinating the acquisition of backups >>> in >>>>> a >>>>>>>> running cluster and protecting against split-brain. >>>>>>>> >>>>>>>> >>>>>>>> Justin >>>>>>>> >>>>>>>> On Thu, Mar 23, 2023 at 7:37 AM Roy Cohen <roy_co...@hotmail.com> >>>>> wrote: >>>>>>>> >>>>>>>> Hello everyone >>>>>>>> >>>>>>>> We have a setup of three Artemis brokers (very old version don’t ask >>>>> :)) >>>>>>>> >>>>>>>> We would like to configure the co located backups such that the >>> backups >>>>>>>> are sent in this order: >>>>>>>> >>>>>>>> Broker01 -> Broker02 >>>>>>>> Broker02 -> Broker03 >>>>>>>> Broker03 -> Broker01 >>>>>>>> >>>>>>>> >>>>>>>> I was reading on co located backups here: >>>>>>>> >>>>>>>> >>>>>>> >>>>> >>> >> https://activemq.apache.org/components/artemis/documentation/1.0.0/ha.html >>>>>>>> >>>>>>>> however not sure I fully understand how to configure the xml section >>> to >>>>>>>> achieve that. >>>>>>>> >>>>>>>> Shall I add excludes in each broker, i.e. >>>>>>>> >>>>>>>> <colocated> >>>>>>>> <excludes> >>>>>>>> <connector-ref>...</connector-ref> >>>>>>>> </excludes> >>>>>>>> >>>>>>>> Any help would be appreciated. >>>>>>>> >>>>>>>> Many thanks in advance ! >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>> >>>>> >>> >>> >>
