Understood and yes there’s a reason. Don’t I need @apache.org email in order to join that Slack workspace ?
> On 28 Mar 2023, at 15:52, Justin Bertram <jbert...@apache.org> wrote: > > Typically we like to keep everything on the list so the whole community can > benefit from the discussion. However, if there's a specific reason that > privacy is a concern then you can email me directly or you can find me on > the ASF Slack in #activemq. > > > Justin > > On Tue, Mar 28, 2023 at 7:56 AM Roy Cohen <roy_co...@hotmail.com> wrote: > >> Hi Justin >> >> Would it be able to reach out to you directly to discuss a couple of >> points around the discussion below privately ? >> >> Thanks >> Roy >> >> >>> On 28 Mar 2023, at 10:26, Roy Cohen <roy_co...@hotmail.com> wrote: >>> >>> You have indeed ! :) >>> >>> On 27 Mar 2023, at 19:35, Justin Bertram <jbert...@apache.org> wrote: >>> >>> I wrote that on a completely different thread [1] related to MQTT >> retained >>> messages in a cluster. It is not related to this thread or your issue >>> generally. >>> >>> >>> Justin >>> >>> [1] https://lists.apache.org/thread/oq41shfpv108m739km3rhs4tfj76c1zf >>> >>> On Mon, Mar 27, 2023 at 1:28 PM Roy Cohen <roy_co...@hotmail.com> wrote: >>> >>>> To quote: >>>> >>>> “This functionality isn't supported, and while it may be technically >>>> feasible to implement I'm not sure how much sense it makes overall.” >>>> >>>> On 27 Mar 2023, at 19:16, Justin Bertram <jbert...@apache.org> wrote: >>>> >>>> I'm not sure where I may have indicated that either one of those things >>>> isn't supported. >>>> >>>> In any case, you can do either. >>>> >>>> >>>> Justin >>>> >>>> On Mon, Mar 27, 2023 at 1:07 PM Roy Cohen <roy_co...@hotmail.com> >> wrote: >>>> >>>>> Just to be clear: When you say “isn’t supported” do you mean a third >>>>> broker or co located backups when running each broker on its own VM ? >>>>> >>>>>> On 27 Mar 2023, at 19:04, Roy Cohen <roy_co...@hotmail.com> wrote: >>>>>> >>>>>> Will do Justin and many thanks for all the additional details which I >>>>> will certainly bring forward internally, much appreciated >>>>>> >>>>>> On 27 Mar 2023, at 18:58, Justin Bertram <jbert...@apache.org> wrote: >>>>>> >>>>>> I recently added a new section to the clustering documentation >>>> regarding >>>>>> things to keep in mind regarding performance [1]. >>>>>> >>>>>> Also, it's worth noting that often the bottleneck in messaging is not >>>> the >>>>>> broker itself but rather the consumer(s). It might be worth ensuring >>>> that >>>>>> the bottleneck really is the broker. As noted in the new documentation >>>>> [1], >>>>>> adding brokers to a cluster can actually *reduce* throughput in >> certain >>>>>> circumstances. >>>>>> >>>>>> Let me know if using group-name works for you. >>>>>> >>>>>> >>>>>> Justin >>>>>> >>>>>> [1] >>>>>> >>>>> >>>> >> https://github.com/apache/activemq-artemis/blob/main/docs/user-manual/en/clusters.md#performance-considerations >>>>>> >>>>>> On Mon, Mar 27, 2023 at 12:44 PM Roy Cohen <roy_co...@hotmail.com> >>>>> wrote: >>>>>> >>>>>>> I haven’t tried he group-name yet. >>>>>>> >>>>>>> With regards to the third broker: The architects believe it’ll >> improve >>>>>>> performance given the amount of messages the brokers need to process >>>> (in >>>>>>> other words “throw more resources at it…”) >>>>>>> >>>>>>> >>>>>>>> On 27 Mar 2023, at 18:28, Justin Bertram <jbert...@apache.org> >> wrote: >>>>>>>> >>>>>>>>> What would you suggest is to do ? >>>>>>>> >>>>>>>> Did you try my previous suggestion already (i.e. using the >>>> "group-name" >>>>>>>> element in the "master" or "slave" element of "colocated")? >>>>>>>> >>>>>>>> Aside from that, do you know why you were asked to add another >> broker? >>>>>>>> Depending on the reason it may not be a good solution. >>>>>>>> >>>>>>>> >>>>>>>> Justin >>>>>>>> >>>>>>>> On Mon, Mar 27, 2023 at 12:07 PM Roy Cohen <roy_co...@hotmail.com> >>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi Justin >>>>>>>>> >>>>>>>>> It is a good question I honestly don’t have the answer for. I >>>>> inherited >>>>>>>>> this configuration and was asked to add a third broker and to >> ensure >>>>>>> the co >>>>>>>>> located backups are being done in such a way that each broker >> points >>>>> on >>>>>>>>> another. Perhaps those who asked for it don’t fully understand >>>> Artemis >>>>>>> ! :) >>>>>>>>> >>>>>>>>> That said, those co located backup on the existing setup with two >>>>>>> brokers >>>>>>>>> do work as we have been enabled to recover lost messages in the >> past. >>>>> So >>>>>>>>> even not optimal, technically it does work ? >>>>>>>>> >>>>>>>>> I can only imagine that those who initially designed it about 5 >> years >>>>>>> ago >>>>>>>>> did not use a shared storage to avoid latency. >>>>>>>>> >>>>>>>>> What would you suggest is to do ? >>>>>>>>> >>>>>>>>> On 27 Mar 2023, at 18:00, Justin Bertram <jbert...@apache.org> >>>> wrote: >>>>>>>>> >>>>>>>>> Your screenshot didn't come through the list. >>>>>>>>> >>>>>>>>> In any case, I'm pretty confused at this point. You're clearly >> using >>>> a >>>>>>>>> colocated configuration that will request a backup from another >>>> broker >>>>>>> in >>>>>>>>> the cluster, but you say you're not running multiple brokers in the >>>>> same >>>>>>>>> JVM. If you aren't running multiple brokers in the same JVM then >> what >>>>>>> are >>>>>>>>> you using the colocated configuration for? The whole point of the >>>>>>> colocated >>>>>>>>> configuration is to run multiple brokers in the same JVM (i.e. a >>>>> primary >>>>>>>>> broker and also a backup broker for another primary in the >> cluster). >>>>>>>>> >>>>>>>>> >>>>>>>>> Justin >>>>>>>>> >>>>>>>>> On Mon, Mar 27, 2023 at 11:42 AM Roy Cohen <roy_co...@hotmail.com> >>>>>>> wrote: >>>>>>>>> >>>>>>>>>> I don’t believe we are. >>>>>>>>>> >>>>>>>>>> So assume three Virtual Machines on Azure. >>>>>>>>>> >>>>>>>>>> Each VM runs one Artemis broker >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> All of their ha policy section on all three brokers look like >> that: >>>>>>>>>> >>>>>>>>>> <ha-policy> >>>>>>>>>> <replication> >>>>>>>>>> <colocated> >>>>>>>>>> <max-backups>1</max-backups> >>>>>>>>>> <request-backup>true</request-backup> >>>>>>>>>> >>>>>>>>>> >> <backup-request-retry-interval>1000</backup-request-retry-interval> >>>>>>>>>> <excludes> >>>>>>>>>> <connector-ref>my-connector</connector-ref> >>>>>>>>>> <connector-ref>thishostname.mydomain</connector-ref> >>>>>>>>>> </excludes> >>>>>>>>>> <master> >>>>>>>>>> <check-for-live-server>true</check-for-live-server> >>>>>>>>>> </master> >>>>>>>>>> <slave> >>>>>>>>>> <allow-failback>true</allow-failback> >>>>>>>>>> <restart-backup>true</restart-backup> >>>>>>>>>> <scale-down/> >>>>>>>>>> </slave> >>>>>>>>>> </colocated> >>>>>>>>>> </replication> >>>>>>>>>> </ha-policy> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 27 Mar 2023, at 17:26, Justin Bertram <jbert...@apache.org> >>>>> wrote: >>>>>>>>>> >>>>>>>>>> We are not running multiple brokers on the same JVM but a single >>>>>>> instance >>>>>>>>>> >>>>>>>>>> per VM, so each one has a dedicated JVM and VM >>>>>>>>>> >>>>>>>>>> Based on your previous message I was under the impression you were >>>>>>> using >>>>>>>>>> the "colocated" feature. *If* you're using this then you >> definitely >>>>> are >>>>>>>>>> running multiple brokers in the same JVM because that's precisely >>>>> what >>>>>>>>> that >>>>>>>>>> feature does. It runs a primary and a backup broker in the *same >>>>> JVM*. >>>>>>> If >>>>>>>>>> you aren't using a "colocated" configuration then I'm not sure >> what >>>>> the >>>>>>>>>> original question is about. Can you clarify? >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Justin >>>>>>>>>> >>>>>>>>>> On Mon, Mar 27, 2023 at 11:07 AM Roy Cohen <roy_co...@hotmail.com >>> >>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> Hi Justin >>>>>>>>>> >>>>>>>>>> Thank you for your input. >>>>>>>>>> >>>>>>>>>> Sorry, should have been clearer on our setup - We are not running >>>>>>>>> multiple >>>>>>>>>> brokers on the same JVM but a single instance per VM, so each one >>>>> has a >>>>>>>>>> dedicated JVM and VM >>>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> Roy >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 27 Mar 2023, at 16:59, Justin Bertram <jbert...@apache.org> >>>>> wrote: >>>>>>>>>> >>>>>>>>>> I'm not entirely sure if the configuration you want is possible. >> You >>>>>>>>>> >>>>>>>>>> might >>>>>>>>>> >>>>>>>>>> try using the "group-name" element in the "master" or "slave" >>>> element >>>>>>> of >>>>>>>>>> "colocated." Only servers with the same group-name will pair >>>>> together. >>>>>>>>>> >>>>>>>>>> Aside from that I would actually recommend against using colocated >>>>>>>>>> >>>>>>>>>> brokers. >>>>>>>>>> >>>>>>>>>> The original use-case for this functionality was very early cloud >>>>>>>>>> infrastructure where durable, attached storage was not readily >>>>>>> available. >>>>>>>>>> However, since then most (if not all) cloud environments support >>>>>>> durable >>>>>>>>>> storage separate from the broker so that if the broker goes down a >>>>> new, >>>>>>>>>> identical broker can be spun-up relatively quickly and attached to >>>>> the >>>>>>>>>> >>>>>>>>>> same >>>>>>>>>> >>>>>>>>>> storage. This provides functional high availability without the >> need >>>>>>> for >>>>>>>>>> any idle backups or replication of any kind which functionally >>>>>>> nullifies >>>>>>>>>> this feature. >>>>>>>>>> >>>>>>>>>> Additionally, it turns out that (surprise!) configuring & running >>>>>>>>>> >>>>>>>>>> multiple >>>>>>>>>> >>>>>>>>>> brokers in the same JVM is difficult and error-prone not to >> mention >>>>> the >>>>>>>>>> complication of dynamically coordinating the acquisition of >> backups >>>>> in >>>>>>> a >>>>>>>>>> running cluster and protecting against split-brain. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Justin >>>>>>>>>> >>>>>>>>>> On Thu, Mar 23, 2023 at 7:37 AM Roy Cohen <roy_co...@hotmail.com> >>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> Hello everyone >>>>>>>>>> >>>>>>>>>> We have a setup of three Artemis brokers (very old version don’t >> ask >>>>>>> :)) >>>>>>>>>> >>>>>>>>>> We would like to configure the co located backups such that the >>>>> backups >>>>>>>>>> are sent in this order: >>>>>>>>>> >>>>>>>>>> Broker01 -> Broker02 >>>>>>>>>> Broker02 -> Broker03 >>>>>>>>>> Broker03 -> Broker01 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> I was reading on co located backups here: >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>> >>>>> >>>> >> https://activemq.apache.org/components/artemis/documentation/1.0.0/ha.html >>>>>>>>>> >>>>>>>>>> however not sure I fully understand how to configure the xml >> section >>>>> to >>>>>>>>>> achieve that. >>>>>>>>>> >>>>>>>>>> Shall I add excludes in each broker, i.e. >>>>>>>>>> >>>>>>>>>> <colocated> >>>>>>>>>> <excludes> >>>>>>>>>> <connector-ref>...</connector-ref> >>>>>>>>>> </excludes> >>>>>>>>>> >>>>>>>>>> Any help would be appreciated. >>>>>>>>>> >>>>>>>>>> Many thanks in advance ! >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>> >>>>>>> >>>>> >>>>> >>>> >> >>
