Sorry, just to add. I could create a kubernetes service for publishers with affinity to 2 of 4 brokers and another service for consumers with affinity to the other 2 but looking for something more dynamic if possible, to be able to scale out seamlessly.
On Thu, 11 Mar 2021 at 13:52, David Martin <dav...@qoritek.com> wrote: > Hi, > > Looking to host an Artemis cluster in Kubernetes and am not sure how to > achieve full local resilience. (Clusters for DR and remote distribution > will be added later using the mirroring feature introduced with v2.16). > > It is configured as 3 active cluster members using static discovery > because the particular cloud provider does not officially support UDP on > its managed Kubernetes service network. > > There are no backup brokers (active/passive) because the stateful set > takes care of restarting failed pods immediately. > > Each broker has its own networked storage so is resilient in terms of > local state. > > Message redistribution is ON_DEMAND. Publishing is to topics and consuming > is from durable topic subscription queues. > > Publishers and consumers are connecting round-robin with client IP > affinity/stickiness. > > What I'm concerned about is the possibility of journal corruption on one > broker. Publishers and consumers will failover to either of the remaining 2 > brokers which is fine but some data could be lost permanently as follows. > > Hypothetically, consider that Publisher 1 is publishing to Broker 1 and > Publisher 2 is publishing to Broker 3. Consumer 1 is consuming from Broker > 2 and Consumer 2 is consuming from Broker 1. There are more consumers and > publishers but using 2 of each just to illustrate. > > Publisher 1 -> Broker 1 -> Broker 2 -> Consumer 1 > Publisher 2 -> Broker 3 -> Broker 2 -> Consumer 1 > Publisher 1 -> Broker 1 -> Consumer 2 > Publisher 2 -> Broker 3 -> Broker 1 -> Consumer 2 > > This all works very well with full data integrity and good performance :) > > However if say Broker 1's journal got corrupted and it went down > permanently as a result, any data from Publisher 1 which hadn't yet been > distributed to Consumer 1 (via Broker 2) or *particularly* Consumer 2 > (directly) would be lost (unless the journal could be recovered). > > Is there some straightforward configuration to avoid or reduce this > possibility? Perhaps a 4 broker cluster could have affinity for publishers > on 2 brokers and affinity for consumers on the other 2, somehow? > > > Thanks for any advice you can offer. > > > Dave Martin. > > >
