Your understanding of this is good and you've got the right concepts here. The only two reasons I can think of that you shouldn't use a mesh (which I think is the same as a complete graph, though you drew a distinction between the two in your email so maybe there's a difference I'm not understanding) is if you have non-homogeneous network links where the most efficient routing between two brokers isn't necessarily the path "directly" (from a network-of-brokers standpoint, not from a network standpoint) between the two, or if you don't want to have to specify all of the other clusters so you'd use a hub-and-spoke topology so you only have to specify the hub node in your config.
On Fri, Mar 13, 2015 at 12:21 AM, James A. Robinson <jim.robin...@gmail.com> wrote: > Hi folks, > > I joined this list because we're starting to look into setting up > ActiveMQ for our developers. My preference at this time is to set > up a highly reliable system that doesn't have single points of > failure. > > Due to the concern about single points of failure I didn't want to > introduce a shared persistence layer. That's why I've been looking > at replicated LevelDB. > > After spending some time reading up on ActiveMQ my first inclination > has been to propose setting up 3-node replicated LevelDB clusters, > and to build a network of brokers out of sets of those 3-node > clusters. > > E.g., start with two clusters, and then keep adding on as we see > the need in terms of capacity. > > [amq-1a, amq-1b, amq-1c] > [amq-2a, amq-2b, amq-2c] > ... > [amq-9a, amq-9b, amq-9c] > ... > > I'm not sure what the best option is w/re to the network of broker > topology. I could see mesh working, but my first instinct would > be to set up a complete graph, and have each broker cluster configured > with a uni-directional network connector to every other cluster > using the masterslave scheme. > > I am wondering if this is a naive approach, and if so why it wouldn't > be a good idea. > > If it is reasonable, I'm still trying to fully understand how one > deals with forwarding messages from brokers w/o consumers to those > brokers w/ consumers. The page > > http://activemq.apache.org/networks-of-brokers.html > > discusses this topic and mentions rebalanceClusterClients, and it > isn't clear to me how that option will help if the clusters and > clients are fairly static: > > rebalanceClusterClients: > if true, connected clients will be asked to rebalance across a > cluster of brokers when a new broker joins the network of brokers > (note: priorityBackup=true can override) > > Reading the list of options available for transport connectors, it > seems reasonable to configure the transport connector with: > > updateClusterClients="true" > rebalanceClusterClients="true" > updateClusterClientsOnRemove="true" > > But if one has mostly static clients and we aren't frequently > bringing broker clusters up and down, I would assume there would > still be a problem of having messages on a broker that doesn't have > consumers attached. If I'm wrong I'd like to understand what I'm > missing. > > The network-of-brokers page, after mentioning rebalanceClusterClients > as one solution, continues the discussion and states that: > > Another, is to allow replay of messages back to the origin as > there is no local consumer on that broker. > > There is a destination policy that allows this behavior for queues > by configuring a conditionalNetworkBridgeFilterFactory with > replayWhenNoConsumers=true. The conditionalNetworkBridgeFilterFactory > provides an optional replayDelay based on the broker-in time. > > N.B.: When using replayWhenNoConsumers=true for versions < 5.9, > it is necessary to also disable the cursors duplicate detection > using enableAudit=false as the cursor could mark the replayed > messages as duplicates (depending on the time window between > playing and replaying these messages over the network bridge). > The problem is fully explained in this blog post. > > And I see in that ticket AMQ-4607 says this about the network > connector options: > > networkTTL sets both consumerTTL and messageTTL a value of -1 > denotes infinite hops. > In a mesh, use consumerTTL=1, messageTTL=-1 to allow consumers to > bounce around the mesh and messages to follow demand. > In a linear topology use networkTTL == length of network (as before) > > So would it be reasonable to set up a complete graph, with each > 3-node cluster configured with a uni-directional network connector > to every other 3-node cluster using masterslave, and to set > > consumerTTL="1" > messageTTL="-1" > decreaseNetworkConsumerPriority="true" > > and a networkBridgeFilterFactory/conditionalNetworkBridgeFilterFactory > with > > replayWhenNoConsumers="true" > > I'd appreciate any input from the folks here, including any pointers > on any more material I should read. > > Jim >
