I'm interested in a variation of this for load balancing. Assume the volume of data coming from publishers is very large, and you need N brokers (think N is 10 or more). Furthermore, you want N+1 or N+2 redundancy so that if any 1 or 2 brokers die, there is sufficient capacity to continue handling all of the traffic (after affected publishers reconnect). The brokers might all be behind a hardware load balancer (e.g. F5 or NetScaler) so that all publishers connect via a virtual IP.
Now, from the consumer side, we also have many consumers. Each consumer only subscribes to a subset of the available topics, and we arrange via design of our topics such that one consumer can always keep up with the volume of data published on one topic. But the consumer must connect to every broker. So, we need a fan-in variation of a failover transport that connects to every broker and actively consumes data from all brokers. If any broker disconnects, the consumer would use the exponential backoff reconnect logic to reconnect when the broker becomes available. So, the backup=true option isn't helpful. Instead, we want something like "fan_in=true". It's not hard to build this kind of fan in logic on top of the failover transport, but it would be cool if the failover transport was capable of doing fan in directly. On Thu, Jan 29, 2009 at 8:13 AM, Gary Tully <gary.tu...@gmail.com> wrote: > have you looked at the failover transport? see: > http://activemq.apache.org/failover-transport-reference.html > with the backup=true option, a connection to all listed brokers will > be created so that they are in hot standby in > the event that the first connection is dropped. In this way, failover > can be very fast. > > 2009/1/29 kaykay <kaykay.uni...@gmail.com>: > > > > This thread is old but just curious if there has been a recent update of > this > > w.r.t ActiveMQ 5.2 . The problem that I am trying to solve is similar > where > > a consumer listens to multiple brokers (as a failover redundancy issue > > instead of listening to a single broker). > > > > > > Stepan Koltsov wrote: > >> > >> I'm playing with 5.0. > >> > >> How do you think, is it hard to write new Transport that consumes from > >> multiple brokers? > >> > >> S. > >> > >> > >> ttmdev wrote: > >>> > >>> Yes, I got similar results in my consumer testing. What version of AMQ > >>> are you using? > >>> > >>> Your only recourse may be to multi thread your consumer and have it > >>> create a connection to each of the brokers. > >>> > >> > >> > > > > -- > > View this message in context: > http://www.nabble.com/Single-consumer-for-multiple-brokers--tp15768836p21729358.html > > Sent from the ActiveMQ - User mailing list archive at Nabble.com. > > > > > > > > -- > http://blog.garytully.com > > Open Source SOA > http://FUSESource.com >
