On 07/17/2013 10:17 AM, Steve Huston wrote:
Hi Steve,
-----Original Message-----
From: Rothkin, Steve (NY81) [mailto:steve.roth...@honeywell.com]
Sent: Wednesday, July 17, 2013 8:57 AM
To: users@qpid.apache.org
Subject: Queue mirroring: clustering vs. replication
Subject: RE: Queue mirroring with message grouping and clustering on
Windows
I'm considering a Qpid.22 implementation under MS Windows for
message queueing. In the future we might go to a mixed
environment with both Windows and Linux computers.
For fault tolerance, I want the queues to be mirrored across 2
to
3 computers which are connected by high speed LAN. Each queue
will have multiple consumers on different computers (including
some NOT hosting the queue), so I also need to use the message
grouping feature to ensure that messages from a single source
are not processed
out of order.
What are the differences between choosing to use clustering to implement
the solution vs. using ha-replication of individual queues in a non-clustered
setup?
You summarized it well below, but also, essentially, reusing the capabilities
already present, or reinventing them to suit the particulars of your use case.
From what I've read so far it seems that with clustering my work would be a
lot easier as the broker/resource manager combination would take care of
failover while with non-clustered I'd have to take care of a bunch of things in
my code -- designating master, detecting failure (which might take longer),
failover (including finding and connecting to the new master).
Right. Don't forget to account for all the ways things can fail. Brokers,
nodes, network links, etc.
On the other hand it seems that without clustering, I could pick different
masters for different queues (instead of having cluster resource manager
designate a specific host as master for all of its queues) which would be one
way of spreading the load of handling client connections.
You could also run multiple clustered brokers on the same nodes and have
broker-cluster-1 serve one set of queues and broker-cluster-2 serve a different
set. I'm not sure about the automatability of keeping the primaries on
different nodes, but you could write a script to check it and move the primary
around if needed.
I could also have
different operating systems hosting replicas of the same queue (I haven't
found a cluster resource manager that runs on both Windows and Linux).
That's a great point - clusters are generally uniform.
I'm also wondering about geographic site mirroring and failover. We have 2
sites connected by a WAN. The same services are available at both locations.
Normally a request received at one location would be processed there (and
sending it to the other site for processing would be more "expensive" as
would queue mirroring to the other site). But there might be situations
where one entire site goes down -- if that were to happen it might be good
to have the other site detect and finish processing any in-flight messages
from the down site.
There are more factors in this type of design than I'd want to try and cover in
email. I'm working with a customer now that is starting to run tests on a
multi-site environment, but their goals and constraints and resources may be
quite different from yours. I think you've got a good grasp of the facilities
available (queue dup, federation, clustering) but if you want further in-depth
help, please let me know.
-Steve
Steve's done a great job with the clustering facts, I don't have much to add. If
you get interested in integrating another resource manager let me know, I'd like
to help. I need to do an integration with pacemaker in the near future. Also any
other issues you have, you can discuss them on this list or raise a JIRA if you
have a specific bug that needs fixing.
Cheers,
Alan.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org
