Hi all, It's been quite a struggle to get messages from one queue to a queue on another broker. The simplest but also most restrictive way is to use qpid-route queue add <dest> <src> <queue> however this is simply a point-to-point solution. and setting it up with --durable requires much more and often results in error..
Message based service oriented architecture requires services (message consumers) to be fed based on specific routing rules or simply by having each service type have it's own queue. A robust message bus will need a local (yes, localhost..) broker that will be federated with brokers on other nodes. Both clients and services will need local queues that usually keep persistent messages. Thus ensuring that network issues will not cause failure of either service or client. a simple setup of three nodes N1,N2 and N3 would require: client C1,C2 on N1 service S1,S2 on N2 service S1,S2 on N3 clients C1 and C2 (on N1) will request service S1 or S2 for a specific function by sending a message to local (on N1) queue 'QS1' or 'QS2'. Routing will make sure messages from queue 'QS1' (on N1) will be forwarded to queue 'QS1' on either N2 or N3 depending on load and availability of service S1 on N2 and N3. How should this work in the broker? QS1 on N1 holds messages sent by C1 and C2. QS1 on N2 and QS1 N3 will receive messages from QS1 on N1 if there is a consumer on their queue. messages should only be sent to either N2 or N3 to ensure no messages are duplicated. (load balancing) This will allow dynamical upscaling of services on N2 and N3 and a future N4 or N5.. Currently there are no options to configure this kind of setup unless clustering is used and that is a really 'expensive' setup where all queues are replicated everywhere and network disruptions can cause message duplication. There was a proposal to allow topic exchanges to have a load balance option that would implement a similar scheme. However this is not part of QPID at this time. (see: https://issues.apache.org/jira/browse/QPID-4077 ) If the proposed add-on feature 4077 isn't necessary then how can we implement the described scenario with qpid? Also the target environment will be many brokers spanning multiple large networks.. Any ideas? Best regards, Jan Rinze Peterzon.
