[CONF] Apache Qpid > New Page

[confluence]

New Page Page edited by Martin Ritchie Purpose This page is intended to outline the known use cases for running multiple Java Brokers, addressing logged issues and limitations of the current implementation (as of V0.5). It is not about clustering proper. Use Cases High Volume Transient Broker Description The key characteristics of this application scenario are: Broker availability is key Traffic is pub-sub Message Order is not important This use case relates to applications with a high residual message load i.e. where message data on the broker remains in memory for some time or consumption lags production such that a backlog is constantly present in the broker queues. This paradigm is reasonably common, partly because publication threads are generally handling only the simple publish call where we often see consumption threads handling writes to RDBMS or other time expensive processing. Thus a rate gap opens up, and creates a data tailback. In this scenario, particularly for deployments on a 32bit VM, the broker can exhaust a 3GB heap or start to perform poorly as it approaches max heap. Result Broker side OoM or performance degradation requiring bounce. Messages in flight not processed, client applications experience connection loss. Possible Solution A In this scenario, it would be possible to use 2 brokers. Publishing clients would use 2 connections and publish alternately to each broker, providing a simplistic load balancing solution. Consuming clients would then consume from the 2 brokers, using the same topic name etc. The consumer could choose to consume in parallel, thus potentially speeding up processing time or by taking messages singly from the two sources alternately. Possible Solution B * Again, using 2 brokers but this time working with the assumption that the application data flows across the broker can be separated by source/destination. An easy solution for this is to simply divide the required traffic by source or destination and put an amount on each broker. This may necessitate multiple consuming connections (to each of the brokers) on the client side where there are multiple sources feeding the same client. Alternatively, for some applications, the clients can be segmented in pairs of publisher-consumer by flow. Possible Solution C * Notes * Note that encouraging applications to use round robin consumers in this paradigm may also pay dividends, particularly where a secondart write can be run in parallel i.e. inserts to an RDBMS are not mutually exclusive. Low Volume Persistent Broker Description Result Possible Solution Change Notification Preferences View Online | View Change | Add Comment --------------------------------------------------------------------- Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:commits-subscr...@qpid.apache.org