Qpid Java Broker - Guidance for 64Bit VMPage added by Marnie McCormackUser Guidance for large heaps using 64Bit VMBackgroundThe Qpid Java Broker's performance and scalability is bound by the availability of heap to hold in flight data and message references particularly for transient only brokers (i.e. those using the MemoryMessageStore). Historically, we have been limited to ~3GB of useable heap memory. Using a 64Bit VM opens this up, with increased addressable memory space. We performed testing on a variety of heap sizes, to provide user guidance on utilising a 64Bit VM. Testing/ResultsHeap Size & Broker PerformanceThe testing looked at two areas performance and cpu usage whilst increasing the heap up to 18GB. Testing has shown that a 6GB heap performs very similarly to a 3GB Heap on the 32Bit VM so if your storage requirements are just above the 3GB limit then there should be no impact to moving to a 6GB 64Bit Heap. Performance/throughput is largely similar. Heap Size & Max MessageThe maximal performance testing was performed with heaps ranging up to 18GB. This allows a 6 fold increase in the amount of transient messages that the broker can hold. Testing with 32KB messages showed that over 180,000 messages can safely be stored using an 18GB heap with in-memory storage (i.e. transient only broker). Depending on your usage pattern and volumes the impact of using a large heap is an approximate drop in performance of 2.5% for every 1GB over 6GB for point to point (single publisher single consumer) and 3.3% for publish & subscribe (single publisher to 5 consumers). For a broker backed with a BDB persistent store (BDBMessageStore) then the number of messagses that can be stored in the broker is bound instead by available disk. We tested up to 500,000 messages in a 256MB heap successfully retained. Heap Size & CPU UtilisationDuring both point to point and publish & subscribe testing the CPU usage scaled linearly with the additional data stored in the broker. Testing has shown that for every extra 6GB approximately one additional core is required for GC processing, housekeeping tasks, etc. So if your current testing shows you need 3 cores then to use a 12GB heap this requirement would increase to 4 cores. Additional Test InfoTesting has been performed on a 64Bit VM with a variety of Xmx values. Testing was performed on a 8-way server with 32GB of RAM using JDK 1.6.0_18(b07).
Change Notification Preferences
View Online
|
Add Comment
|
