Hi Rob, Thanks, I'll raise a JIRA for this with my changes then.
Helen On Tue, May 7, 2013 at 12:18 AM, Rob Godfrey <[email protected]>wrote: > Hi Helen, > > thanks for taking the time to investigate this. Your suggestions sound very > sensible, and it would be good to get them reviewed and incorporated as > soon as possible. Would it be possible for you to raise a JIRA [1] for > this and attach your changes as a patch file? - that will be the quickest > way for us to be able to get these improvements applied. > > Many thanks for your efforts, > Rob > > > > [1] https://issues.apache.org/jira/browse/QPID > > > On 7 May 2013 03:10, Helen Kwong <[email protected]> wrote: > > > Hello, > > > > My team is using the Java broker and Java client, version 0.16, and > looking > > to lower the client's memory footprint on our servers. We did some heap > > analysis and found that the consumption is coming mostly from > > AMQAnyDestination instances, each having a retained size of 2864B, and > > since we have 6000 queues on each of our 2 brokers, this amounts to about > > ~33MB, which is valuable real estate for us. In our analysis we found a > few > > possible optimizations in the Qpid code that would reduce the per-queue > > heap consumption and which don't seem high risk, and were wondering how > > feasible these changes are. > > > > The first optimization involves cacheing / resuing redundant maps, while > > the rest involves making the default value of a variable the constant > empty > > Map or List, instead of a new empty Map or List. > > > > > > 1. In Address / AddressParser, cacheing / reusing the options Maps for > > queues created with the same options string. (This optimization gives us > > the most significant savings, but has the highest risk.) > > > > All our queues are created with the same options string, which means each > > corresponding AMQDestination has an Address that has an _options Map that > > is the same for all queues, i.e., 12K copies of the same map. As far as > we > > can tell, the _options map is effectively immutable, i.e., there is no > code > > path by which an Address’s _options map can be modified. (Is this > correct?) > > So a possible improvement is that in > > org.apache.qpid.messaging.util.AddressParser, we cache the options map > for > > each options string that we've already encountered, and if the options > > string passed in has already been seen, we use the stored options map for > > that Address. This way, for queues having the same options, their Address > > options will reference the same Map. (For our queues, each Address > _options > > Map takes up 1384 B, so with 12K queues we'd save about 15.5MB.) > > > > > > 2. AMQDestination's _link field -- > > org.apache.qpid.client.messaging.address.Link$Subscription > > > > Subscription's args field is by default a new HashMap with default > capacity > > 16. In our use case it remains empty for all queues. A possible > > optimization is to set the default value as Collections.emptyMap() > instead. > > As far was we can tell, Subscription.getArgs() is not used to get the map > > and then modify it. For us this saves 128B per queue. > > > > > > 3. AMQDestination's _targetNode field -- > > org.apache.qpid.client.messaging.address.Node$ExchangeNode > > > > Optimization A: The field _exchangeOptions is never used and can be > > removed. It's an empty new HashMap with default capacity 16. For us this > > saves 128B per queue. > > > > Optimization B: The parent class Node has a _bindings List that is by > > default a new ArrayList with the default capacity of 10. In our use case > > _bindings remains empty for all queues. A possible optimization is to set > > the default value as Collections.emptyList() instead, and if > addBindings() > > is called, lazily set it as a new empty list first. For us this saves 80B > > per queue. > > > > > > 4. AMQDestination's _sourceNode field -- > > org.apache.qpid.client.messaging.address.Node$QueueNode > > > > Same 2 optimizations as (3) (with the unused field for QueueNode being > > _queueOptions). > > > > > > Overall: For us, the 5 optimizations in (2), (3), and (4) would save us > > 128B x 3 + 80B x 2 = 544B per queue, and (1) would save us 1384B per > queue; > > all together this would save 1928B out of 2864B, or 67%, per queue for > us. > > > > We'd appreciate feedback on how feasible these changes are and whether we > > are making any incorrect assumptions. Thanks a lot! > > > > Helen > > >
