[
https://issues.apache.org/jira/browse/AMQ-7091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16678790#comment-16678790
]
Gary Tully commented on AMQ-7091:
---------------------------------
[~alanprot] you have to ask the computer to see if there is any negative
performance impact. This reference count in memory is an optimisation
introduced for a performance issue, probably a typical fanout case with lots of
subs. See the linked issue. This issue is about that cache being unbounded in
terms of memory usage, the solution is to have the data in the pageFile and
pageCache such that it can get flushed from memory at the cost of accessing
from pages in normal operation.
I would like to see the actual trade off in publish/ack latency quantified. It
may well be negligible; but we need to respect the original use case to verify.
In addition, if this warrants a version update, the auto migration path needs
to be validated.
> O(n) Memory consumption when broker has inactive durable subscribes causing
> OOM
> -------------------------------------------------------------------------------
>
> Key: AMQ-7091
> URL: https://issues.apache.org/jira/browse/AMQ-7091
> Project: ActiveMQ
> Issue Type: Bug
> Components: KahaDB
> Affects Versions: 5.15.7
> Reporter: Alan Protasio
> Priority: Major
> Attachments: After.png, Before.png,
> InactiveDurableSubscriberTest.java, memoryAllocation.jpg
>
>
> Hi :D
> One of our brokers was bouncing indefinitely due OOM even though the load
> (TPS) was pretty low.
> Getting the memory dump I could see that almost 90% of the memory was being
> used by
> [messageReferences|https://github.com/apache/activemq/blob/master/activemq-kahadb-store/src/main/java/org/apache/activemq/store/kahadb/MessageDatabase.java#L2368]
> TreeMap to keep track of what messages were already acked by all Subscribes
> in order to delete them.
> This seems to be a problem as if the broker has an inactive durable
> subscribe, the memory footprint increase proportionally (O) with the number
> of messages sent to the topic in question, causing the broker to die due OOM
> sooner or later (the high memory footprint continue even after a restart).
> You can find attached (memoryAllocation.jpg) a screen shot showing my broker
> using 90% of the memory to keep track of those messages, making it barely
> usable.
> Looking at the code, I could do a change to change the messageReferences to
> use a BTreeIndex:
> final TreeMap<Long, Long> messageReferences = new TreeMap<>();
> + BTreeIndex<Long, Long> messageReferences;
> Making this change, the memory allocation of the broker stabilized and the
> broker didn't run OOM anymore.
> Attached you can see the code that I used to reproduce this scenario, also
> the memory utilization (HEAP and GC graphs) before and after this change.
> Before the change the broker died in 5 minutes and I could send 480000. After
> then change the broker was still pretty healthy after 5 minutes and i could
> send 2265000 to the topic (almost 5x more due high GC pauses).
>
> All test are passing: mvn clean install -Dactivemq.tests=all
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
