[
https://issues.apache.org/jira/browse/AMQ-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14296173#comment-14296173
]
Sergiy Barlabanov commented on AMQ-5542:
----------------------------------------
This is correct. This is what we have in production. We have some messages in
DLQs. They stay there for max 2 days. After that a special job removes messages
older than 2 days from the DLQs.
Now we expect that ActiveMQ will always retain nearly all data files for the
last two days because of those messages in the DLQs. So this would take quite a
lot of space I think and we did not expect that when we set up the servers -
may be we will have to get a larger SAN volume for that.
Another consideration is that this is not only the space which will be eaten by
KahaDB but also the time ActiveMQ needs to recover after the crash. ActiveMQ
will need quite a lot of time to replay all the data files which are sitting
there just because of several DLQ messages.
And the periodic cleanup may take more time to check all the data files (and
KahaDB cleanup is a single threaded storage blocking operation).
> KahaDB data files containing acknowledgements are deleted during cleanup
> ------------------------------------------------------------------------
>
> Key: AMQ-5542
> URL: https://issues.apache.org/jira/browse/AMQ-5542
> Project: ActiveMQ
> Issue Type: Bug
> Components: Message Store
> Affects Versions: 5.10.0, 5.10.1
> Reporter: Sergiy Barlabanov
> Attachments: AMQ-5542.patch, AdjustedAMQ2832Test.patch
>
>
> AMQ-2832 was not fixed cleanly.
> The commit dd68c61e65f24b7dc498b36e34960a4bc46ded4b by Gary from 8.10.2010
> introduced a problem by deleting too many files.
> Scenarios we are facing currently in production:
> Data file #1 contains unconsumed messages sitting in a DLQ. So this file is
> not a cleanup candidate.
> The next file #2 contains acks of some messages from file #1. This file is
> not a cleanup candidate (because of ackMessageFileMap logic).
> The next file #3 contains acks of some messages from file #2. And this file
> is deleted during the cleanup procedure. So on Broker restart all messages
> from #2, whose acks were from the deleted file #3, are replayed!
> The reason is gcCandidates variable, which is a copy of gcCandidateSet (see
> MessageDatabase#checkpointUpdate at the end of the method -
> org/apache/activemq/store/kahadb/MessageDatabase.java:1659 on 5.10.0 tag). So
> when a candidate is deleted from gcCandidateSet
> (org/apache/activemq/store/kahadb/MessageDatabase.java:1668 on 5.10.0 tag),
> gcCandidates still contains that candidate and the comparison on
> org/apache/activemq/store/kahadb/MessageDatabase.java:1666 works wrong!
> I will try to adjust AMQ2832Test.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
