jvarenina opened a new pull request #7234: URL: https://github.com/apache/geode/pull/7234
Issue: An OpLog files are compacted, but the .drf file is left because it contains deletes of entries in previous .crfs. The .crf file is deleted, but the orphaned .drf is not until all previous .crf files (.crfs with smaller id) are deleted. The problem is that compacted Oplog object representing orphaned .drf file holds a structure in memory (Oplog.regionMap) that contains information that is not useful after the compaction and it takes certain amount of memory. Besides, there is a special case when creating .krf files that, depending on the execution order, could make the problem more severe (it could leave pendingKrfTags structure on the regionMap and this could take up a significant amount of memory). This pendingKrfTags HashMap is actually empty, but consumes memory because it was used previously and the size of the HashMap was not reduced after it is cleared. This special case usually happens when new Oplog is rolled out and previous Oplog is immediately marked as eligible for compaction. Compaction and .krf creation start at the similar time and compactor cancels creation of .krf file. The pendingKrfTags structure is usually cleared when .krf file is created, but since compaction canceled creation of .krf, the pendingKrfTags structure remain in memory until Oplog representing orphaned .drf file is deleted. Solution: Clear the regionMap data structure of the Oplog when it is compacted (currently it is deleted when the Oplog is destroyed). Co-authored-by: Alberto Gomez <[email protected]> <!-- Thank you for submitting a contribution to Apache Geode. --> <!-- In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: --> ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Has your PR been rebased against the latest commit within the target branch (typically `develop`)? - [x] Is your initial contribution a single, squashed commit? - [x] Does `gradlew build` run cleanly? - [x] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? <!-- Note: Please ensure that once the PR is submitted, check Concourse for build issues and submit an update to your PR as soon as possible. If you need help, please send an email to [email protected]. --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
