[ https://issues.apache.org/jira/browse/GEODE-6564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17170510#comment-17170510 ]
ASF subversion and git services commented on GEODE-6564: -------------------------------------------------------- Commit f64838f695d56029c0a8b08804f16d5ca9da7156 in geode's branch refs/heads/support/1.12 from Darrel Schneider [ https://gitbox.apache.org/repos/asf?p=geode.git;h=f64838f ] GEODE-6564: fix entryExpiryTasks leak (#5419) (cherry picked from commit 53abb22cd295441f8794e15550b72ec81113ec05) > Clearing a replicated region with expiration causes a memory leak > ----------------------------------------------------------------- > > Key: GEODE-6564 > URL: https://issues.apache.org/jira/browse/GEODE-6564 > Project: Geode > Issue Type: Bug > Components: regions > Reporter: Barrett Oglesby > Assignee: Darrel Schneider > Priority: Major > Labels: pull-request-available > Fix For: 1.14.0 > > > Clearing a replicated region with expiration causes a memory leak > Both the RegionEntries and EntryExpiryTasks are still live after loading > entries into the region and then clearing it. > Server Startup: > {noformat} > num #instances #bytes class name > ---------------------------------------------- > 1: 29856 2797840 [C > 4: 2038 520600 [B > Total 187711 10089624 > {noformat} > Load 100 entries with 600k payload (representing a session): > {noformat} > num #instances #bytes class name > ---------------------------------------------- > 1: 2496 60666440 [B > 2: 30157 2828496 [C > 73: 100 7200 > org.apache.geode.internal.cache.entries.VersionedStatsRegionEntryHeapStringKey1 > 93: 100 4800 org.apache.geode.internal.cache.EntryExpiryTask > Total 190737 70240472 > {noformat} > Clear region: > {noformat} > num #instances #bytes class name > ---------------------------------------------- > 1: 2398 60505944 [B > 2: 30448 2849456 [C > 74: 100 7200 > org.apache.geode.internal.cache.entries.VersionedStatsRegionEntryHeapStringKey1 > 100: 100 4800 org.apache.geode.internal.cache.EntryExpiryTask > Total 192199 70373048 > {noformat} > Load and clear another 100 entries: > {noformat} > num #instances #bytes class name > ---------------------------------------------- > 1: 2503 120511688 [B > 2: 30506 2854384 [C > 46: 200 14400 > org.apache.geode.internal.cache.entries.VersionedStatsRegionEntryHeapStringKey1 > 61: 200 9600 org.apache.geode.internal.cache.EntryExpiryTask > Total 193272 130421432 > {noformat} > Load and clear another 100 entries: > {noformat} > num #instances #bytes class name > ---------------------------------------------- > 1: 2600 180517240 [B > 2: 30562 2859584 [C > 33: 300 21600 > org.apache.geode.internal.cache.entries.VersionedStatsRegionEntryHeapStringKey1 > 47: 300 14400 org.apache.geode.internal.cache.EntryExpiryTask > Total 194310 190468176 > {noformat} > A heap dump shows the VersionedStatsRegionEntryHeapStringKey1 instances are > referenced by the DistributedRegion entryExpiryTasks: > {noformat} > --> org.apache.geode.internal.cache.DistributedRegion@0x76adbbb88 (816 bytes) > (field entryExpiryTasks:) > --> java.util.concurrent.ConcurrentHashMap@0x76adbc028 (100 bytes) (field > table:) > --> [Ljava.util.concurrent.ConcurrentHashMap$Node;@0x76ee85358 (4112 bytes) > (Element 276 of [Ljava.util.concurrent.ConcurrentHashMap$Node;@0x76ee85358:) > --> java.util.concurrent.ConcurrentHashMap$Node@0x76edc4e20 (44 bytes) (field > next:) > --> java.util.concurrent.ConcurrentHashMap$Node@0x76edc32f0 (44 bytes) (field > key:) > --> > org.apache.geode.internal.cache.entries.VersionedStatsRegionEntryHeapStringKey1@0x76edc3210 > (86 bytes) > {noformat} > LocalRegion.cancelAllEntryExpiryTasks is called when the region is cleared: > {noformat} > java.lang.Exception: Stack trace > at java.lang.Thread.dumpStack(Thread.java:1333) > at > org.apache.geode.internal.cache.LocalRegion.cancelAllEntryExpiryTasks(LocalRegion.java:8202) > at > org.apache.geode.internal.cache.LocalRegion.clearRegionLocally(LocalRegion.java:9094) > at > org.apache.geode.internal.cache.DistributedRegion.cmnClearRegion(DistributedRegion.java:1962) > at > org.apache.geode.internal.cache.LocalRegion.basicClear(LocalRegion.java:8998) > at > org.apache.geode.internal.cache.DistributedRegion.basicClear(DistributedRegion.java:1939) > at > org.apache.geode.internal.cache.LocalRegion.basicBridgeClear(LocalRegion.java:8988) > at > org.apache.geode.internal.cache.tier.sockets.command.ClearRegion.cmdExecute(ClearRegion.java:123) > {noformat} > But it doesn't clear the entryExpiryTasks map: > {noformat} > LocalRegion.clearRegionLocally before cancelAllEntryExpiryTasks > entryExpiryTasks=100 > LocalRegion.clearRegionLocally after cancelAllEntryExpiryTasks > entryExpiryTasks=100 > {noformat} > As a test, I added this call to the bottom of the cancelAllEntryExpiryTasks > method: > {noformat} > this.entryExpiryTasks.clear(); > {noformat} > This addressed the leak: > {noformat} > Server Startup: Total 182414 9855616 > Load/Clear 1: Total 191049 10315832 > Load/Clear 2: Total 191978 10329664 > Load/Clear 3: Total 192638 10360360 > {noformat} > As a work-around, a Function that clears the region by using removeAll on > batches of keys also addresses the leak: > {noformat} > Server Startup: Total 182297 9849312 > Load/Clear 1: Total 185932 10019248 > Load/Clear 2: Total 191855 10278816 > Load/Clear 3: Total 192511 10313168 > Load/Clear 4: Total 193424 10352008 > {noformat} -- This message was sent by Atlassian Jira (v8.3.4#803005)