[
https://issues.apache.org/jira/browse/CASSANDRA-11767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15306517#comment-15306517
]
Stefania commented on CASSANDRA-11767:
--------------------------------------
I'm not able to reproduce this locally, I'm not sure if it's because I keep on
hitting CASSANDRA-11393.
We need to reproduce this issue with {{-Dcassandra.debugrefcount=true}} in
order to work out where the resources were allocated. I suspect it is an
sstable reader that was not released, possibly due to the interrupted
compaction during a drain. The problem happened in 2.2.
[~philipthompson] would you be able to run this test a few times with
{{-Dcassandra.debugrefcount=true}}?
> dtest failure in
> upgrade_tests.upgrade_through_versions_test.ProtoV3Upgrade_AllVersions_EndsAt_Trunk_HEAD.rolling_upgrade_test
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-11767
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11767
> Project: Cassandra
> Issue Type: Bug
> Reporter: Philip Thompson
> Assignee: Stefania
> Priority: Minor
> Labels: dtest
> Attachments: node1.log, node1_debug.log, node2.log, node2_debug.log,
> node3.log, node3_debug.log
>
>
> No cassci link, as jenkins is failing to show failures for this test, but I'm
> seeing detected leaks.
> Relevant log section:
> {code}
> ERROR [Reference-Reaper:1] 2016-05-11 16:17:47,616 Ref.java:187 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@41f74411) to class
> org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Cleanup@1328587359:/mnt/tmp/dtest-X5kTWw/test/node1/data0/system/local-7ad54392bcdd35a684174e047860b377/la-12-big-Data.db
> was not released before the reference was garbage collected
> DEBUG [Reference-Reaper:1] 2016-05-11 16:17:47,617 FileCacheService.java:177
> - Invalidating cache for
> /mnt/tmp/dtest-X5kTWw/test/node1/data2/system/local-7ad54392bcdd35a684174e047860b377/tmplink-la-13-big-Data.db
> ERROR [Reference-Reaper:1] 2016-05-11 16:17:47,617 Ref.java:187 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@4b0f2f61) to class
> org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Cleanup@235776100:/mnt/tmp/dtest-X5kTWw/test/node1/data2/system/local-7ad54392bcdd35a684174e047860b377/tmplink-la-13-big-Data.db
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2016-05-11 16:17:47,617 Ref.java:187 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@75a1b278) to class
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$1@1061799113:[Memory@[0..8),
> Memory@[0..50)] was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2016-05-11 16:17:47,617 Ref.java:187 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@d0f8066) to class
> org.apache.cassandra.io.util.MmappedSegmentedFile$Cleanup@1847744299:/mnt/tmp/dtest-X5kTWw/test/node1/data2/system/local-7ad54392bcdd35a684174e047860b377/tmplink-la-13-big-Index.db
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2016-05-11 16:17:47,617 Ref.java:187 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@16680ae5) to class
> org.apache.cassandra.io.util.MmappedSegmentedFile$Cleanup@1744239469:/mnt/tmp/dtest-X5kTWw/test/node1/data0/system/local-7ad54392bcdd35a684174e047860b377/la-12-big-Index.db
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2016-05-11 16:17:47,618 Ref.java:187 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@160b4b45) to class
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$1@1785515424:[[OffHeapBitSet]]
> was not released before the reference was garbage collected
> {code}
> Logs are attached. node1 is the one experiencing the issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
