[jira] [Commented] (CASSANDRA-9117) LEAK DETECTED during repair, startup
[
https://issues.apache.org/jira/browse/CASSANDRA-9117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15180494#comment-15180494
]
Sebastian Estevez commented on CASSANDRA-9117:
--
For future reference I ended up openning CASSANDRA-10079 and it's fixed in
2.1.12
> LEAK DETECTED during repair, startup
>
>
> Key: CASSANDRA-9117
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9117
> Project: Cassandra
> Issue Type: Bug
>Reporter: Tyler Hobbs
>Assignee: Marcus Eriksson
> Fix For: 2.2.0 beta 1
>
> Attachments:
> 0001-dont-initialize-writer-before-checking-if-iter-is-em.patch, node1.log,
> node2.log.gz
>
>
> When running the
> {{incremental_repair_test.TestIncRepair.multiple_repair_test}} dtest, the
> following error logs show up:
> {noformat}
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,491 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@83f047e) to class
> org.apache.cassandra.io.util.SafeMemory$MemoryTidy@1631580268:Memory@[7f354800bdc0..7f354800bde8)
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,493 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@50bc8f67) to class
> org.apache.cassandra.io.util.SafeMemory$MemoryTidy@191552666:Memory@[7f354800ba90..7f354800bdb0)
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,493 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@7fd10877) to class
> org.apache.cassandra.io.util.SafeMemory$MemoryTidy@1954741807:Memory@[7f3548101190..7f3548101194)
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,494 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@578550ac) to class
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$1@1903393047:[[OffHeapBitSet]]
> was not released before the reference was garbage collected
> {noformat}
> The test is being run against trunk (commit {{1dff098e}}). I've attached a
> DEBUG-level log from the test run.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9117) LEAK DETECTED during repair, startup
[
https://issues.apache.org/jira/browse/CASSANDRA-9117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697305#comment-14697305
]
Sebastian Estevez commented on CASSANDRA-9117:
--
Sure thing
> LEAK DETECTED during repair, startup
>
>
> Key: CASSANDRA-9117
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9117
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Tyler Hobbs
>Assignee: Marcus Eriksson
> Fix For: 2.2.0 beta 1
>
> Attachments:
> 0001-dont-initialize-writer-before-checking-if-iter-is-em.patch, node1.log,
> node2.log.gz
>
>
> When running the
> {{incremental_repair_test.TestIncRepair.multiple_repair_test}} dtest, the
> following error logs show up:
> {noformat}
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,491 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@83f047e) to class
> org.apache.cassandra.io.util.SafeMemory$MemoryTidy@1631580268:Memory@[7f354800bdc0..7f354800bde8)
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,493 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@50bc8f67) to class
> org.apache.cassandra.io.util.SafeMemory$MemoryTidy@191552666:Memory@[7f354800ba90..7f354800bdb0)
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,493 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@7fd10877) to class
> org.apache.cassandra.io.util.SafeMemory$MemoryTidy@1954741807:Memory@[7f3548101190..7f3548101194)
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,494 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@578550ac) to class
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$1@1903393047:[[OffHeapBitSet]]
> was not released before the reference was garbage collected
> {noformat}
> The test is being run against trunk (commit {{1dff098e}}). I've attached a
> DEBUG-level log from the test run.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9117) LEAK DETECTED during repair, startup
[
https://issues.apache.org/jira/browse/CASSANDRA-9117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697301#comment-14697301
]
Marcus Eriksson commented on CASSANDRA-9117:
[~victortrac] yours is probably CASSANDRA-9998
[~sebastian.este...@datastax.com] could you open a new ticket for that?
> LEAK DETECTED during repair, startup
>
>
> Key: CASSANDRA-9117
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9117
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Tyler Hobbs
>Assignee: Marcus Eriksson
> Fix For: 2.2.0 beta 1
>
> Attachments:
> 0001-dont-initialize-writer-before-checking-if-iter-is-em.patch, node1.log,
> node2.log.gz
>
>
> When running the
> {{incremental_repair_test.TestIncRepair.multiple_repair_test}} dtest, the
> following error logs show up:
> {noformat}
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,491 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@83f047e) to class
> org.apache.cassandra.io.util.SafeMemory$MemoryTidy@1631580268:Memory@[7f354800bdc0..7f354800bde8)
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,493 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@50bc8f67) to class
> org.apache.cassandra.io.util.SafeMemory$MemoryTidy@191552666:Memory@[7f354800ba90..7f354800bdb0)
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,493 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@7fd10877) to class
> org.apache.cassandra.io.util.SafeMemory$MemoryTidy@1954741807:Memory@[7f3548101190..7f3548101194)
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,494 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@578550ac) to class
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$1@1903393047:[[OffHeapBitSet]]
> was not released before the reference was garbage collected
> {noformat}
> The test is being run against trunk (commit {{1dff098e}}). I've attached a
> DEBUG-level log from the test run.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9117) LEAK DETECTED during repair, startup
[
https://issues.apache.org/jira/browse/CASSANDRA-9117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697248#comment-14697248
]
Victor Trac commented on CASSANDRA-9117:
Seeing this during a -pr repair on 2.2.0:
{code}
ERROR [Reference-Reaper:1] 2015-08-14 15:37:04,809 Ref.java:187 - LEAK
DETECTED: a reference
(org.apache.cassandra.utils.concurrent.Ref$State@645aba1e) to class
org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier@785147240:/mnt/cassandra/data/crawl/link2-b1fa78203b0711e5a72aa1fb779f0f22/la-2257-big
was not released before the reference was garbage collected
ERROR [Reference-Reaper:1] 2015-08-14 15:37:04,809 Ref.java:187 - LEAK
DETECTED: a reference (org.apache.cassandra.utils.concurrent.Ref$State@713b0b)
to class
org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier@679274203:/mnt/cassandra/data/crawl/link2-b1fa78203b0711e5a72aa1fb779f0f22/la-2266-big
was not released before the reference was garbage collected
ERROR [Reference-Reaper:1] 2015-08-14 15:37:04,809 Ref.java:187 - LEAK
DETECTED: a reference (org.apache.cassandra.utils.concurrent.Ref$State@9db4f67)
to class
org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier@440947325:/mnt/cassandra/data/crawl/link2-b1fa78203b0711e5a72aa1fb779f0f22/la-2244-big
was not released before the reference was garbage collected
{code}
> LEAK DETECTED during repair, startup
>
>
> Key: CASSANDRA-9117
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9117
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Tyler Hobbs
>Assignee: Marcus Eriksson
> Fix For: 2.2.0 beta 1
>
> Attachments:
> 0001-dont-initialize-writer-before-checking-if-iter-is-em.patch, node1.log,
> node2.log.gz
>
>
> When running the
> {{incremental_repair_test.TestIncRepair.multiple_repair_test}} dtest, the
> following error logs show up:
> {noformat}
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,491 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@83f047e) to class
> org.apache.cassandra.io.util.SafeMemory$MemoryTidy@1631580268:Memory@[7f354800bdc0..7f354800bde8)
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,493 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@50bc8f67) to class
> org.apache.cassandra.io.util.SafeMemory$MemoryTidy@191552666:Memory@[7f354800ba90..7f354800bdb0)
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,493 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@7fd10877) to class
> org.apache.cassandra.io.util.SafeMemory$MemoryTidy@1954741807:Memory@[7f3548101190..7f3548101194)
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,494 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@578550ac) to class
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$1@1903393047:[[OffHeapBitSet]]
> was not released before the reference was garbage collected
> {noformat}
> The test is being run against trunk (commit {{1dff098e}}). I've attached a
> DEBUG-level log from the test run.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9117) LEAK DETECTED during repair, startup
[
https://issues.apache.org/jira/browse/CASSANDRA-9117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697241#comment-14697241
]
Sebastian Estevez commented on CASSANDRA-9117:
--
[~tjake] thanks for looking at this.
Correction, it happened after a node was drained not during startup:
2015-08-14 13:37:07,049 - Drain the node
2015-08-14 13:37:11,943 - Drained
2015-08-14 13:37:37,055 Ref.java:179 - LEAK DETECTED:
> LEAK DETECTED during repair, startup
>
>
> Key: CASSANDRA-9117
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9117
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Tyler Hobbs
>Assignee: Marcus Eriksson
> Fix For: 2.2.0 beta 1
>
> Attachments:
> 0001-dont-initialize-writer-before-checking-if-iter-is-em.patch, node1.log,
> node2.log.gz
>
>
> When running the
> {{incremental_repair_test.TestIncRepair.multiple_repair_test}} dtest, the
> following error logs show up:
> {noformat}
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,491 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@83f047e) to class
> org.apache.cassandra.io.util.SafeMemory$MemoryTidy@1631580268:Memory@[7f354800bdc0..7f354800bde8)
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,493 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@50bc8f67) to class
> org.apache.cassandra.io.util.SafeMemory$MemoryTidy@191552666:Memory@[7f354800ba90..7f354800bdb0)
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,493 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@7fd10877) to class
> org.apache.cassandra.io.util.SafeMemory$MemoryTidy@1954741807:Memory@[7f3548101190..7f3548101194)
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,494 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@578550ac) to class
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$1@1903393047:[[OffHeapBitSet]]
> was not released before the reference was garbage collected
> {noformat}
> The test is being run against trunk (commit {{1dff098e}}). I've attached a
> DEBUG-level log from the test run.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9117) LEAK DETECTED during repair, startup
[
https://issues.apache.org/jira/browse/CASSANDRA-9117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697190#comment-14697190
]
Sebastian Estevez commented on CASSANDRA-9117:
--
Seeing this issue on 2.1.8:
https://dl.dropboxusercontent.com/u/4179566/cassandra-system.log
{code}
ERROR [Reference-Reaper:1] 2015-08-14 13:37:37,055 Ref.java:179 - LEAK
DETECTED: a reference (org.apache.cassandra.utils.concurrent.Ref$State@5534701)
to class
org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$1@194296283:[[OffHeapBitSet]]
was not released before the reference was garbage collected
ERROR [Reference-Reaper:1] 2015-08-14 13:37:37,057 Ref.java:179 - LEAK
DETECTED: a reference (org.apache.cassandra.utils.concurrent.Ref$State@fab2c71)
to class
org.apache.cassandra.io.util.MmappedSegmentedFile$Cleanup@1252635616:/var/lib/cassandra/data/metric/metric-811fa5402a3b11e5a2c0870545c0f352/metric-metric-ka-6883-Index.db
was not released before the reference was garbage collected
ERROR [Reference-Reaper:1] 2015-08-14 13:37:37,057 Ref.java:179 - LEAK
DETECTED: a reference
(org.apache.cassandra.utils.concurrent.Ref$State@555d8efb) to class
org.apache.cassandra.io.util.MmappedSegmentedFile$Cleanup@1252635616:/var/lib/cassandra/data/metric/metric-811fa5402a3b11e5a2c0870545c0f352/metric-metric-ka-6883-Index.db
was not released before the reference was garbage collected
ERROR [Reference-Reaper:1] 2015-08-14 13:37:37,057 Ref.java:179 - LEAK
DETECTED: a reference
(org.apache.cassandra.utils.concurrent.Ref$State@7b29bfea) to class
org.apache.cassandra.io.util.MmappedSegmentedFile$Cleanup@1252635616:/var/lib/cassandra/data/metric/metric-811fa5402a3b11e5a2c0870545c0f352/metric-metric-ka-6883-Index.db
was not released before the reference was garbage collected
ERROR [Reference-Reaper:1] 2015-08-14 13:37:37,057 Ref.java:179 - LEAK
DETECTED: a reference
(org.apache.cassandra.utils.concurrent.Ref$State@2d37dc5a) to class
org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$1@713444527:[[OffHeapBitSet]]
was not released before the reference was garbage collected
ERROR [Reference-Reaper:1] 2015-08-14 13:37:37,057 Ref.java:179 - LEAK
DETECTED: a reference
(org.apache.cassandra.utils.concurrent.Ref$State@13153552) to class
org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$1@713444527:[[OffHeapBitSet]]
was not released before the reference was garbage collected
ERROR [Reference-Reaper:1] 2015-08-14 13:37:37,057 Ref.java:179 - LEAK
DETECTED: a reference
(org.apache.cassandra.utils.concurrent.Ref$State@25f51e35) to class
org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$1@713444527:[[OffHeapBitSet]]
was not released before the reference was garbage collected
ERROR [Reference-Reaper:1] 2015-08-14 13:37:37,057 Ref.java:179 - LEAK
DETECTED: a reference
(org.apache.cassandra.utils.concurrent.Ref$State@3633d3dd) to class
org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$1@194296283:[[OffHeapBitSet]]
was not released before the reference was garbage collected
ERROR [Reference-Reaper:1] 2015-08-14 13:37:37,057 Ref.java:179 - LEAK
DETECTED: a reference
(org.apache.cassandra.utils.concurrent.Ref$State@2ec81280) to class
org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$1@194296283:[[OffHeapBitSet]]
was not released before the reference was garbage collected
ERROR [Reference-Reaper:1] 2015-08-14 13:37:37,058 Ref.java:179 - LEAK
DETECTED: a reference
(org.apache.cassandra.utils.concurrent.Ref$State@144d1dae) to class
org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$1@194296283:[[OffHeapBitSet]]
was not released before the reference was garbage collected
ERROR [Reference-Reaper:1] 2015-08-14 13:37:37,058 Ref.java:179 - LEAK
DETECTED: a reference
(org.apache.cassandra.utils.concurrent.Ref$State@1944bda4) to class
org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$1@194296283:[[OffHeapBitSet]]
was not released before the reference was garbage collected
ERROR [Reference-Reaper:1] 2015-08-14 13:37:37,058 Ref.java:179 - LEAK
DETECTED: a reference
(org.apache.cassandra.utils.concurrent.Ref$State@31c1386a) to class
org.apache.cassandra.io.util.MmappedSegmentedFile$Cleanup@1601396928:/var/lib/cassandra/data/metric/metric-811fa5402a3b11e5a2c0870545c0f352/metric-metric-ka-8229-Index.db
was not released before the reference was garbage collected
{code}
> LEAK DETECTED during repair, startup
>
>
> Key: CASSANDRA-9117
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9117
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Tyler Hobbs
>Assignee: Marcus Eriksson
> Fix For: 2.2.0 beta 1
>
> Attachments:
> 0001-dont-initialize-writer-before-checking-if-iter-is-em.patch, node1.log,
> node2
[jira] [Commented] (CASSANDRA-9117) LEAK DETECTED during repair, startup
[
https://issues.apache.org/jira/browse/CASSANDRA-9117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697204#comment-14697204
]
T Jake Luciani commented on CASSANDRA-9117:
---
[~sebastian.este...@datastax.com] After commit log replay someone called drain
{code}
INFO [RMI TCP Connection(1895)-127.0.0.1] 2015-08-14 13:37:07,049
StorageService.java:1115 - DRAINING: starting drain process
{code}
> LEAK DETECTED during repair, startup
>
>
> Key: CASSANDRA-9117
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9117
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Tyler Hobbs
>Assignee: Marcus Eriksson
> Fix For: 2.2.0 beta 1
>
> Attachments:
> 0001-dont-initialize-writer-before-checking-if-iter-is-em.patch, node1.log,
> node2.log.gz
>
>
> When running the
> {{incremental_repair_test.TestIncRepair.multiple_repair_test}} dtest, the
> following error logs show up:
> {noformat}
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,491 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@83f047e) to class
> org.apache.cassandra.io.util.SafeMemory$MemoryTidy@1631580268:Memory@[7f354800bdc0..7f354800bde8)
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,493 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@50bc8f67) to class
> org.apache.cassandra.io.util.SafeMemory$MemoryTidy@191552666:Memory@[7f354800ba90..7f354800bdb0)
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,493 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@7fd10877) to class
> org.apache.cassandra.io.util.SafeMemory$MemoryTidy@1954741807:Memory@[7f3548101190..7f3548101194)
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,494 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@578550ac) to class
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$1@1903393047:[[OffHeapBitSet]]
> was not released before the reference was garbage collected
> {noformat}
> The test is being run against trunk (commit {{1dff098e}}). I've attached a
> DEBUG-level log from the test run.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9117) LEAK DETECTED during repair, startup
[
https://issues.apache.org/jira/browse/CASSANDRA-9117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697197#comment-14697197
]
Sebastian Estevez commented on CASSANDRA-9117:
--
This was not during a repair ^^
> LEAK DETECTED during repair, startup
>
>
> Key: CASSANDRA-9117
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9117
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Tyler Hobbs
>Assignee: Marcus Eriksson
> Fix For: 2.2.0 beta 1
>
> Attachments:
> 0001-dont-initialize-writer-before-checking-if-iter-is-em.patch, node1.log,
> node2.log.gz
>
>
> When running the
> {{incremental_repair_test.TestIncRepair.multiple_repair_test}} dtest, the
> following error logs show up:
> {noformat}
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,491 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@83f047e) to class
> org.apache.cassandra.io.util.SafeMemory$MemoryTidy@1631580268:Memory@[7f354800bdc0..7f354800bde8)
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,493 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@50bc8f67) to class
> org.apache.cassandra.io.util.SafeMemory$MemoryTidy@191552666:Memory@[7f354800ba90..7f354800bdb0)
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,493 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@7fd10877) to class
> org.apache.cassandra.io.util.SafeMemory$MemoryTidy@1954741807:Memory@[7f3548101190..7f3548101194)
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,494 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@578550ac) to class
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$1@1903393047:[[OffHeapBitSet]]
> was not released before the reference was garbage collected
> {noformat}
> The test is being run against trunk (commit {{1dff098e}}). I've attached a
> DEBUG-level log from the test run.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9117) LEAK DETECTED during repair, startup
[
https://issues.apache.org/jira/browse/CASSANDRA-9117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14654118#comment-14654118
]
Mark Manley commented on CASSANDRA-9117:
I'm still seeing this in 2.2.0:
{code}
ERROR [MessagingService-Outgoing-/10.20.44.122] 2015-07-30 18:55:00,589
OutboundTcpConnection.java:316 - error writing to /10.20.44.122
ERROR [MessagingService-Outgoing-/10.20.44.74] 2015-07-31 10:52:39,346
OutboundTcpConnection.java:316 - error writing to /10.20.44.74
ERROR [STREAM-OUT-/10.20.44.108] 2015-07-31 20:22:17,052 StreamSession.java:518
- [Stream #6f73e430-37c1-11e5-9fb4-a322a3bdb126] Streaming error occurred
ERROR [STREAM-IN-/10.20.44.108] 2015-07-31 20:22:18,513 StreamSession.java:518
- [Stream #6f73e430-37c1-11e5-9fb4-a322a3bdb126] Streaming error occurred
ERROR [Reference-Reaper:1] 2015-07-31 20:22:23,444 Ref.java:187 - LEAK
DETECTED: a reference
(org.apache.cassandra.utils.concurrent.Ref$State@35ceb976) to class
org.apache.cassandra.io.util.SafeMemory$MemoryTidy@990466495:Memory@[7f426f54e880..7f426f54e884)
was not released before the reference was garbage collected
ERROR [Reference-Reaper:1] 2015-07-31 20:22:23,445 Ref.java:187 - LEAK
DETECTED: a reference
(org.apache.cassandra.utils.concurrent.Ref$State@3f36d206) to class
org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$1@2050251652:[[OffHeapBitSet]]
was not released before the reference was garbage collected
ERROR [Reference-Reaper:1] 2015-07-31 20:22:23,445 Ref.java:187 - LEAK
DETECTED: a reference
(org.apache.cassandra.utils.concurrent.Ref$State@3af158bd) to class
org.apache.cassandra.io.util.SafeMemory$MemoryTidy@1258578677:Memory@[7f56a130f400..7f56a130fa40)
was not released before the reference was garbage collected
ERROR [Reference-Reaper:1] 2015-07-31 20:22:23,445 Ref.java:187 - LEAK
DETECTED: a reference
(org.apache.cassandra.utils.concurrent.Ref$State@422b3f71) to class
org.apache.cassandra.io.util.SafeMemory$MemoryTidy@438019275:Memory@[7f56a0158150..7f56a01581a0)
was not released before the reference was garbage collected
ERROR [MessagingService-Outgoing-/10.20.44.108] 2015-08-02 00:21:30,685
OutboundTcpConnection.java:316 - error writing to /10.20.44.108
{code}
> LEAK DETECTED during repair, startup
>
>
> Key: CASSANDRA-9117
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9117
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Tyler Hobbs
>Assignee: Marcus Eriksson
> Fix For: 2.2.0 beta 1
>
> Attachments:
> 0001-dont-initialize-writer-before-checking-if-iter-is-em.patch, node1.log,
> node2.log.gz
>
>
> When running the
> {{incremental_repair_test.TestIncRepair.multiple_repair_test}} dtest, the
> following error logs show up:
> {noformat}
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,491 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@83f047e) to class
> org.apache.cassandra.io.util.SafeMemory$MemoryTidy@1631580268:Memory@[7f354800bdc0..7f354800bde8)
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,493 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@50bc8f67) to class
> org.apache.cassandra.io.util.SafeMemory$MemoryTidy@191552666:Memory@[7f354800ba90..7f354800bdb0)
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,493 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@7fd10877) to class
> org.apache.cassandra.io.util.SafeMemory$MemoryTidy@1954741807:Memory@[7f3548101190..7f3548101194)
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,494 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@578550ac) to class
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$1@1903393047:[[OffHeapBitSet]]
> was not released before the reference was garbage collected
> {noformat}
> The test is being run against trunk (commit {{1dff098e}}). I've attached a
> DEBUG-level log from the test run.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9117) LEAK DETECTED during repair, startup
[
https://issues.apache.org/jira/browse/CASSANDRA-9117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14488313#comment-14488313
]
Benedict commented on CASSANDRA-9117:
-
Perfect. Thanks!
> LEAK DETECTED during repair, startup
>
>
> Key: CASSANDRA-9117
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9117
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Tyler Hobbs
>Assignee: Benedict
> Fix For: 3.0
>
> Attachments: node1.log, node2.log.gz
>
>
> When running the
> {{incremental_repair_test.TestIncRepair.multiple_repair_test}} dtest, the
> following error logs show up:
> {noformat}
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,491 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@83f047e) to class
> org.apache.cassandra.io.util.SafeMemory$MemoryTidy@1631580268:Memory@[7f354800bdc0..7f354800bde8)
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,493 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@50bc8f67) to class
> org.apache.cassandra.io.util.SafeMemory$MemoryTidy@191552666:Memory@[7f354800ba90..7f354800bdb0)
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,493 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@7fd10877) to class
> org.apache.cassandra.io.util.SafeMemory$MemoryTidy@1954741807:Memory@[7f3548101190..7f3548101194)
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,494 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@578550ac) to class
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$1@1903393047:[[OffHeapBitSet]]
> was not released before the reference was garbage collected
> {noformat}
> The test is being run against trunk (commit {{1dff098e}}). I've attached a
> DEBUG-level log from the test run.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9117) LEAK DETECTED during repair, startup
[
https://issues.apache.org/jira/browse/CASSANDRA-9117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14488300#comment-14488300
]
Joshua McKenzie commented on CASSANDRA-9117:
I should have something useful to say about it by tomorrow morning to mid-day.
:)
> LEAK DETECTED during repair, startup
>
>
> Key: CASSANDRA-9117
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9117
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Tyler Hobbs
>Assignee: Benedict
> Fix For: 3.0
>
> Attachments: node1.log, node2.log.gz
>
>
> When running the
> {{incremental_repair_test.TestIncRepair.multiple_repair_test}} dtest, the
> following error logs show up:
> {noformat}
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,491 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@83f047e) to class
> org.apache.cassandra.io.util.SafeMemory$MemoryTidy@1631580268:Memory@[7f354800bdc0..7f354800bde8)
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,493 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@50bc8f67) to class
> org.apache.cassandra.io.util.SafeMemory$MemoryTidy@191552666:Memory@[7f354800ba90..7f354800bdb0)
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,493 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@7fd10877) to class
> org.apache.cassandra.io.util.SafeMemory$MemoryTidy@1954741807:Memory@[7f3548101190..7f3548101194)
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,494 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@578550ac) to class
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$1@1903393047:[[OffHeapBitSet]]
> was not released before the reference was garbage collected
> {noformat}
> The test is being run against trunk (commit {{1dff098e}}). I've attached a
> DEBUG-level log from the test run.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9117) LEAK DETECTED during repair, startup
[
https://issues.apache.org/jira/browse/CASSANDRA-9117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14488297#comment-14488297
]
Benedict commented on CASSANDRA-9117:
-
I won't have time to look into this for at least a week. If we could get a
bisect in the meantime, it might help. [~JoshuaMcKenzie]: how are your thoughts
coming along on CASSANDRA-8894?
> LEAK DETECTED during repair, startup
>
>
> Key: CASSANDRA-9117
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9117
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Tyler Hobbs
>Assignee: Benedict
> Fix For: 3.0
>
> Attachments: node1.log, node2.log.gz
>
>
> When running the
> {{incremental_repair_test.TestIncRepair.multiple_repair_test}} dtest, the
> following error logs show up:
> {noformat}
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,491 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@83f047e) to class
> org.apache.cassandra.io.util.SafeMemory$MemoryTidy@1631580268:Memory@[7f354800bdc0..7f354800bde8)
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,493 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@50bc8f67) to class
> org.apache.cassandra.io.util.SafeMemory$MemoryTidy@191552666:Memory@[7f354800ba90..7f354800bdb0)
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,493 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@7fd10877) to class
> org.apache.cassandra.io.util.SafeMemory$MemoryTidy@1954741807:Memory@[7f3548101190..7f3548101194)
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,494 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@578550ac) to class
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$1@1903393047:[[OffHeapBitSet]]
> was not released before the reference was garbage collected
> {noformat}
> The test is being run against trunk (commit {{1dff098e}}). I've attached a
> DEBUG-level log from the test run.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9117) LEAK DETECTED during repair, startup
[
https://issues.apache.org/jira/browse/CASSANDRA-9117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14488278#comment-14488278
]
Tyler Hobbs commented on CASSANDRA-9117:
There are more dtests that are affected by this than I thought at first. This
also affects {{topology_test.TestTopology.decomission_test}},
{{secondary_indexes_test.TestSecondaryIndexes.test_6924_dropping_ks}},
{{incremental_repair_test.TestIncRepair.sstable_repairedset_test}}, and
{{counter_tests.TestCounters.upgrade_test}}. It's important that we get the
dtests into a reliable state soon, especially so that we can properly test
CASSANDRA-8099. I don't want to rush the review of CASSANDRA-8984 just so that
we can fix the dtests. Can you try to look into this (using the patch from
8984, if that's useful)?
> LEAK DETECTED during repair, startup
>
>
> Key: CASSANDRA-9117
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9117
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Tyler Hobbs
>Assignee: Benedict
> Fix For: 3.0
>
> Attachments: node1.log, node2.log.gz
>
>
> When running the
> {{incremental_repair_test.TestIncRepair.multiple_repair_test}} dtest, the
> following error logs show up:
> {noformat}
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,491 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@83f047e) to class
> org.apache.cassandra.io.util.SafeMemory$MemoryTidy@1631580268:Memory@[7f354800bdc0..7f354800bde8)
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,493 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@50bc8f67) to class
> org.apache.cassandra.io.util.SafeMemory$MemoryTidy@191552666:Memory@[7f354800ba90..7f354800bdb0)
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,493 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@7fd10877) to class
> org.apache.cassandra.io.util.SafeMemory$MemoryTidy@1954741807:Memory@[7f3548101190..7f3548101194)
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,494 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@578550ac) to class
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$1@1903393047:[[OffHeapBitSet]]
> was not released before the reference was garbage collected
> {noformat}
> The test is being run against trunk (commit {{1dff098e}}). I've attached a
> DEBUG-level log from the test run.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9117) LEAK DETECTED during repair, startup
[
https://issues.apache.org/jira/browse/CASSANDRA-9117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14395154#comment-14395154
]
Benedict commented on CASSANDRA-9117:
-
I was looking into a flood of LEAK DETECTED earlier today on trunk, but since
they don't affect 2.1, I was hoping to wait until CASSANDRA-8984 is committed,
since I fully expect that to fix them if they're bugs anywhere in the main
innards of resource management.
> LEAK DETECTED during repair, startup
>
>
> Key: CASSANDRA-9117
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9117
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Tyler Hobbs
>Assignee: Benedict
> Fix For: 3.0
>
> Attachments: node1.log, node2.log.gz
>
>
> When running the
> {{incremental_repair_test.TestIncRepair.multiple_repair_test}} dtest, the
> following error logs show up:
> {noformat}
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,491 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@83f047e) to class
> org.apache.cassandra.io.util.SafeMemory$MemoryTidy@1631580268:Memory@[7f354800bdc0..7f354800bde8)
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,493 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@50bc8f67) to class
> org.apache.cassandra.io.util.SafeMemory$MemoryTidy@191552666:Memory@[7f354800ba90..7f354800bdb0)
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,493 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@7fd10877) to class
> org.apache.cassandra.io.util.SafeMemory$MemoryTidy@1954741807:Memory@[7f3548101190..7f3548101194)
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-04-03 15:48:25,494 Ref.java:181 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@578550ac) to class
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$1@1903393047:[[OffHeapBitSet]]
> was not released before the reference was garbage collected
> {noformat}
> The test is being run against trunk (commit {{1dff098e}}). I've attached a
> DEBUG-level log from the test run.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
