[
https://issues.apache.org/jira/browse/CASSANDRA-11904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dan Kinder updated CASSANDRA-11904:
-----------------------------------
Attachment: system.log.2016-06-10_0514
Attached a log, search for "Memory was freed"
Note that this was on upgrade from 2.2.5 to 3.6. I can't do any further
experimentation now because we had too many issues and gave up on this upgrade
and stuck with 2.2.6. (Unfortunate, because we really wanted the jbod
improvements introduced in 3.2.)
> Exception in thread Thread[CompactionExecutor:13358,1,main]
> java.lang.AssertionError: Memory was freed
> ------------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-11904
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11904
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
> Reporter: Valentin Martinjuk
> Assignee: Marcus Eriksson
> Attachments: system.log.2016-06-10_0514
>
>
> We have Cassandra cluster 2.2.5 with two datacenters(3 nodes each).
> We observe ERRORs below on all nodes. The ERROR is repeated every minute.
> No any complains from customers. Do we have any chance to fix it without
> restart?
> {code}
> ERROR [CompactionExecutor:13996] 2016-05-26 21:20:46,700
> CassandraDaemon.java:185 - Exception in thread
> Thread[CompactionExecutor:13996,1,main]
> java.lang.AssertionError: Memory was freed
> at
> org.apache.cassandra.io.util.SafeMemory.checkBounds(SafeMemory.java:103)
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at org.apache.cassandra.io.util.Memory.getInt(Memory.java:292)
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at
> org.apache.cassandra.io.sstable.IndexSummary.getPositionInSummary(IndexSummary.java:148)
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at
> org.apache.cassandra.io.sstable.IndexSummary.fillTemporaryKey(IndexSummary.java:162)
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at
> org.apache.cassandra.io.sstable.IndexSummary.binarySearch(IndexSummary.java:121)
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.getSampleIndexesForRanges(SSTableReader.java:1398)
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.estimatedKeysForRanges(SSTableReader.java:1354)
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at
> org.apache.cassandra.db.compaction.AbstractCompactionStrategy.worthDroppingTombstones(AbstractCompactionStrategy.java:403)
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at
> org.apache.cassandra.db.compaction.LeveledCompactionStrategy.findDroppableSSTable(LeveledCompactionStrategy.java:412)
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at
> org.apache.cassandra.db.compaction.LeveledCompactionStrategy.getNextBackgroundTask(LeveledCompactionStrategy.java:101)
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at
> org.apache.cassandra.db.compaction.WrappingCompactionStrategy.getNextBackgroundTask(WrappingCompactionStrategy.java:88)
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:250)
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_74]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[na:1.8.0_74]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_74]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_74]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_74]
> ERROR [CompactionExecutor:13996] 2016-05-26 21:21:46,702
> CassandraDaemon.java:185 - Exception in thread
> Thread[CompactionExecutor:13996,1,main]
> java.lang.AssertionError: Memory was freed
> at
> org.apache.cassandra.io.util.SafeMemory.checkBounds(SafeMemory.java:103)
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at org.apache.cassandra.io.util.Memory.getInt(Memory.java:292)
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at
> org.apache.cassandra.io.sstable.IndexSummary.getPositionInSummary(IndexSummary.java:148)
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at
> org.apache.cassandra.io.sstable.IndexSummary.fillTemporaryKey(IndexSummary.java:162)
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at
> org.apache.cassandra.io.sstable.IndexSummary.binarySearch(IndexSummary.java:121)
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.getSampleIndexesForRanges(SSTableReader.java:1398)
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.estimatedKeysForRanges(SSTableReader.java:1354)
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at
> org.apache.cassandra.db.compaction.AbstractCompactionStrategy.worthDroppingTombstones(AbstractCompactionStrategy.java:403)
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at
> org.apache.cassandra.db.compaction.LeveledCompactionStrategy.findDroppableSSTable(LeveledCompactionStrategy.java:412)
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at
> org.apache.cassandra.db.compaction.LeveledCompactionStrategy.getNextBackgroundTask(LeveledCompactionStrategy.java:101)
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at
> org.apache.cassandra.db.compaction.WrappingCompactionStrategy.getNextBackgroundTask(WrappingCompactionStrategy.java:88)
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:250)
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_74]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[na:1.8.0_74]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_74]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_74]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_74]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
