[
https://issues.apache.org/jira/browse/CASSANDRA-2626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13039115#comment-13039115
]
Shotaro Kamio commented on CASSANDRA-2626:
------------------------------------------
I was able to reproduce the stack overflow problem on cassandra 0.8.0 trunk.
The DataTracker creates unmodifiableSet of 'Set<SSTableReader> compacting' on
switching memtable and other operations. It creates unmodifiableSet one level
deeper every time memtables is switched. When the number of switching memtable
reaches some level and its size() is called, the stack overflow exception
occurs if the call stack exceeds stack limit.
Stack trace below (dumped by simple wrapper class MyUnmodifiableSet) shows
size() is called by constructor of HashSet in View.markCompacting()
(DataTracker.java).
If the nesting is unavoidable, a solution is to use UnmodifiableSet in apache
commons collections library. It doesn't create nests of unmodifiable collection.
Steps to reproduce the stack overflow:
1. Create single cassandra node with standard column families.
2. Disable compaction and set MemtableThroughputInMB and
MemtableOperationsInMillions to small value in order to flush memtables
frequently.
3. Insert many data for many keys.
4. When number of sstables exceeds 2000 (this parameter may vary in
environments), run "nodetool compact". The error will be logged.
-------
* Change for debug in DataTracker.java:
{quote}
public View(Memtable memtable, Set<Memtable> pendingFlush,
Set<SSTableReader> sstables, Set<SSTableReader> compacting)
\{
this.memtable = memtable;
// this.memtablesPendingFlush =
Collections.unmodifiableSet(pendingFlush);
// this.sstables = Collections.unmodifiableSet(sstables);
// this.compacting = Collections.unmodifiableSet(compacting);
this.memtablesPendingFlush = new MyUnmodifiableSet(pendingFlush);
this.sstables = new MyUnmodifiableSet(sstables);
this.compacting = new MyUnmodifiableSet(compacting);
\}
{quote}
-------
* Stacktrace by Thread.dumpStack() in MyUnmodifiableSet.
at java.lang.Thread.dumpStack(Thread.java:1249)
at
org.apache.cassandra.db.MyUnmodifiableSet.size(MyUnmodifiableSet.java:35)
at
java.util.Collections$UnmodifiableCollection.size(Collections.java:998)
at
org.apache.cassandra.db.MyUnmodifiableSet.size(MyUnmodifiableSet.java:36)
at
java.util.Collections$UnmodifiableCollection.size(Collections.java:998)
at
org.apache.cassandra.db.MyUnmodifiableSet.size(MyUnmodifiableSet.java:36)
......
at
java.util.Collections$UnmodifiableCollection.size(Collections.java:998)
at
org.apache.cassandra.db.MyUnmodifiableSet.size(MyUnmodifiableSet.java:36)
at
java.util.Collections$UnmodifiableCollection.size(Collections.java:998)
at
org.apache.cassandra.db.MyUnmodifiableSet.size(MyUnmodifiableSet.java:36)
at
java.util.Collections$UnmodifiableCollection.size(Collections.java:998)
at
org.apache.cassandra.db.MyUnmodifiableSet.size(MyUnmodifiableSet.java:36)
at
java.util.Collections$UnmodifiableCollection.size(Collections.java:998)
at
org.apache.cassandra.db.MyUnmodifiableSet.size(MyUnmodifiableSet.java:36)
at java.util.HashSet.<init>(HashSet.java:99)
at
org.apache.cassandra.db.DataTracker$View.markCompacting(DataTracker.java:495)
at
org.apache.cassandra.db.DataTracker.markCompacting(DataTracker.java:188)
at
org.apache.cassandra.db.CompactionManager$4.call(CompactionManager.java:312)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
------
> stack overflow while compacting
> -------------------------------
>
> Key: CASSANDRA-2626
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2626
> Project: Cassandra
> Issue Type: Bug
> Components: Documentation & website
> Affects Versions: 0.8 beta 1
> Reporter: Terje Marthinussen
>
> This is a trunk build from May 3.
> After adding CASSANDRA-2401, I have gotten the following on several nodes.
> I am not 100% sure right now if it is related to 2401 but it may seem likely.
> Unfortunately, as often is the case with stack overflows, I don't see the
> start of the stack
> ERROR [CompactionExecutor:17] 2011-05-09 07:56:32,479
> AbstractCassandraDaemon.java (line 112) Fatal exception in thread
> Thread[CompactionExecutor:17,1,main]
> java.lang.StackOverflowError
> at
> java.util.Collections$UnmodifiableCollection.size(Collections.java:998)
> at
> java.util.Collections$UnmodifiableCollection.size(Collections.java:998)
> at
> java.util.Collections$UnmodifiableCollection.size(Collections.java:998)
> at
> java.util.Collections$UnmodifiableCollection.size(Collections.java:998)
> at
> java.util.Collections$UnmodifiableCollection.size(Collections.java:998)
> at
> java.util.Collections$UnmodifiableCollection.size(Collections.java:998)
> at
> java.util.Collections$UnmodifiableCollection.size(Collections.java:998)
> at
> java.util.Collections$UnmodifiableCollection.size(Collections.java:998)
> at
> java.util.Collections$UnmodifiableCollection.size(Collections.java:998)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
