[
https://issues.apache.org/jira/browse/CASSANDRA-8399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14262576#comment-14262576
]
Benedict commented on CASSANDRA-8399:
-------------------------------------
But surely we have a bug if we're trying to compact an obsoleted sstable? If
markCompacting fails to acquire a reference we really shouldn't be compacting
it, as we're either duplicating data or reintroducing dropped data. The comment
on markCompacting even states this is a valid option, but decides against it
based on slightly erroneous logic:
{noformat}
* Note that we could acquire references on the marked sstables and release
them in
* unmarkCompacting, but since we will never call markObsolete on a sstable
marked
* as compacting (unless there is a serious bug), we can skip this.
{noformat}
This justification for not doing this doesn't seem sufficient, because
markObsolete is unrelated to releaseReference. If it's also not true that it
can be done safely without an infinite loop, it's doubly wrong :) For 2.0 I
could see us not wanting to risk new behaviour, but for 2.1 I think we should
try to get on the case and deal with it since it is essentially a bug.
One possible way of dealing with this without acquiring a reference, at least
to begin with (since it is more correct to do so, and I think we should for
2.1), is to releaseReference() in unmarkCompacting(), NOT in
markCompactedSSTablesReplaced, IFF the sstable is marked obsolete. i.e. we
simply markObsolete each of the sstables, then let unmarkCompacting cleanup.
Long term, I think we could do with redesigning and formalising this whole
process a lot more clearly. Objects that encapsulate only correct behaviour are
much better than ensuring a plethora of different but related methods are
correctly interleaved, as it's hard to retain at any one time what the
behaviour of every other possible method call is.
> Reference Counter exception when dropping user type
> ---------------------------------------------------
>
> Key: CASSANDRA-8399
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8399
> Project: Cassandra
> Issue Type: Bug
> Reporter: Philip Thompson
> Assignee: Joshua McKenzie
> Fix For: 2.1.3
>
> Attachments: 8399_fix_empty_results.txt, 8399_v2.txt, node2.log,
> ubuntu-8399.log
>
>
> When running the dtest
> {{user_types_test.py:TestUserTypes.test_type_keyspace_permission_isolation}}
> with the current 2.1-HEAD code, very frequently, but not always, when
> dropping a type, the following exception is seen:{code}
> ERROR [MigrationStage:1] 2014-12-01 13:54:54,824 CassandraDaemon.java:170 -
> Exception in thread Thread[MigrationStage:1,5,main]
> java.lang.AssertionError: Reference counter -1 for
> /var/folders/v3/z4wf_34n1q506_xjdy49gb780000gn/T/dtest-eW2RXj/test/node2/data/system/schema_keyspaces-b0f2235744583cdb9631c43e59ce3676/system-sche
> ma_keyspaces-ka-14-Data.db
> at
> org.apache.cassandra.io.sstable.SSTableReader.releaseReference(SSTableReader.java:1662)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.SSTableScanner.close(SSTableScanner.java:164)
> ~[main/:na]
> at
> org.apache.cassandra.utils.MergeIterator.close(MergeIterator.java:62)
> ~[main/:na]
> at
> org.apache.cassandra.db.ColumnFamilyStore$8.close(ColumnFamilyStore.java:1943)
> ~[main/:na]
> at
> org.apache.cassandra.db.ColumnFamilyStore.filter(ColumnFamilyStore.java:2116)
> ~[main/:na]
> at
> org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:2029)
> ~[main/:na]
> at
> org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:1963)
> ~[main/:na]
> at
> org.apache.cassandra.db.SystemKeyspace.serializedSchema(SystemKeyspace.java:744)
> ~[main/:na]
> at
> org.apache.cassandra.db.SystemKeyspace.serializedSchema(SystemKeyspace.java:731)
> ~[main/:na]
> at org.apache.cassandra.config.Schema.updateVersion(Schema.java:374)
> ~[main/:na]
> at
> org.apache.cassandra.config.Schema.updateVersionAndAnnounce(Schema.java:399)
> ~[main/:na]
> at
> org.apache.cassandra.db.DefsTables.mergeSchema(DefsTables.java:167)
> ~[main/:na]
> at
> org.apache.cassandra.db.DefinitionsUpdateVerbHandler$1.runMayThrow(DefinitionsUpdateVerbHandler.java:49)
> ~[main/:na]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> ~[main/:na]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> ~[na:1.7.0_67]
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> ~[na:1.7.0_67]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> ~[na:1.7.0_67]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_67]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_67]{code}
> Log of the node with the error is attached.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
