Stefan Miklosovic created CASSANDRA-20829:
---------------------------------------------
Summary: Secondary index implementations do not integrate with
IndexGCTransaction when compaction contains fully expired SSTables
Key: CASSANDRA-20829
URL: https://issues.apache.org/jira/browse/CASSANDRA-20829
Project: Apache Cassandra
Issue Type: Bug
Components: Feature/2i Index, Local/Compaction, Local/Compaction/TWCS
Reporter: Stefan Miklosovic
There is a test (1) which ensures that when data are TTLed and compacted,
IndexGCTransaction is aware of that and it will invoke Indexer.removeRow()
method eventually.
However, this is not work properly when we have fully expired SSTables, e.g. as
the result of a table being TWCS and having TTL on that.
The reason is that in CompactionTask, we are filtering out fully expired ones.
These then do not go to the compaction process and then they are not reacted on
in listener() which contains this logic (4). Eventually, onRowMerge in
IndexGCTransaction will make the diff and in its commit indexer.removeRow(row);
will notify 2i about its removal.
This integration is missing and it is quite a big problem because if there are
secondary index implementation the fact that SSTables were fully expired is not
propagated to them which means that data are never removed from whatever
backend they use.
The solution is to go to the compaction with fully expired SSTables as well
_but only if we detected that respective column family has some indexes_
(1)
[https://github.com/apache/cassandra/blob/cassandra-4.1/test/unit/org/apache/cassandra/index/CustomIndexTest.java#L583-L607]
(2)
[https://github.com/apache/cassandra/blob/cassandra-4.1/src/java/org/apache/cassandra/db/compaction/CompactionTask.java#L174]
(3)
[https://github.com/apache/cassandra/blob/cassandra-4.1/src/java/org/apache/cassandra/db/compaction/CompactionIterator.java#L130]
(4)
https://github.com/apache/cassandra/blob/cassandra-4.1/src/java/org/apache/cassandra/db/compaction/CompactionIterator.java#L235-L252
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
