[
https://issues.apache.org/jira/browse/CASSANDRA-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16413369#comment-16413369
]
Lerh Chuan Low commented on CASSANDRA-8460:
-------------------------------------------
https://github.com/apache/cassandra/compare/trunk...juiceblender:cassandra-8460-single-csm
Bump! Asking for kind souls to review/have a look at what I have at the moment
- the branch has now been updated enough (also with tests) that should reflect
a few of the features I have decided on. I've also tried to make archiving as
non-invasive as possible but given the code organization (and also thinking
about it intuitively, it sort of makes sense) some parts of archiving had to be
known to superclasses, such as {{CompactionTask}} or {{CompactionAwareWriter}}
being aware there really are 2 different directories; one for the hot and one
for the cold.
Highlights:
- New enumeration {{DirectoryType}}. Can be either {{STANDARD}} or {{ARCHIVE}}.
- The decision on whether or not something should be archived is made in
{{TimeWindowCompactionStrategy}}. The decisions can be:
* If somebody turns off archiving, candidates always gets put into standard
* If candidates are already in archive, put them back into archiving
* Otherwise, do the standard check: Are their age past the archiving time?
- CSM is aware that there is such a thing as {{DirectoryType}}. It keeps a
running CompactionStrategy instance in every single directory; both for archive
and standard.
- People can turn off archiving at any time by turning off the archiving flags
in TWCS options. If they choose to do so, any archived SSTables, if compacted,
will be moved back to the standard directories. Otherwise, they stay in
archive. (Maybe I could write a nodetool to move archive back to standard)
- If people turn on archiving, when SSTables are next compacted they are moved
to archive.
Also included comments in various places to try to say what I am trying to do.
Finally, if you don't have time to look through the code, please at least look
through just this file: {{ArchivingCompactionTest}}. All the methods have long
names describing a feature of this archiving compaction, and please let me know
if you disagree with any of them.
There's still a lot left - dtests, Scrubber + I don't know anything about how
it works with Repair etc, it needs some functional testing. Potentially also
separate compaction executor and metrics and concurrent compactors.
Thanks!
Btw, any luck, Jon? I think I may look at writing some terraform scripts to
spin up Cassandra on Debian, which may be useful for you.
> Make it possible to move non-compacting sstables to slow/big storage in DTCS
> ----------------------------------------------------------------------------
>
> Key: CASSANDRA-8460
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8460
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Marcus Eriksson
> Assignee: Lerh Chuan Low
> Priority: Major
> Labels: doc-impacting, dtcs
> Fix For: 4.x
>
>
> It would be nice if we could configure DTCS to have a set of extra data
> directories where we move the sstables once they are older than
> max_sstable_age_days.
> This would enable users to have a quick, small SSD for hot, new data, and big
> spinning disks for data that is rarely read and never compacted.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
