changing compaction strategy
I'm trying to change compaction strategy one node at a time. I'm using jmxterm like this: `echo 'set -b org.apache.cassandra.db:type=ColumnFamilies,keyspace=my_ks,columnfamily=my_cf CompactionParametersJson \{"class":"TimeWindowCompactionStrategy","compaction_window_unit":"HOURS","compaction_window_size":"6"\}' | java -jar jmxterm-1.0-alpha-4-uber.jar --url localhost:7199` and I see this in the cassandra logs: INFO [RMI TCP Connection(37)-127.0.0.1] 2017-03-13 20:29:08,251 CompactionStrategyManager.java:841 - Switching local compaction strategy from CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={max_threshold=32, min_threshold=4}} to CompactionParams{class=org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy, options={min_threshold=4, max_threshold=32, compaction_window_unit=HOURS, compaction_window_size=6}}} After doing this, `nodetool compactionstats` shows 1 pending compaction, but none running. Also, cqlsh describe shows the old compaction strategy still. Am I missing a step?
Re: changing compaction strategy
Some more info: - running C* 3.9 - I tried `nodetool flush` on the column family this change applies to, and while it does seem to trigger compactions, there is still one pending that won't seem to run - I tried `nodetool compact` on the column family as well, with a similar affect Is there a way to tell when/if the local node has successfully updated the compaction strategy? Looking at the sstable files, it seems like they are still based on STCS but I don't know how to be sure. Appreciate any tips or suggestions! On Mon, Mar 13, 2017 at 5:30 PM, Mike Torra wrote: > I'm trying to change compaction strategy one node at a time. I'm using > jmxterm like this: > > `echo 'set -b > org.apache.cassandra.db:type=ColumnFamilies,keyspace=my_ks,columnfamily=my_cf > CompactionParametersJson \{"class":"TimeWindowCompactionStrategy", > "compaction_window_unit":"HOURS","compaction_window_size":"6"\}' | java > -jar jmxterm-1.0-alpha-4-uber.jar --url localhost:7199` > > and I see this in the cassandra logs: > > INFO [RMI TCP Connection(37)-127.0.0.1] 2017-03-13 20:29:08,251 > CompactionStrategyManager.java:841 - Switching local compaction strategy > from > CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, > options={max_threshold=32, min_threshold=4}} to CompactionParams{class=org. > apache.cassandra.db.compaction.TimeWindowCompactionStrategy, > options={min_threshold=4, max_threshold=32, compaction_window_unit=HOURS, > compaction_window_size=6}}} > > After doing this, `nodetool compactionstats` shows 1 pending compaction, > but none running. Also, cqlsh describe shows the old compaction strategy > still. Am I missing a step? >
Re: changing compaction strategy
The rogue pending task is likely a non-issue. If your jmx command went through without errors and you got the log message you can assume it worked. It won't show in the schema unless you run the ALTER statement which affects the whole cluster. If you were switching from STCS then you wouldn't expect any recompaction, as SSTables will just be calculated to be in their relevant window. On the other hand switching from LCS would generate many compactions. The only way to really tell is to confirm SSTables are expiring as you would expect them to, or verify that new SSTables are not being compacted with older ones. You might need to wait for at least one time window to pass to check this.
Impact of Changing Compaction Strategy
Hi, I need to understand whether all existing sstables are recreated/updated when we change compaction strategy from STCS to DTCS? Sstables are immutable by design but do we take an exception for such cases and update same files when an Alter statement is fired to change the compaction strategy? ThanksAnuj Sent from Yahoo Mail on Android
Re: Impact of Changing Compaction Strategy
When you change compaction strategy, nothing happens until the next flush. On the next flush, the new compaction strategy will decide what to do – if you change from STCS to DTCS, it will look at various timestamps of files, and attempt to group them by time windows based on the sstable’s minTimestamp, and your DTCS base_time_seconds and an ever-growing multiple of min_threshold Generally, many people recommend doing a STCS major before changing to DTCS (essentially to force all sstables into the oldest possible bucket). Whether or not that’s ideal for you depends on why you’re using DTCS (do you want to cut down on compaction, or are you setting yourself up for efficient TTL expiration). If it’s the latter, you should be sure you understand the impact of STCS major on your TTL use case (no data will TTL out until all of the data currently on disk is ready to expire). From: Anuj Wadehra Reply-To: "user@cassandra.apache.org" Date: Friday, January 15, 2016 at 10:16 AM To: "user@cassandra.apache.org" Subject: Impact of Changing Compaction Strategy Hi, I need to understand whether all existing sstables are recreated/updated when we change compaction strategy from STCS to DTCS? Sstables are immutable by design but do we take an exception for such cases and update same files when an Alter statement is fired to change the compaction strategy? Thanks Anuj Sent from Yahoo Mail on Android smime.p7s Description: S/MIME cryptographic signature