Hi, I have small Cassandra cluster running on boxes with 256GB SSD and 2TB HDD. Originally SSD was for system and commit log and HDD for data. But unfortunately because of nature of queries, performance was not satisfactory and to improve it, data were moved to SSD as well. Now problem is with SSD size being too small to keep all data. As there's one keyspace created a month, my idea was to move historical keyspaces to HDD and make a symlink. These historical keyspaces are not queried often so it shouldn't affect performance much. I've written simple script that rsyncs data, stops Cassandra, makes symlink and starts Cassandra node again, but I have question if there will be problem if I make hotswap - without stopping Cassandra daemon. So... rsync, rename current keyspace folder in data dir, make symlink to new location on HDD. One good thing is that while keyspace is fully feed with data, it no longer changes. I see on lsof output that even if keyspace is not queried, Cassandra keeps files opened, so I guess it's not safe to hotswap, but I'd like to make sure.
Kind regards - Krzysztof Ksiezyk