[ https://issues.apache.org/jira/browse/CASSANDRA-16860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17610124#comment-17610124 ]
Paulo Motta commented on CASSANDRA-16860: ----------------------------------------- LGTM, nice work! Did a few manual tests: * Clear all by timestamp {noformat} $ docker exec -it cassandra-server nodetool listsnapshots Snapshot Details: Snapshot name Keyspace name Column family name True size Size on disk Creation time Expiration time tag1 test bla 1 KiB 6 KiB 2022-09-27T15:02:48.089Z tag2 test bla 1 KiB 6 KiB 2022-09-27T15:03:41.094Z tag3 test bla 1 KiB 6 KiB 2022-09-27T15:03:51.528Z Total TrueDiskSpaceUsed: 0 bytes $ docker exec -it cassandra-server nodetool clearsnapshot --older-than-timestamp 2022-09-27T15:02:49.089Z --all Requested clearing snapshot(s) for [all keyspaces] with [all snapshots] older than timestamp 2022-09-27T15:02:49.089Z $ docker exec -it cassandra-server nodetool listsnapshots Snapshot Details: Snapshot name Keyspace name Column family name True size Size on disk Creation time Expiration time tag2 test bla 1 KiB 6 KiB 2022-09-27T15:03:41.094Z tag3 test bla 1 KiB 6 KiB 2022-09-27T15:03:51.528Z Total TrueDiskSpaceUsed: 0 bytes {noformat} * Clear all by duration {noformat} $ docker exec -it cassandra-server nodetool listsnapshots Snapshot Details: Snapshot name Keyspace name Column family name True size Size on disk Creation time Expiration time tag4 test bla 1 KiB 6 KiB 2022-09-27T15:07:43.631Z tag2 test bla 1 KiB 6 KiB 2022-09-27T15:03:41.094Z tag3 test bla 1 KiB 6 KiB 2022-09-27T15:03:51.528Z Total TrueDiskSpaceUsed: 0 bytes $ docker exec -it cassandra-server nodetool clearsnapshot --older-than 5m --all Requested clearing snapshot(s) for [all keyspaces] with [all snapshots] older than 300 seconds. $ docker exec -it cassandra-server nodetool listsnapshots Snapshot Details: Snapshot name Keyspace name Column family name True size Size on disk Creation time Expiration time tag4 test bla 1 KiB 6 KiB 2022-09-27T15:07:43.631Z {noformat} * Clear all by duration with TTL {noformat} $ docker exec -it cassandra-server nodetool listsnapshots Snapshot Details: Snapshot name Keyspace name Column family name True size Size on disk Creation time Expiration time tag6 test bla 1.02 KiB 6.02 KiB 2022-09-27T15:11:15.323Z 2022-09-27T15:16:15.323Z tag4 test bla 1 KiB 6 KiB 2022-09-27T15:07:43.631Z tag5 test bla 1.02 KiB 6.02 KiB 2022-09-27T15:10:01.710Z 2022-09-27T15:15:01.710Z Total TrueDiskSpaceUsed: 0 bytes $ docker exec -it cassandra-server nodetool clearsnapshot --older-than 1m --all Requested clearing snapshot(s) for [all keyspaces] with [all snapshots] older than 60 seconds. $ docker exec -it cassandra-server nodetool listsnapshots Snapshot Details: Snapshot name Keyspace name Column family name True size Size on disk Creation time Expiration time tag6 test bla 1.02 KiB 6.02 KiB 2022-09-27T15:11:15.323Z 2022-09-27T15:16:15.323Z Total TrueDiskSpaceUsed: 0 bytes {noformat} * Clear all by duration and keyspace {noformat} $ docker exec -it cassandra-server nodetool listsnapshots Snapshot Details: Snapshot name Keyspace name Column family name True size Size on disk Creation time Expiration time tag6 test2 bla 1.02 KiB 5.99 KiB 2022-09-27T15:22:14.868Z 2022-09-27T15:27:14.868Z tag7 test bla 1 KiB 6 KiB 2022-09-27T15:22:32.051Z Total TrueDiskSpaceUsed: 0 bytes $ docker exec -it cassandra-server nodetool clearsnapshot --older-than 1m --all test2 Requested clearing snapshot(s) for [test2] with [all snapshots] older than 60 seconds. $ docker exec -it cassandra-server nodetool listsnapshots Snapshot Details: Snapshot name Keyspace name Column family name True size Size on disk Creation time Expiration time tag7 test bla 1 KiB 6 KiB 2022-09-27T15:22:32.051Z Total TrueDiskSpaceUsed: 0 bytes {noformat} > Add --older-than option to nodetool clearsnapshot > ------------------------------------------------- > > Key: CASSANDRA-16860 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16860 > Project: Cassandra > Issue Type: New Feature > Components: Local/Snapshots, Tool/nodetool > Reporter: Jack Casey > Assignee: Stefan Miklosovic > Priority: Normal > Fix For: 4.x > > Time Spent: 1h 50m > Remaining Estimate: 0h > > h1. Summary > Opening this issue in reference to [this WIP > PR|https://github.com/apache/cassandra/pull/1148]: > This functionality allows users of Cassandra to remove snapshots ad-hoc, > based on a TTL. This is to address the problem of snapshots accumulating. For > example, an organization I work for aims to keep snapshots for 30 days, > however we don't have any way to easily clean them after those 30 days are up. > This is similar to the goals set in: > https://issues.apache.org/jira/browse/CASSANDRA-16451 however would be > available for Cassandra 3.x. > h1. Functionality > This adds a new command to NodeTool, called {{expiresnapshot}} with the > following options: > NAME > nodetool expiresnapshots - Removes snapshots that are older than a TTL > in days > SYNOPSIS > nodetool [(-h <host> | --host <host>)] [(-p <port> | --port <port>)] > [(-pw <password> | --password <password>)] > [(-pwf <passwordFilePath> | --password-file <passwordFilePath>)] > [(-u <username> | --username <username>)] expiresnapshots [--dry-run] > (-t <ttl> | --ttl <ttl>) > OPTIONS > --dry-run > Run without actually clearing snapshots > -h <host>, --host <host> > Node hostname or ip address > -p <port>, --port <port> > Remote jmx agent port number > -pw <password>, --password <password> > Remote jmx agent password > -pwf <passwordFilePath>, --password-file <passwordFilePath> > Path to the JMX password file > -t <ttl>, --ttl <ttl> > TTL (in days) to expire snapshots > -u <username>, --username <username> > Remote jmx agent username > The snapshot date is taken by converting the default snapshot name timestamps > (epoch time in miliseconds). For this reason, snapshot names that don't > contain a timestamp in this format will not be cleared. > h1. Example Use > This Cassandra environment has a number of snapshots, a few are recent, and a > few outdated: > root@cassandra001:/cassandra# nodetool listsnapshots > Snapshot Details: > Snapshot name Keyspace name Column family name True size Size on disk > 1529173922063 users_keyspace users 362.03 KiB 362.89 KiB > 1629173909461 users_keyspace users 362.03 KiB 362.89 KiB > 1629173922063 users_keyspace users 362.03 KiB 362.89 KiB > 1599173922063 users_keyspace users 362.03 KiB 362.89 KiB > 1629173916816 users_keyspace users 362.03 KiB 362.89 KiB > Total TrueDiskSpaceUsed: 1.77 MiB > To validate the removal runs as expected, we can use the `--dry-run` option: > root@cassandra001:/cassandra# nodetool expiresnapshots --ttl 30 --dry-run > Starting simulated cleanup of snapshots older than 30 days > Clearing (dry run): 1529173922063 > Clearing (dry run): 1599173922063 > Cleared (dry run): 2 snapshots > Now that we are confident the correct snapshots will be removed, we can omit > the {{--dry-run}} flag: > root@cassandra001:/cassandra# nodetool expiresnapshots --ttl 30 > Starting cleanup of snapshots older than 30 days > Clearing: 1529173922063 > Clearing: 1599173922063 > Cleared: 2 snapshots > To confirm our changes are successful, we list the snapshots that still > remain: > root@cassandra001:/cassandra# nodetool listsnapshots > Snapshot Details: > Snapshot name Keyspace name Column family name True size Size on disk > 1629173909461 users_keyspace users 362.03 KiB 362.89 KiB > 1629173922063 users_keyspace users 362.03 KiB 362.89 KiB > 1629173916816 users_keyspace users 362.03 KiB 362.89 KiB > Total TrueDiskSpaceUsed: 1.06 MiB > h1. Next Steps > To be completed: > - Tests > - Documentation updates > I am a new to this repository, and am fuzzy on a few details even after > reading the contribution guide 😅 Any advice on the following would be greatly > appreciated! > - What branch would this type of change be merged into? Currently, I'm > targeting {{apache:trunk}} by default > - Is there a test strategy/pattern for this type of change? I was not able > to find any existing tests for similar {{nodetool}} commands > Thanks! 😄 -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org