incremental backups not completely working?
Today I configured incremental backups in a test node which already has some data on it, and I found that backups are not created for STTables created by a compact: mddione@life:~/src/works/orange/Cassandra$ sudo find /var/lib/cassandra/data/one_cf /var/lib/cassandra/data/one_cf /var/lib/cassandra/data/one_cf/cf_1 /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-4-Statistics.db /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-4-Data.db /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-5-Index.db /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-4-Index.db /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-5-Filter.db /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-5-Data.db /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-4-CompressionInfo.db /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-5-CompressionInfo.db /var/lib/cassandra/data/one_cf/cf_1/backups /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-Index.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-Data.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-CompressionInfo.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-Filter.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-Index.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-Data.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-Index.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-Filter.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-Filter.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-Statistics.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-Data.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-Statistics.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-Data.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-CompressionInfo.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-Statistics.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-CompressionInfo.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-Statistics.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-Filter.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-CompressionInfo.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-Index.db /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-4-Filter.db /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-5-Statistics.db mddione@life:~/src/works/orange/Cassandra$ nodetool compact mddione@life:~/src/works/orange/Cassandra$ sudo find /var/lib/cassandra/data/one_cf /var/lib/cassandra/data/one_cf /var/lib/cassandra/data/one_cf/cf_1 /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-6-Data.db /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-6-Statistics.db /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-6-Index.db /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-6-Filter.db /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-6-CompressionInfo.db /var/lib/cassandra/data/one_cf/cf_1/backups /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-Index.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-Data.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-CompressionInfo.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-Filter.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-Index.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-Data.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-Index.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-Filter.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-Filter.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-Statistics.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-Data.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-Statistics.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-Data.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-CompressionInfo.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-Statistics.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-CompressionInfo.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-Statistics.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-Filter.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-CompressionInfo.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-Index.db as you can see, neither SSTables 4 or 6 have backups; both were created by compactions. -- Marcos Dione SysAdmin Astek Sud-Est pour FT/TGPF/OPF/PORTAIL/DOP/HEBEX @ Marco Polo 04 97 12 62 45 - mdione@orange.com _ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre
Re: incremental backups not completely working?
Incremental backups are only triggered when new data is written to disk, such as a memtable being flushed or data being streamed in from a repair or move. Compaction does not create any new data, so there's no need to back up the result. On Mon, Sep 3, 2012 at 8:45 AM, mdione@orange.com wrote: Today I configured incremental backups in a test node which already has some data on it, and I found that backups are not created for STTables created by a compact: mddione@life:~/src/works/orange/Cassandra$ sudo find /var/lib/cassandra/data/one_cf /var/lib/cassandra/data/one_cf /var/lib/cassandra/data/one_cf/cf_1 /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-4-Statistics.db /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-4-Data.db /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-5-Index.db /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-4-Index.db /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-5-Filter.db /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-5-Data.db /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-4-CompressionInfo.db /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-5-CompressionInfo.db /var/lib/cassandra/data/one_cf/cf_1/backups /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-Index.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-Data.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-CompressionInfo.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-Filter.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-Index.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-Data.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-Index.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-Filter.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-Filter.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-Statistics.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-Data.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-Statistics.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-Data.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-CompressionInfo.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-Statistics.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-CompressionInfo.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-Statistics.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-Filter.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-CompressionInfo.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-Index.db /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-4-Filter.db /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-5-Statistics.db mddione@life:~/src/works/orange/Cassandra$ nodetool compact mddione@life:~/src/works/orange/Cassandra$ sudo find /var/lib/cassandra/data/one_cf /var/lib/cassandra/data/one_cf /var/lib/cassandra/data/one_cf/cf_1 /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-6-Data.db /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-6-Statistics.db /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-6-Index.db /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-6-Filter.db /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-6-CompressionInfo.db /var/lib/cassandra/data/one_cf/cf_1/backups /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-Index.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-Data.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-CompressionInfo.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-Filter.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-Index.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-Data.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-Index.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-Filter.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-Filter.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-Statistics.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-Data.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-Statistics.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-Data.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-CompressionInfo.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-Statistics.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-CompressionInfo.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-Statistics.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-Filter.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-CompressionInfo.db /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-Index.db as you can see, neither SSTables 4 or 6 have backups; both were created by compactions. -- Marcos
