The CF Stats are reporting you have 70GB total space taken up by SSTables, of which 55GB is live. The rest is available for deletion, AFAIK this happens when cassandra detects free space is running low. I've never dug into how/when this happens though.
With that amount of data it seems odd to fill a 300GB space during repair or compaction. Some thoughts: - What version are you using ? There were issues in early 0.7 that resulted in a lot of temp files been left around. - Take a look on disk. In the commit log directory are there a lot of files hanging around? In the data directory for your keyspace do you have any snapshots ? Is there anything in the saved caches directory? - When you've ran out of space what did the data directory look like ? How did you fix the issue? Aaron On 14 Mar 2011, at 07:10, Karl Hiramoto wrote: > Hi, > > I'm looking for advice on reducing disk usage. I've ran out of disk space > two days in a row while running a nightly scheduled nodetool repair && > nodetool compact cronjob. > > I have 6 nodes RF=3 each with 300 GB drives at a hosting company. > GCGraceSeconds = 260000 (3.1 days) > > Every column in the database has a TTL of 86400 (24 hours) to handle > deletion of stale data. 50% of the time the data is only written once, read > 0 or many times then expires. The other 50% of the time it's written multiple > times, resetting the TTL to 24 hours each time. > > > One question, since I use a TTL is it safe to set GCGraceSeconds to 0? I > don't manually delete ever, I just rely on the TTL for deletion, so are > forgotten deletes an issue? > > > > cfstats: > Read Count: 32052 > Read Latency: 3.1280378135529765 ms. > Write Count: 9704525 > Write Latency: 0.009527474760485443 ms. > Pending Tasks: 0 > Column Family: Offer > SSTable count: 12 > Space used (live): 59865089091 > Space used (total): 76111577830 > Memtable Columns Count: 39355 > Memtable Data Size: 14726313 > Memtable Switch Count: 414 > Read Count: 32052 > Read Latency: 3.128 ms. > Write Count: 9704525 > Write Latency: 0.010 ms. > Pending Tasks: 0 > Key cache capacity: 1000 > Key cache size: 1000 > Key cache hit rate: 2.4805931214280473E-4 > Row cache: disabled > Compacted row minimum size: 36 > Compacted row maximum size: 1597 > Compacted row mean size: 1319 >
