Re: time to live rows
Huh... I usually insert, compact, then flush. Apparently I've been doing it wrong my whole life. So it needs like a courtesy flush. Let me try that :) -Kal On Thu, Feb 10, 2011 at 3:06 AM, Sylvain Lebresne sylv...@datastax.com wrote: Kal, you may have to flush before compacting. If you insert then compact, then it's almost certain that the inserts are still in the memtable, and thus not compacted. On Tue, Feb 8, 2011 at 9:54 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: What's the secret recipe that I'm missing? I tried forcing compaction on my column family's JMX bean (org.apache.cassandra.db.ColumnFamilies.Main.Session) in jconsole, after gc_grace had passed (i set it to 60). Thanks, -Kal On Tue, Feb 8, 2011 at 3:46 PM, Benjamin Coverston ben.covers...@datastax.com wrote: On 2/8/11 1:23 PM, Kallin Nagelberg wrote: I did read those articles, but I didn't know know that deleting all the columns on a row was equivalent to deleting the row. Like I mentioned, I did delete all the columns from all my rows and then forced compaction before and after gc_grace had passed, but all the rows still exist. If they never disappear, then won't I run out of resources eventually? -Kal You would, if there weren't a way to get rid of tombstones: http://wiki.apache.org/cassandra/DistributedDeletes -- Ben Coverston DataStax -- The Apache Cassandra Company
Re: time to live rows
Kal, you may have to flush before compacting. If you insert then compact, then it's almost certain that the inserts are still in the memtable, and thus not compacted. On Tue, Feb 8, 2011 at 9:54 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: What's the secret recipe that I'm missing? I tried forcing compaction on my column family's JMX bean (org.apache.cassandra.db.ColumnFamilies.Main.Session) in jconsole, after gc_grace had passed (i set it to 60). Thanks, -Kal On Tue, Feb 8, 2011 at 3:46 PM, Benjamin Coverston ben.covers...@datastax.com wrote: On 2/8/11 1:23 PM, Kallin Nagelberg wrote: I did read those articles, but I didn't know know that deleting all the columns on a row was equivalent to deleting the row. Like I mentioned, I did delete all the columns from all my rows and then forced compaction before and after gc_grace had passed, but all the rows still exist. If they never disappear, then won't I run out of resources eventually? -Kal You would, if there weren't a way to get rid of tombstones: http://wiki.apache.org/cassandra/DistributedDeletes -- Ben Coverston DataStax -- The Apache Cassandra Company
Re: time to live rows
AFAIK 2nd index only works for operator EQ. -邮件原件- 发件人: Kallin Nagelberg [mailto:kallin.nagelb...@gmail.com] 发送时间: 2011年2月9日 3:36 收件人: user@cassandra.apache.org 主题: Re: time to live rows I'm thinking if this row expiry notion doesn't pan out then I might create a 'lastAccessed' column with a secondary index (i think that's right) on it. Then I can periodically run a query to find all lastAccessed columns less than a certain value and manually delete them. Sound reasonable? -Kal
Re: time to live rows
So the empty row will be ultimately removed then? Is there a way to for the GC to verify this? Thanks, -Kal On Tue, Feb 8, 2011 at 2:21 AM, Stu Hood stuh...@gmail.com wrote: The expired columns were converted into tombstones, which will live for the GC timeout. The empty row will be cleaned up when those tombstones are removed. Returning the empty row is unfortunate... we'd love to find a more appropriate solution that might not involve endless scanning. See http://wiki.apache.org/cassandra/FAQ#i_deleted_what_gives http://wiki.apache.org/cassandra/FAQ#range_ghosts On Mon, Feb 7, 2011 at 1:49 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: I also tried forcing a major compaction on the column family using JMX but the row remains. On Mon, Feb 7, 2011 at 4:43 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: I tried that but I still see the row coming back on a list columnfamily in the CLI. My concern is that there will be a pointer to an empty row for all eternity. -Kal On Mon, Feb 7, 2011 at 4:38 PM, Aaron Morton aa...@thelastpickle.com wrote: Deleting all the columns in a row via TTL has the same affect as deleting th row, the data will physically by removed during compaction. Aaron On 08 Feb, 2011,at 10:24 AM, Bill Speirs bill.spe...@gmail.com wrote: I don't think this is supported (but I could be completely wrong). However, I'd love to see this functionality as well. How would one go about requesting such a feature? Bill- On Mon, Feb 7, 2011 at 4:15 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: Hey, I have read about the new TTL columns in Cassandra 0.7. In my case I'd like to expire an entire row automatically after a certain amount of time. Is this possible as well? Thanks, -Kal
Re: time to live rows
I hope you don't consider this a hijack of the thread... What I'd like to know is the following: The GC removes TTL rows some time after they expire, at its convenience. But will they stop being returned as soon as they expire? (This is the expected behavior...) On Tue, Feb 8, 2011 at 5:11 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: So the empty row will be ultimately removed then? Is there a way to for the GC to verify this? Thanks, -Kal On Tue, Feb 8, 2011 at 2:21 AM, Stu Hood stuh...@gmail.com wrote: The expired columns were converted into tombstones, which will live for the GC timeout. The empty row will be cleaned up when those tombstones are removed. Returning the empty row is unfortunate... we'd love to find a more appropriate solution that might not involve endless scanning. See http://wiki.apache.org/cassandra/FAQ#i_deleted_what_gives http://wiki.apache.org/cassandra/FAQ#range_ghosts On Mon, Feb 7, 2011 at 1:49 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: I also tried forcing a major compaction on the column family using JMX but the row remains. On Mon, Feb 7, 2011 at 4:43 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: I tried that but I still see the row coming back on a list columnfamily in the CLI. My concern is that there will be a pointer to an empty row for all eternity. -Kal On Mon, Feb 7, 2011 at 4:38 PM, Aaron Morton aa...@thelastpickle.com wrote: Deleting all the columns in a row via TTL has the same affect as deleting th row, the data will physically by removed during compaction. Aaron On 08 Feb, 2011,at 10:24 AM, Bill Speirs bill.spe...@gmail.com wrote: I don't think this is supported (but I could be completely wrong). However, I'd love to see this functionality as well. How would one go about requesting such a feature? Bill- On Mon, Feb 7, 2011 at 4:15 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: Hey, I have read about the new TTL columns in Cassandra 0.7. In my case I'd like to expire an entire row automatically after a certain amount of time. Is this possible as well? Thanks, -Kal
Re: time to live rows
So the empty row will be ultimately removed then? Is there a way to for the GC to verify this? Set a GcGraceSecond very low and force a major compaction. Thanks, -Kal On Tue, Feb 8, 2011 at 2:21 AM, Stu Hood stuh...@gmail.com wrote: The expired columns were converted into tombstones, which will live for the GC timeout. The empty row will be cleaned up when those tombstones are removed. Returning the empty row is unfortunate... we'd love to find a more appropriate solution that might not involve endless scanning. See http://wiki.apache.org/cassandra/FAQ#i_deleted_what_gives http://wiki.apache.org/cassandra/FAQ#range_ghosts On Mon, Feb 7, 2011 at 1:49 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: I also tried forcing a major compaction on the column family using JMX but the row remains. On Mon, Feb 7, 2011 at 4:43 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: I tried that but I still see the row coming back on a list columnfamily in the CLI. My concern is that there will be a pointer to an empty row for all eternity. -Kal On Mon, Feb 7, 2011 at 4:38 PM, Aaron Morton aa...@thelastpickle.com wrote: Deleting all the columns in a row via TTL has the same affect as deleting th row, the data will physically by removed during compaction. Aaron On 08 Feb, 2011,at 10:24 AM, Bill Speirs bill.spe...@gmail.com wrote: I don't think this is supported (but I could be completely wrong). However, I'd love to see this functionality as well. How would one go about requesting such a feature? Bill- On Mon, Feb 7, 2011 at 4:15 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: Hey, I have read about the new TTL columns in Cassandra 0.7. In my case I'd like to expire an entire row automatically after a certain amount of time. Is this possible as well? Thanks, -Kal
Re: time to live rows
I hope you don't consider this a hijack of the thread... What I'd like to know is the following: The GC removes TTL rows some time after they expire, at its convenience. But will they stop being returned as soon as they expire? (This is the expected behavior...) It is the individual column that have TTL. When a column expires, it becomes a delete tombstone. Now, a row with tombstones (even only them) will show during range request. But the explanation is here: http://wiki.apache.org/cassandra/FAQ#range_ghosts On Tue, Feb 8, 2011 at 5:11 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: So the empty row will be ultimately removed then? Is there a way to for the GC to verify this? Thanks, -Kal On Tue, Feb 8, 2011 at 2:21 AM, Stu Hood stuh...@gmail.com wrote: The expired columns were converted into tombstones, which will live for the GC timeout. The empty row will be cleaned up when those tombstones are removed. Returning the empty row is unfortunate... we'd love to find a more appropriate solution that might not involve endless scanning. See http://wiki.apache.org/cassandra/FAQ#i_deleted_what_gives http://wiki.apache.org/cassandra/FAQ#range_ghosts On Mon, Feb 7, 2011 at 1:49 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: I also tried forcing a major compaction on the column family using JMX but the row remains. On Mon, Feb 7, 2011 at 4:43 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: I tried that but I still see the row coming back on a list columnfamily in the CLI. My concern is that there will be a pointer to an empty row for all eternity. -Kal On Mon, Feb 7, 2011 at 4:38 PM, Aaron Morton aa...@thelastpickle.com wrote: Deleting all the columns in a row via TTL has the same affect as deleting th row, the data will physically by removed during compaction. Aaron On 08 Feb, 2011,at 10:24 AM, Bill Speirs bill.spe...@gmail.com wrote: I don't think this is supported (but I could be completely wrong). However, I'd love to see this functionality as well. How would one go about requesting such a feature? Bill- On Mon, Feb 7, 2011 at 4:15 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: Hey, I have read about the new TTL columns in Cassandra 0.7. In my case I'd like to expire an entire row automatically after a certain amount of time. Is this possible as well? Thanks, -Kal
Re: time to live rows
I'm trying to set the gc_grace_seconds column family parameter but no luck.. I got the name of it from the comment in cassandra.yaml: # - gc_grace_seconds: specifies the time to wait before garbage #collecting tombstones (deletion markers). defaults to 864000 (10 #days). See http://wiki.apache.org/cassandra/DistributedDeletes create column family Session with comparator = UTF8Type and keys_cached = 1 and memtable_flush_after = 1440 and memtable_throughput = 32 and gc_grace_seconds = 60; error is 'No enum const class org.apache.cassandra.cli.CliUserHelp$ColumnFamilyArgument.GC_GRACE_SECONDS'. Thanks, -Kal On Tue, Feb 8, 2011 at 11:02 AM, Sylvain Lebresne sylv...@datastax.com wrote: I hope you don't consider this a hijack of the thread... What I'd like to know is the following: The GC removes TTL rows some time after they expire, at its convenience. But will they stop being returned as soon as they expire? (This is the expected behavior...) It is the individual column that have TTL. When a column expires, it becomes a delete tombstone. Now, a row with tombstones (even only them) will show during range request. But the explanation is here: http://wiki.apache.org/cassandra/FAQ#range_ghosts On Tue, Feb 8, 2011 at 5:11 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: So the empty row will be ultimately removed then? Is there a way to for the GC to verify this? Thanks, -Kal On Tue, Feb 8, 2011 at 2:21 AM, Stu Hood stuh...@gmail.com wrote: The expired columns were converted into tombstones, which will live for the GC timeout. The empty row will be cleaned up when those tombstones are removed. Returning the empty row is unfortunate... we'd love to find a more appropriate solution that might not involve endless scanning. See http://wiki.apache.org/cassandra/FAQ#i_deleted_what_gives http://wiki.apache.org/cassandra/FAQ#range_ghosts On Mon, Feb 7, 2011 at 1:49 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: I also tried forcing a major compaction on the column family using JMX but the row remains. On Mon, Feb 7, 2011 at 4:43 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: I tried that but I still see the row coming back on a list columnfamily in the CLI. My concern is that there will be a pointer to an empty row for all eternity. -Kal On Mon, Feb 7, 2011 at 4:38 PM, Aaron Morton aa...@thelastpickle.com wrote: Deleting all the columns in a row via TTL has the same affect as deleting th row, the data will physically by removed during compaction. Aaron On 08 Feb, 2011,at 10:24 AM, Bill Speirs bill.spe...@gmail.com wrote: I don't think this is supported (but I could be completely wrong). However, I'd love to see this functionality as well. How would one go about requesting such a feature? Bill- On Mon, Feb 7, 2011 at 4:15 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: Hey, I have read about the new TTL columns in Cassandra 0.7. In my case I'd like to expire an entire row automatically after a certain amount of time. Is this possible as well? Thanks, -Kal
Re: time to live rows
Did you force a major compaction (with jconsole or nodetool) after gc_grace has elapsed ? On Tue, Feb 8, 2011 at 5:46 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: Thanks, gc_grace works in the CLI. However, I'm not observing the desired effect. I am setting TTL on a single column in my column family, and I see the columns disappear when using 'list Session' (my columnfamily) in the CLI. I created the column family with gc_grace = 60, and after observing for a few minutes I am still seeing all the rows come back, none of them with columns. I was hoping the GC would delete the empty rows. -Kal On Tue, Feb 8, 2011 at 11:39 AM, Sylvain Lebresne sylv...@datastax.com wrote: Not very logically, It's actually gc_grace, not gc_grace_seconds in the CLI. On Tue, Feb 8, 2011 at 5:34 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: I'm trying to set the gc_grace_seconds column family parameter but no luck.. I got the name of it from the comment in cassandra.yaml: # - gc_grace_seconds: specifies the time to wait before garbage #collecting tombstones (deletion markers). defaults to 864000 (10 #days). See http://wiki.apache.org/cassandra/DistributedDeletes create column family Session with comparator = UTF8Type and keys_cached = 1 and memtable_flush_after = 1440 and memtable_throughput = 32 and gc_grace_seconds = 60; error is 'No enum const class org.apache.cassandra.cli.CliUserHelp$ColumnFamilyArgument.GC_GRACE_SECONDS'. Thanks, -Kal On Tue, Feb 8, 2011 at 11:02 AM, Sylvain Lebresne sylv...@datastax.com wrote: I hope you don't consider this a hijack of the thread... What I'd like to know is the following: The GC removes TTL rows some time after they expire, at its convenience. But will they stop being returned as soon as they expire? (This is the expected behavior...) It is the individual column that have TTL. When a column expires, it becomes a delete tombstone. Now, a row with tombstones (even only them) will show during range request. But the explanation is here: http://wiki.apache.org/cassandra/FAQ#range_ghosts On Tue, Feb 8, 2011 at 5:11 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: So the empty row will be ultimately removed then? Is there a way to for the GC to verify this? Thanks, -Kal On Tue, Feb 8, 2011 at 2:21 AM, Stu Hood stuh...@gmail.com wrote: The expired columns were converted into tombstones, which will live for the GC timeout. The empty row will be cleaned up when those tombstones are removed. Returning the empty row is unfortunate... we'd love to find a more appropriate solution that might not involve endless scanning. See http://wiki.apache.org/cassandra/FAQ#i_deleted_what_gives http://wiki.apache.org/cassandra/FAQ#range_ghosts On Mon, Feb 7, 2011 at 1:49 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: I also tried forcing a major compaction on the column family using JMX but the row remains. On Mon, Feb 7, 2011 at 4:43 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: I tried that but I still see the row coming back on a list columnfamily in the CLI. My concern is that there will be a pointer to an empty row for all eternity. -Kal On Mon, Feb 7, 2011 at 4:38 PM, Aaron Morton aa...@thelastpickle.com wrote: Deleting all the columns in a row via TTL has the same affect as deleting th row, the data will physically by removed during compaction. Aaron On 08 Feb, 2011,at 10:24 AM, Bill Speirs bill.spe...@gmail.com wrote: I don't think this is supported (but I could be completely wrong). However, I'd love to see this functionality as well. How would one go about requesting such a feature? Bill- On Mon, Feb 7, 2011 at 4:15 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: Hey, I have read about the new TTL columns in Cassandra 0.7. In my case I'd like to expire an entire row automatically after a certain amount of time. Is this possible as well? Thanks, -Kal
Re: time to live rows
Yes I did, on the org.apache.cassandra.db.ColumnFamilies.Main.Session object. -Kal On Tue, Feb 8, 2011 at 12:00 PM, Sylvain Lebresne sylv...@datastax.com wrote: Did you force a major compaction (with jconsole or nodetool) after gc_grace has elapsed ? On Tue, Feb 8, 2011 at 5:46 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: Thanks, gc_grace works in the CLI. However, I'm not observing the desired effect. I am setting TTL on a single column in my column family, and I see the columns disappear when using 'list Session' (my columnfamily) in the CLI. I created the column family with gc_grace = 60, and after observing for a few minutes I am still seeing all the rows come back, none of them with columns. I was hoping the GC would delete the empty rows. -Kal On Tue, Feb 8, 2011 at 11:39 AM, Sylvain Lebresne sylv...@datastax.com wrote: Not very logically, It's actually gc_grace, not gc_grace_seconds in the CLI. On Tue, Feb 8, 2011 at 5:34 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: I'm trying to set the gc_grace_seconds column family parameter but no luck.. I got the name of it from the comment in cassandra.yaml: # - gc_grace_seconds: specifies the time to wait before garbage # collecting tombstones (deletion markers). defaults to 864000 (10 # days). See http://wiki.apache.org/cassandra/DistributedDeletes create column family Session with comparator = UTF8Type and keys_cached = 1 and memtable_flush_after = 1440 and memtable_throughput = 32 and gc_grace_seconds = 60; error is 'No enum const class org.apache.cassandra.cli.CliUserHelp$ColumnFamilyArgument.GC_GRACE_SECONDS'. Thanks, -Kal On Tue, Feb 8, 2011 at 11:02 AM, Sylvain Lebresne sylv...@datastax.com wrote: I hope you don't consider this a hijack of the thread... What I'd like to know is the following: The GC removes TTL rows some time after they expire, at its convenience. But will they stop being returned as soon as they expire? (This is the expected behavior...) It is the individual column that have TTL. When a column expires, it becomes a delete tombstone. Now, a row with tombstones (even only them) will show during range request. But the explanation is here: http://wiki.apache.org/cassandra/FAQ#range_ghosts On Tue, Feb 8, 2011 at 5:11 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: So the empty row will be ultimately removed then? Is there a way to for the GC to verify this? Thanks, -Kal On Tue, Feb 8, 2011 at 2:21 AM, Stu Hood stuh...@gmail.com wrote: The expired columns were converted into tombstones, which will live for the GC timeout. The empty row will be cleaned up when those tombstones are removed. Returning the empty row is unfortunate... we'd love to find a more appropriate solution that might not involve endless scanning. See http://wiki.apache.org/cassandra/FAQ#i_deleted_what_gives http://wiki.apache.org/cassandra/FAQ#range_ghosts On Mon, Feb 7, 2011 at 1:49 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: I also tried forcing a major compaction on the column family using JMX but the row remains. On Mon, Feb 7, 2011 at 4:43 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: I tried that but I still see the row coming back on a list columnfamily in the CLI. My concern is that there will be a pointer to an empty row for all eternity. -Kal On Mon, Feb 7, 2011 at 4:38 PM, Aaron Morton aa...@thelastpickle.com wrote: Deleting all the columns in a row via TTL has the same affect as deleting th row, the data will physically by removed during compaction. Aaron On 08 Feb, 2011,at 10:24 AM, Bill Speirs bill.spe...@gmail.com wrote: I don't think this is supported (but I could be completely wrong). However, I'd love to see this functionality as well. How would one go about requesting such a feature? Bill- On Mon, Feb 7, 2011 at 4:15 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: Hey, I have read about the new TTL columns in Cassandra 0.7. In my case I'd like to expire an entire row automatically after a certain amount of time. Is this possible as well? Thanks, -Kal
Re: time to live rows
I'm thinking if this row expiry notion doesn't pan out then I might create a 'lastAccessed' column with a secondary index (i think that's right) on it. Then I can periodically run a query to find all lastAccessed columns less than a certain value and manually delete them. Sound reasonable? -Kal On Tue, Feb 8, 2011 at 12:09 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: Yes I did, on the org.apache.cassandra.db.ColumnFamilies.Main.Session object. -Kal On Tue, Feb 8, 2011 at 12:00 PM, Sylvain Lebresne sylv...@datastax.com wrote: Did you force a major compaction (with jconsole or nodetool) after gc_grace has elapsed ? On Tue, Feb 8, 2011 at 5:46 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: Thanks, gc_grace works in the CLI. However, I'm not observing the desired effect. I am setting TTL on a single column in my column family, and I see the columns disappear when using 'list Session' (my columnfamily) in the CLI. I created the column family with gc_grace = 60, and after observing for a few minutes I am still seeing all the rows come back, none of them with columns. I was hoping the GC would delete the empty rows. -Kal On Tue, Feb 8, 2011 at 11:39 AM, Sylvain Lebresne sylv...@datastax.com wrote: Not very logically, It's actually gc_grace, not gc_grace_seconds in the CLI. On Tue, Feb 8, 2011 at 5:34 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: I'm trying to set the gc_grace_seconds column family parameter but no luck.. I got the name of it from the comment in cassandra.yaml: # - gc_grace_seconds: specifies the time to wait before garbage # collecting tombstones (deletion markers). defaults to 864000 (10 # days). See http://wiki.apache.org/cassandra/DistributedDeletes create column family Session with comparator = UTF8Type and keys_cached = 1 and memtable_flush_after = 1440 and memtable_throughput = 32 and gc_grace_seconds = 60; error is 'No enum const class org.apache.cassandra.cli.CliUserHelp$ColumnFamilyArgument.GC_GRACE_SECONDS'. Thanks, -Kal On Tue, Feb 8, 2011 at 11:02 AM, Sylvain Lebresne sylv...@datastax.com wrote: I hope you don't consider this a hijack of the thread... What I'd like to know is the following: The GC removes TTL rows some time after they expire, at its convenience. But will they stop being returned as soon as they expire? (This is the expected behavior...) It is the individual column that have TTL. When a column expires, it becomes a delete tombstone. Now, a row with tombstones (even only them) will show during range request. But the explanation is here: http://wiki.apache.org/cassandra/FAQ#range_ghosts On Tue, Feb 8, 2011 at 5:11 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: So the empty row will be ultimately removed then? Is there a way to for the GC to verify this? Thanks, -Kal On Tue, Feb 8, 2011 at 2:21 AM, Stu Hood stuh...@gmail.com wrote: The expired columns were converted into tombstones, which will live for the GC timeout. The empty row will be cleaned up when those tombstones are removed. Returning the empty row is unfortunate... we'd love to find a more appropriate solution that might not involve endless scanning. See http://wiki.apache.org/cassandra/FAQ#i_deleted_what_gives http://wiki.apache.org/cassandra/FAQ#range_ghosts On Mon, Feb 7, 2011 at 1:49 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: I also tried forcing a major compaction on the column family using JMX but the row remains. On Mon, Feb 7, 2011 at 4:43 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: I tried that but I still see the row coming back on a list columnfamily in the CLI. My concern is that there will be a pointer to an empty row for all eternity. -Kal On Mon, Feb 7, 2011 at 4:38 PM, Aaron Morton aa...@thelastpickle.com wrote: Deleting all the columns in a row via TTL has the same affect as deleting th row, the data will physically by removed during compaction. Aaron On 08 Feb, 2011,at 10:24 AM, Bill Speirs bill.spe...@gmail.com wrote: I don't think this is supported (but I could be completely wrong). However, I'd love to see this functionality as well. How would one go about requesting such a feature? Bill- On Mon, Feb 7, 2011 at 4:15 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: Hey, I have read about the new TTL columns in Cassandra 0.7. In my case I'd like to expire an entire row automatically after a certain amount of time. Is this possible as well? Thanks, -Kal
RE: time to live rows
You will have the same problem. You just have to learn to ignore empty rows when you query data. See articles on delete mentioned earlier. http://wiki.apache.org/cassandra/FAQ#i_deleted_what_gives http://wiki.apache.org/cassandra/FAQ#range_ghosts -Original Message- From: Kallin Nagelberg [mailto:kallin.nagelb...@gmail.com] Sent: Tuesday, February 08, 2011 1:36 PM To: user@cassandra.apache.org Subject: Re: time to live rows I'm thinking if this row expiry notion doesn't pan out then I might create a 'lastAccessed' column with a secondary index (i think that's right) on it. Then I can periodically run a query to find all lastAccessed columns less than a certain value and manually delete them. Sound reasonable? -Kal On Tue, Feb 8, 2011 at 12:09 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: Yes I did, on the org.apache.cassandra.db.ColumnFamilies.Main.Session object. -Kal On Tue, Feb 8, 2011 at 12:00 PM, Sylvain Lebresne sylv...@datastax.com wrote: Did you force a major compaction (with jconsole or nodetool) after gc_grace has elapsed ? On Tue, Feb 8, 2011 at 5:46 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: Thanks, gc_grace works in the CLI. However, I'm not observing the desired effect. I am setting TTL on a single column in my column family, and I see the columns disappear when using 'list Session' (my columnfamily) in the CLI. I created the column family with gc_grace = 60, and after observing for a few minutes I am still seeing all the rows come back, none of them with columns. I was hoping the GC would delete the empty rows. -Kal On Tue, Feb 8, 2011 at 11:39 AM, Sylvain Lebresne sylv...@datastax.com wrote: Not very logically, It's actually gc_grace, not gc_grace_seconds in the CLI. On Tue, Feb 8, 2011 at 5:34 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: I'm trying to set the gc_grace_seconds column family parameter but no luck.. I got the name of it from the comment in cassandra.yaml: # - gc_grace_seconds: specifies the time to wait before garbage # collecting tombstones (deletion markers). defaults to 864000 (10 # days). See http://wiki.apache.org/cassandra/DistributedDeletes create column family Session with comparator = UTF8Type and keys_cached = 1 and memtable_flush_after = 1440 and memtable_throughput = 32 and gc_grace_seconds = 60; error is 'No enum const class org.apache.cassandra.cli.CliUserHelp$ColumnFamilyArgument.GC_GRACE_SECONDS'. Thanks, -Kal On Tue, Feb 8, 2011 at 11:02 AM, Sylvain Lebresne sylv...@datastax.com wrote: I hope you don't consider this a hijack of the thread... What I'd like to know is the following: The GC removes TTL rows some time after they expire, at its convenience. But will they stop being returned as soon as they expire? (This is the expected behavior...) It is the individual column that have TTL. When a column expires, it becomes a delete tombstone. Now, a row with tombstones (even only them) will show during range request. But the explanation is here: http://wiki.apache.org/cassandra/FAQ#range_ghosts On Tue, Feb 8, 2011 at 5:11 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: So the empty row will be ultimately removed then? Is there a way to for the GC to verify this? Thanks, -Kal On Tue, Feb 8, 2011 at 2:21 AM, Stu Hood stuh...@gmail.com wrote: The expired columns were converted into tombstones, which will live for the GC timeout. The empty row will be cleaned up when those tombstones are removed. Returning the empty row is unfortunate... we'd love to find a more appropriate solution that might not involve endless scanning. See http://wiki.apache.org/cassandra/FAQ#i_deleted_what_gives http://wiki.apache.org/cassandra/FAQ#range_ghosts On Mon, Feb 7, 2011 at 1:49 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: I also tried forcing a major compaction on the column family using JMX but the row remains. On Mon, Feb 7, 2011 at 4:43 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: I tried that but I still see the row coming back on a list columnfamily in the CLI. My concern is that there will be a pointer to an empty row for all eternity. -Kal On Mon, Feb 7, 2011 at 4:38 PM, Aaron Morton aa...@thelastpickle.com wrote: Deleting all the columns in a row via TTL has the same affect as deleting th row, the data will physically by removed during compaction. Aaron On 08 Feb, 2011,at 10:24 AM, Bill Speirs bill.spe...@gmail.com wrote: I don't think this is supported (but I could be completely wrong). However, I'd love to see this functionality as well. How would one
Re: time to live rows
I did read those articles, but I didn't know know that deleting all the columns on a row was equivalent to deleting the row. Like I mentioned, I did delete all the columns from all my rows and then forced compaction before and after gc_grace had passed, but all the rows still exist. If they never disappear, then won't I run out of resources eventually? -Kal On Tue, Feb 8, 2011 at 3:09 PM, Jeremiah Jordan jeremiah.jor...@morningstar.com wrote: You will have the same problem. You just have to learn to ignore empty rows when you query data. See articles on delete mentioned earlier. http://wiki.apache.org/cassandra/FAQ#i_deleted_what_gives http://wiki.apache.org/cassandra/FAQ#range_ghosts -Original Message- From: Kallin Nagelberg [mailto:kallin.nagelb...@gmail.com] Sent: Tuesday, February 08, 2011 1:36 PM To: user@cassandra.apache.org Subject: Re: time to live rows I'm thinking if this row expiry notion doesn't pan out then I might create a 'lastAccessed' column with a secondary index (i think that's right) on it. Then I can periodically run a query to find all lastAccessed columns less than a certain value and manually delete them. Sound reasonable? -Kal On Tue, Feb 8, 2011 at 12:09 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: Yes I did, on the org.apache.cassandra.db.ColumnFamilies.Main.Session object. -Kal On Tue, Feb 8, 2011 at 12:00 PM, Sylvain Lebresne sylv...@datastax.com wrote: Did you force a major compaction (with jconsole or nodetool) after gc_grace has elapsed ? On Tue, Feb 8, 2011 at 5:46 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: Thanks, gc_grace works in the CLI. However, I'm not observing the desired effect. I am setting TTL on a single column in my column family, and I see the columns disappear when using 'list Session' (my columnfamily) in the CLI. I created the column family with gc_grace = 60, and after observing for a few minutes I am still seeing all the rows come back, none of them with columns. I was hoping the GC would delete the empty rows. -Kal On Tue, Feb 8, 2011 at 11:39 AM, Sylvain Lebresne sylv...@datastax.com wrote: Not very logically, It's actually gc_grace, not gc_grace_seconds in the CLI. On Tue, Feb 8, 2011 at 5:34 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: I'm trying to set the gc_grace_seconds column family parameter but no luck.. I got the name of it from the comment in cassandra.yaml: # - gc_grace_seconds: specifies the time to wait before garbage # collecting tombstones (deletion markers). defaults to 864000 (10 # days). See http://wiki.apache.org/cassandra/DistributedDeletes create column family Session with comparator = UTF8Type and keys_cached = 1 and memtable_flush_after = 1440 and memtable_throughput = 32 and gc_grace_seconds = 60; error is 'No enum const class org.apache.cassandra.cli.CliUserHelp$ColumnFamilyArgument.GC_GRACE_SECONDS'. Thanks, -Kal On Tue, Feb 8, 2011 at 11:02 AM, Sylvain Lebresne sylv...@datastax.com wrote: I hope you don't consider this a hijack of the thread... What I'd like to know is the following: The GC removes TTL rows some time after they expire, at its convenience. But will they stop being returned as soon as they expire? (This is the expected behavior...) It is the individual column that have TTL. When a column expires, it becomes a delete tombstone. Now, a row with tombstones (even only them) will show during range request. But the explanation is here: http://wiki.apache.org/cassandra/FAQ#range_ghosts On Tue, Feb 8, 2011 at 5:11 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: So the empty row will be ultimately removed then? Is there a way to for the GC to verify this? Thanks, -Kal On Tue, Feb 8, 2011 at 2:21 AM, Stu Hood stuh...@gmail.com wrote: The expired columns were converted into tombstones, which will live for the GC timeout. The empty row will be cleaned up when those tombstones are removed. Returning the empty row is unfortunate... we'd love to find a more appropriate solution that might not involve endless scanning. See http://wiki.apache.org/cassandra/FAQ#i_deleted_what_gives http://wiki.apache.org/cassandra/FAQ#range_ghosts On Mon, Feb 7, 2011 at 1:49 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: I also tried forcing a major compaction on the column family using JMX but the row remains. On Mon, Feb 7, 2011 at 4:43 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: I tried that but I still see the row coming back on a list columnfamily in the CLI. My concern is that there will be a pointer to an empty row for all eternity. -Kal On Mon, Feb 7, 2011 at 4:38 PM, Aaron Morton aa
Re: time to live rows
On 2/8/11 1:23 PM, Kallin Nagelberg wrote: I did read those articles, but I didn't know know that deleting all the columns on a row was equivalent to deleting the row. Like I mentioned, I did delete all the columns from all my rows and then forced compaction before and after gc_grace had passed, but all the rows still exist. If they never disappear, then won't I run out of resources eventually? -Kal You would, if there weren't a way to get rid of tombstones: http://wiki.apache.org/cassandra/DistributedDeletes -- Ben Coverston DataStax -- The Apache Cassandra Company
Re: time to live rows
What's the secret recipe that I'm missing? I tried forcing compaction on my column family's JMX bean (org.apache.cassandra.db.ColumnFamilies.Main.Session) in jconsole, after gc_grace had passed (i set it to 60). Thanks, -Kal On Tue, Feb 8, 2011 at 3:46 PM, Benjamin Coverston ben.covers...@datastax.com wrote: On 2/8/11 1:23 PM, Kallin Nagelberg wrote: I did read those articles, but I didn't know know that deleting all the columns on a row was equivalent to deleting the row. Like I mentioned, I did delete all the columns from all my rows and then forced compaction before and after gc_grace had passed, but all the rows still exist. If they never disappear, then won't I run out of resources eventually? -Kal You would, if there weren't a way to get rid of tombstones: http://wiki.apache.org/cassandra/DistributedDeletes -- Ben Coverston DataStax -- The Apache Cassandra Company
time to live rows
Hey, I have read about the new TTL columns in Cassandra 0.7. In my case I'd like to expire an entire row automatically after a certain amount of time. Is this possible as well? Thanks, -Kal
Re: time to live rows
I don't think this is supported (but I could be completely wrong). However, I'd love to see this functionality as well. How would one go about requesting such a feature? Bill- On Mon, Feb 7, 2011 at 4:15 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: Hey, I have read about the new TTL columns in Cassandra 0.7. In my case I'd like to expire an entire row automatically after a certain amount of time. Is this possible as well? Thanks, -Kal
Re: time to live rows
Deleting all the columns in a row via TTL has the same affect as deleting th row, the data will physically by removed during compaction.AaronOn 08 Feb, 2011,at 10:24 AM, Bill Speirs bill.spe...@gmail.com wrote:I don't think this is supported (but I could be completely wrong). However, I'd love to see this functionality as well. How would one go about requesting such a feature? Bill- On Mon, Feb 7, 2011 at 4:15 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: Hey, I have read about the new TTL columns in Cassandra 0.7. In my case I'd like to expire an entire row automatically after a certain amount of time. Is this possible as well? Thanks, -Kal
Re: time to live rows
I tried that but I still see the row coming back on a list columnfamily in the CLI. My concern is that there will be a pointer to an empty row for all eternity. -Kal On Mon, Feb 7, 2011 at 4:38 PM, Aaron Morton aa...@thelastpickle.com wrote: Deleting all the columns in a row via TTL has the same affect as deleting th row, the data will physically by removed during compaction. Aaron On 08 Feb, 2011,at 10:24 AM, Bill Speirs bill.spe...@gmail.com wrote: I don't think this is supported (but I could be completely wrong). However, I'd love to see this functionality as well. How would one go about requesting such a feature? Bill- On Mon, Feb 7, 2011 at 4:15 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: Hey, I have read about the new TTL columns in Cassandra 0.7. In my case I'd like to expire an entire row automatically after a certain amount of time. Is this possible as well? Thanks, -Kal
Re: time to live rows
I also tried forcing a major compaction on the column family using JMX but the row remains. On Mon, Feb 7, 2011 at 4:43 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: I tried that but I still see the row coming back on a list columnfamily in the CLI. My concern is that there will be a pointer to an empty row for all eternity. -Kal On Mon, Feb 7, 2011 at 4:38 PM, Aaron Morton aa...@thelastpickle.com wrote: Deleting all the columns in a row via TTL has the same affect as deleting th row, the data will physically by removed during compaction. Aaron On 08 Feb, 2011,at 10:24 AM, Bill Speirs bill.spe...@gmail.com wrote: I don't think this is supported (but I could be completely wrong). However, I'd love to see this functionality as well. How would one go about requesting such a feature? Bill- On Mon, Feb 7, 2011 at 4:15 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: Hey, I have read about the new TTL columns in Cassandra 0.7. In my case I'd like to expire an entire row automatically after a certain amount of time. Is this possible as well? Thanks, -Kal
Re: time to live rows
The expired columns were converted into tombstones, which will live for the GC timeout. The empty row will be cleaned up when those tombstones are removed. Returning the empty row is unfortunate... we'd love to find a more appropriate solution that might not involve endless scanning. See http://wiki.apache.org/cassandra/FAQ#i_deleted_what_gives http://wiki.apache.org/cassandra/FAQ#range_ghosts On Mon, Feb 7, 2011 at 1:49 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: I also tried forcing a major compaction on the column family using JMX but the row remains. On Mon, Feb 7, 2011 at 4:43 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: I tried that but I still see the row coming back on a list columnfamily in the CLI. My concern is that there will be a pointer to an empty row for all eternity. -Kal On Mon, Feb 7, 2011 at 4:38 PM, Aaron Morton aa...@thelastpickle.com wrote: Deleting all the columns in a row via TTL has the same affect as deleting th row, the data will physically by removed during compaction. Aaron On 08 Feb, 2011,at 10:24 AM, Bill Speirs bill.spe...@gmail.com wrote: I don't think this is supported (but I could be completely wrong). However, I'd love to see this functionality as well. How would one go about requesting such a feature? Bill- On Mon, Feb 7, 2011 at 4:15 PM, Kallin Nagelberg kallin.nagelb...@gmail.com wrote: Hey, I have read about the new TTL columns in Cassandra 0.7. In my case I'd like to expire an entire row automatically after a certain amount of time. Is this possible as well? Thanks, -Kal
