Re: manual deletes with TWCS

[Alexander Dejanovski]

Hi John,

if all your data is TTLed then you'll be fine and purge should happen in
due time as long as your sstables don't overlap on timestamp (which can
only happen through repair mechanisms).
The tombstones will get purged when the ssatble that contain them also
fully expire.

Cheers

Le ven. 5 mai 2017 à 23:04, John Sanda  a écrit :

> This is involving TTLed data, and I actually would want to delete all
> related partitions across all time windows. Let's say I have a time series
> partitioned by day with a 7 day TTL and a window size of one day. If I
> delete partitions for the past seven days, would I still run into the issue
> of data purge being postponed?
>
> On Fri, May 5, 2017 at 4:57 PM, Jon Haddad 
> wrote:
>
>> You cannot.
>>
>> From Alex’s TLP post:
>> http://thelastpickle.com/blog/2016/12/08/TWCS-part1.html
>>
>> TWCS is no fit for workload that perform deletes on non TTLed data.
>> Consider that SSTables from different time windows will never be compacted
>> together, so data inserted on day 1 and deleted on day 2 will have the
>> tombstone and the shadowed cells living in different time windows. Unless a
>> major compaction is performed (which shouldn’t), and while the deletion
>> will seem effective when running queries, space will never be reclaimed on
>> disk.
>> Deletes can be performed on TTLed data if needed, but the partition will
>> then exist in different time windows, which will postpone actual deletion
>> from disk until both time windows fully expire.
>>
>>
>> On May 5, 2017, at 1:54 PM, John Sanda  wrote:
>>
>> How problematic is it to perform deletes when using TWCS? I am currently
>> using TWCS and have some new use cases for performing deletes. So far I
>> have avoided performing deletes, but I am wondering what issues I might run
>> into.
>>
>>
>> - John
>>
>>
>>
>
>
> --
>
> - John
>
-- 
-
Alexander Dejanovski
France
@alexanderdeja

Consultant
Apache Cassandra Consulting
http://www.thelastpickle.com

Re: manual deletes with TWCS

[John Sanda]

This is involving TTLed data, and I actually would want to delete all
related partitions across all time windows. Let's say I have a time series
partitioned by day with a 7 day TTL and a window size of one day. If I
delete partitions for the past seven days, would I still run into the issue
of data purge being postponed?

On Fri, May 5, 2017 at 4:57 PM, Jon Haddad 
wrote:

> You cannot.
>
> From Alex’s TLP post: http://thelastpickle.com/blog/2016/12/08/TWCS-
> part1.html
>
> TWCS is no fit for workload that perform deletes on non TTLed data.
> Consider that SSTables from different time windows will never be compacted
> together, so data inserted on day 1 and deleted on day 2 will have the
> tombstone and the shadowed cells living in different time windows. Unless a
> major compaction is performed (which shouldn’t), and while the deletion
> will seem effective when running queries, space will never be reclaimed on
> disk.
> Deletes can be performed on TTLed data if needed, but the partition will
> then exist in different time windows, which will postpone actual deletion
> from disk until both time windows fully expire.
>
>
> On May 5, 2017, at 1:54 PM, John Sanda  wrote:
>
> How problematic is it to perform deletes when using TWCS? I am currently
> using TWCS and have some new use cases for performing deletes. So far I
> have avoided performing deletes, but I am wondering what issues I might run
> into.
>
>
> - John
>
>
>


-- 

- John

Re: manual deletes with TWCS

[Jon Haddad]

You cannot.

From Alex’s TLP post: http://thelastpickle.com/blog/2016/12/08/TWCS-part1.html 


TWCS is no fit for workload that perform deletes on non TTLed data. Consider 
that SSTables from different time windows will never be compacted together, so 
data inserted on day 1 and deleted on day 2 will have the tombstone and the 
shadowed cells living in different time windows. Unless a major compaction is 
performed (which shouldn’t), and while the deletion will seem effective when 
running queries, space will never be reclaimed on disk.
Deletes can be performed on TTLed data if needed, but the partition will then 
exist in different time windows, which will postpone actual deletion from disk 
until both time windows fully expire.


> On May 5, 2017, at 1:54 PM, John Sanda  wrote:
> 
> How problematic is it to perform deletes when using TWCS? I am currently 
> using TWCS and have some new use cases for performing deletes. So far I have 
> avoided performing deletes, but I am wondering what issues I might run into.
> 
> 
> - John

manual deletes with TWCS

[John Sanda]

How problematic is it to perform deletes when using TWCS? I am currently
using TWCS and have some new use cases for performing deletes. So far I
have avoided performing deletes, but I am wondering what issues I might run
into.


- John