[
https://issues.apache.org/jira/browse/SAMZA-677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14542892#comment-14542892
]
Naveen Somasundaram commented on SAMZA-677:
-------------------------------------------
1. One possible way is set the TTL to same value as the Kafka's retention time.
Kafka will wipe out data with a fixed time, within that time interval it's OK
to restore your store.
2. We store some meta information with each key-value (preferably in the key -
but this will cause log compaction to fail), so we need to think more about
this in detail, but essentially a meta information as a part of the message
itself. And we do a restore, we skip the ones which have expired and write a
expired_key -> null, which will trigger log compaction and wipe the data out
eventually.
> Support changelog for stores with TTL
> -------------------------------------
>
> Key: SAMZA-677
> URL: https://issues.apache.org/jira/browse/SAMZA-677
> Project: Samza
> Issue Type: Improvement
> Affects Versions: 0.10.0
> Reporter: Naveen Somasundaram
>
> We support TTL for RocksDB in Samza, but what we don't support is TTL in the
> changelog itself. It becomes a little tricky because we don't have a way to
> know from the underlying store that the record has expired. This will enable
> durable storage for TTL based stores.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
