[
https://issues.apache.org/jira/browse/KAFKA-4212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17909412#comment-17909412
]
Matthias J. Sax commented on KAFKA-4212:
----------------------------------------
I just found
[https://github.com/apache/kafka/commit/88823c6016ea2e306340938994d9e122abf3c6c0#diff-6d7c3538dbdac56edfc18b1dc1d7b865c2a7850971ec4215d684dde80c8ac831]
– seems it does actually address this ticket (merge in AK 2.1)...
It's of course still not really recommended to enable TTL, but it seems it
should be possible. – Of course, the underlying changelog should be configured
with cleanup.policy "compact,delete" with a retention time similar to the TTL.
And there is also a difference in semantics: RocksDB TTL is purely wall-clock
time based, while retention time uses record timestamps which are compared to
broker wall-clock time.
But nevertheless, it seems we can resolve this ticket?
> Add a key-value store that is a TTL persistent cache
> ----------------------------------------------------
>
> Key: KAFKA-4212
> URL: https://issues.apache.org/jira/browse/KAFKA-4212
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Affects Versions: 0.10.0.1
> Reporter: Elias Levy
> Priority: Major
> Labels: api
>
> Some jobs needs to maintain as state a large set of key-values for some
> period of time. I.e. they need to maintain a TTL cache of values potentially
> larger than memory.
> Currently Kafka Streams provides non-windowed and windowed key-value stores.
> Neither is an exact fit to this use case.
> The {{RocksDBStore}}, a {{KeyValueStore}}, stores one value per key as
> required, but does not support expiration. The TTL option of RocksDB is
> explicitly not used.
> The {{RocksDBWindowsStore}}, a {{WindowsStore}}, can expire items via segment
> dropping, but it stores multiple items per key, based on their timestamp.
> But this store can be repurposed as a cache by fetching the items in reverse
> chronological order and returning the first item found.
> KAFKA-2594 introduced a fixed-capacity in-memory LRU caching store, but here
> we desire a variable-capacity memory-overflowing TTL caching store.
> Although {{RocksDBWindowsStore}} can be repurposed as a cache, it would be
> useful to have an official and proper TTL cache API and implementation.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
