[
https://issues.apache.org/jira/browse/KAFKA-4212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17845142#comment-17845142
]
Ayoub Omari commented on KAFKA-4212:
------------------------------------
My company was looking for such feature earlier this year. For work-around we
use a processor+punctuator similar to what [~savulchik] suggests. The problem
is that we end up doing this for every KS application in our system.
Potential suggestion: In _Topology_ class we can both _addStateStore_ and
{_}addProcessor{_}, why not overloading _addStateStore_ with additional TTL
parameter and defining the processor+punctuator within it (hence the user
wouldn't have to do it himself). If we are unsure how frequently we should
punctuate, we can give control to the user (TTL would be a class with
additional period field)
{code:java}
TTL(value: Duration, punctuateInterval: Duration){code}
> Add a key-value store that is a TTL persistent cache
> ----------------------------------------------------
>
> Key: KAFKA-4212
> URL: https://issues.apache.org/jira/browse/KAFKA-4212
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Affects Versions: 0.10.0.1
> Reporter: Elias Levy
> Priority: Major
> Labels: api
>
> Some jobs needs to maintain as state a large set of key-values for some
> period of time. I.e. they need to maintain a TTL cache of values potentially
> larger than memory.
> Currently Kafka Streams provides non-windowed and windowed key-value stores.
> Neither is an exact fit to this use case.
> The {{RocksDBStore}}, a {{KeyValueStore}}, stores one value per key as
> required, but does not support expiration. The TTL option of RocksDB is
> explicitly not used.
> The {{RocksDBWindowsStore}}, a {{WindowsStore}}, can expire items via segment
> dropping, but it stores multiple items per key, based on their timestamp.
> But this store can be repurposed as a cache by fetching the items in reverse
> chronological order and returning the first item found.
> KAFKA-2594 introduced a fixed-capacity in-memory LRU caching store, but here
> we desire a variable-capacity memory-overflowing TTL caching store.
> Although {{RocksDBWindowsStore}} can be repurposed as a cache, it would be
> useful to have an official and proper TTL cache API and implementation.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
