Elias Levy created KAFKA-4212:
---------------------------------
Summary: Add a key-value store that is a TTL persistent cache
Key: KAFKA-4212
URL: https://issues.apache.org/jira/browse/KAFKA-4212
Project: Kafka
Issue Type: Improvement
Components: streams
Affects Versions: 0.10.0.1
Reporter: Elias Levy
Assignee: Guozhang Wang
Some jobs needs to maintain as state a large set of key-values for some period
of time. I.e. they need to maintain a TTL cache of values potentially larger
than memory.
Currently Kafka Streams provides non-windowed and windowed key-value stores.
Neither is an exact fit to this use case.
The {{RocksDBStore}}, a {{KeyValueStore}}, stores one value per key as
required, but does not support expiration. The TTL option of RocksDB is
explicitly not used.
The {{RocksDBWindowsStore}}, a {{WindowsStore}}, can expire items via segment
dropping, but it stores multiple items per key, based on their timestamp. But
this store can be repurposed as a cache by fetching the items in reverse
chronological order and returning the first item found.
KAFKA-2594 introduced a fixed-capacity in-memory LRU caching store, but here we
desire a variable-capacity memory-overflowing TTL caching store.
Although {{RocksDBWindowsStore}} can be repurposed as a cache, it would be
useful to have an official and proper TTL cache API and implementation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
