[
https://issues.apache.org/jira/browse/KAFKA-4887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15922826#comment-15922826
]
Elias Levy commented on KAFKA-4887:
-----------------------------------
It is a bit complicated. I am abusing the {{RocksDBWnidowStore}} as a TTL
cache (something I've covered in other JIRA tickets). In this particular case
the store is used to perform a join between two streams where in theory one of
the streams is keyed by two values, while the other stream is keyed only by a
single of those values. In practice the streams are only keyed by their shared
value and when items from the stream that in theory is keyed by two values, as
they are inserted with a single key, there may be collisions. To get around
that I turn on the feature to allow duplicates and the join is performed by
iterating over duplicates and matching the second key value.
> Enabling caching on a persistent window store breaks support for duplicate
> insertion
> ------------------------------------------------------------------------------------
>
> Key: KAFKA-4887
> URL: https://issues.apache.org/jira/browse/KAFKA-4887
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 0.10.2.0
> Reporter: Elias Levy
>
> {{CachingWindowStore}} and {{RocksDBWindowStore}} interact badly when
> duplicate insertion support is enabled by passing {{true}} as the fourth
> argument to {{windowed}} in the state store supplier.
> When the feature is enabled, {{RocksDBWindowStore}} correct handles
> duplicates by assigning a unique sequence number to each element on insertion
> and using the number within the key.
> When caching is enabled by calling {{enableCaching}} on the supplier,
> {{CachingWindowStore}} fails to the the same. Thus, of multiple values
> inserted with the same key, only the last one survives.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
