[
https://issues.apache.org/jira/browse/KAFKA-4587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17069296#comment-17069296
]
Matthias J. Sax commented on KAFKA-4587:
----------------------------------------
It is still relevant. In 2.1 release we added `suppress()` operator (cf
[https://cwiki.apache.org/confluence/display/KAFKA/KIP-328%3A+Ability+to+suppress+updates+for+KTables])
that is one step into this direction but caching and forwarding is still
coupled in the DSL.
Overall, for cleaner semantics it would be good to decouple caching for local
stores (and writing into the changelog topics) from downstream rate control as
we have `suppress()` for downstream rate control now.
> Rethink Unification of Caching with Dedupping
> ---------------------------------------------
>
> Key: KAFKA-4587
> URL: https://issues.apache.org/jira/browse/KAFKA-4587
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Reporter: Guozhang Wang
> Priority: Major
>
> This is discussed in PR https://github.com/apache/kafka/pull/1588
> In order to support user-customizable state store suppliers in the DSL we did
> the following:
> 1) Introduce a {{TupleForwarder}} to forward tuples from cached stores that
> is wrapping user customized stores.
> 2) Narrow the scope to only dedup on forwarding if it is the default
> CachingXXStore with wrapper RocksDB.
> With this, the unification of dedupping and caching is less useful now, and
> we are complicating the inner implementations for forwarding a lot. We need
> to re-consider this feature with finer granularity of turning on / off
> caching per store, potentially with explicit triggers.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
