[
https://issues.apache.org/jira/browse/KAFKA-6036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guozhang Wang resolved KAFKA-6036.
----------------------------------
Resolution: Fixed
Fix Version/s: 2.2.0
> Enable logical materialization to physical materialization
> ----------------------------------------------------------
>
> Key: KAFKA-6036
> URL: https://issues.apache.org/jira/browse/KAFKA-6036
> Project: Kafka
> Issue Type: Sub-task
> Components: streams
> Reporter: Guozhang Wang
> Assignee: Guozhang Wang
> Priority: Major
> Fix For: 2.2.0
>
>
> Today whenever users specify a queryable store name for KTable, we would
> always add a physical state store in the translated processor topology.
> For some scenarios, we should consider not physically materialize the KTable
> but only "logically" materialize it when you have some simple transformation
> operations or even join operations that generated new KTables, and which
> needs to be materialized with a state store, you can use the changelog topic
> of the previous KTable and applies the transformation logic upon restoration
> instead of creating a new changelog topic. For example:
> {code}
> table1 = builder.table("topic1");
> table2 = table1.filter(..).join(table3); // table2 needs to be materialized
> for joining
> {code}
> We can actually set the {{getter}} function of table2's materialized store,
> say {{state2}} to be reading from {{topic1}} and then apply the filter
> operator, instead of creating a new {{state2-changelog}} topic in this case.
> We can come up with a general internal impl optimizations to determine when
> to logically materialize, and whether we should actually allow users of DSL
> to "hint" whether to materialize or not (it then may need a KIP).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
