Thanks for the KIP Almog. Overall, I agree that the source-topic optimization for KTables can be tricky. However, I am not a big fan of changing the configs.
It might be better to disable this optimization in the code on a per-table basis instead. For this case, one could rewrite the code: // source-topic optimization applies builder.table(); // source-topic optimization disabled builder.stream().toTable(); I agree that using `builder.table().mapValue()` should be a rejected alternative because there is still at `table()` operator and thus a source-topic subject to optimization. This issue is avoided by reading the topic as `KStream` though, and thus `stream().toTable()` seems to be a clean solution IMHO. -Matthias On 1/19/21 7:44 AM, Almog Gavra wrote: > I like that API! I'll wait a day or two to give people the opportunity to > chime in and then update the wiki. > > Thanks for the feedback, > Almog > > On Mon, Jan 18, 2021 at 10:00 PM Guozhang Wang <wangg...@gmail.com> wrote: > >> Thanks for the KIP Almog! >> >> I am in favor of the motivation of this proposal, and just throwing out an >> alternative about the API here: what do you think if we reuse the >> `TOPOLOGY_OPTIMIZATION` config but allow it it accept a list of strings >> instead of a single string, in the following semantics: >> >> * "none": disable all optimization rules. >> * "all", "-rule1", "-rule2": enable all optimization except rule1 and >> rule2. >> * "rule1", "rule2": enable only rule1 and rule2 (i.e. enumerate). >> * "none", "rule1", "rule2": not accepted, throw error. >> >> -------- >> >> Of course, for this KIP's scope itself, we can just do it for one specific >> rule of "source-changelog". I'm just wondering about the general framework >> for extending the current configuration here. >> >> Guozhang >> >> >> On Wed, Jan 13, 2021 at 10:42 AM Almog Gavra <al...@confluent.io> wrote: >> >>> Hello, >>> >>> I would like to start the discussion thread for KIP-705: >>> >>> >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-705%3A+Selectively+Disable+Topology+Optimizations >>> >>> The KIP is proposing an additional streams configuration that allows to >>> selectively disable optimizations under the topology.optimization bucket. >>> Specifically, the first optimization that can be disabled is the >>> source-topic-as-changelog optimization. >>> >>> Cheers, >>> Almog >>> >> >> >> -- >> -- Guozhang >> >
