[
https://issues.apache.org/jira/browse/KAFKA-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911121#comment-16911121
]
Bruno Cadonna commented on KAFKA-8755:
--------------------------------------
bq. In optimized standby since we use the topic in both contexts the limit gets
set back to 0 and we don't seem to make any progress.
That was also my conclusion. See my [first comment on this
ticket|https://issues.apache.org/jira/browse/KAFKA-8755?focusedCommentId=16900693&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16900693]
bq. Forcing the offset to Long.MAX_VALUE when running in optimized topology
does allow things to move and the test passes, but it is not at all clear that
that is the right thing to do.
That is definitely not the right thing to do for an optimized topology because
in that case the stand-by task could overtake the active task. Afterwards when
the stand-by task is "promoted" to an active task it could have a much newer
state than the active task it is supposed to replace.
I was verifying wether the stand-by task of an optimized topology overtakes the
active task when I stumbled over this bug.
For a non-optimized topology it is OK to have an offset limit of
{{Long.MAX_VALUE}} for the changelog because the changelog contains exactly the
content of the state store. Overtaking is impossible.
> Stand-by Task of an Optimized Source Table Does Not Write Anything to its
> State Store
> -------------------------------------------------------------------------------------
>
> Key: KAFKA-8755
> URL: https://issues.apache.org/jira/browse/KAFKA-8755
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 2.4.0
> Reporter: Bruno Cadonna
> Assignee: Chris Pettitt
> Priority: Major
> Labels: newbie
> Attachments: StandbyTaskTest.java
>
>
> With the following topology:
> {code:java}
> builder.table(
> INPUT_TOPIC,
> Consumed.with(Serdes.Integer(), Serdes.Integer()),
> Materialized.<Integer, Integer, KeyValueStore<Bytes, byte[]>>as(stateName)
> )
> {code}
> and with topology optimization turned on, Kafka Streams uses the input topic
> {{INPUT_TOPIC}} as the change log topic for state store {{stateName}}. A
> stand-by task for such a topology should read from {{INPUT_TOPIC}} and should
> write the records to its state store so that the streams client that runs the
> stand-by task can take over the execution of the topology in case of a
> failure with an up-to-date replica of the state.
> Currently, the stand-by task described above reads from the input topic but
> does not write the records to its state store. Thus, after a failure the
> stand-by task cannot provide any up-to-date state store and the streams
> client needs to construct the state from scratch before it can take over the
> execution.
> The described behaviour can be reproduced with the attached test.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
