[
https://issues.apache.org/jira/browse/KAFKA-9177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17339336#comment-17339336
]
Andrey Polyakov commented on KAFKA-9177:
----------------------------------------
It's possible this got missed as part of KAFKA-9113? I'm seeing many messages
like this per second in our 2.6.1 Kafka Streams application logs:
{code}
{"timestamp":{"seconds":1620165908,"nanos":769000000},"thread":"myapp-StreamThread-1","severity":"DEBUG","loggerName":"org.apache.kafka.streams.processor.internals.StoreChangelogReader","message":"stream-thread
[myapp-StreamThread-1] Finished restoring all changelogs []"}
{code}
> Pause completed partitions on restore consumer
> ----------------------------------------------
>
> Key: KAFKA-9177
> URL: https://issues.apache.org/jira/browse/KAFKA-9177
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Reporter: A. Sophie Blee-Goldman
> Assignee: Guozhang Wang
> Priority: Major
> Fix For: 2.6.0
>
>
> The StoreChangelogReader is responsible for tracking and restoring active
> tasks, but once a store has finished restoring it will continue polling for
> records on that partition.
> Ordinarily this doesn't make a difference as a store is not completely
> restored until its entire changelog has been read, so there are no more
> records for poll to return anyway. But if the restoring state is actually an
> optimized source KTable, the changelog is just the source topic and poll will
> keep returning records for that partition until all stores have been restored.
> Note that this isn't a correctness issue since it's just the restore
> consumer, but it is wasteful to be polling for records and throwing them
> away. We should pause completed partitions in StoreChangelogReader so we
> don't slow down the restore consumer in reading from the unfinished changelog
> topics, and avoid wasted network.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
