[ 
https://issues.apache.org/jira/browse/KAFKA-13500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17456736#comment-17456736
 ] 

Guozhang Wang commented on KAFKA-13500:
---------------------------------------

[~mjsax] What I was wondering is that if we moved the restoration / standby to 
a separate thread, do we still need to separate restoration and standbys with 
separate consumers. My thinkings are that: 1) the primary issue was that the 
same thread were doing standby maintenance and regular processing, and if 
that's no longer the case, then keeping the state recovery and the standby 
maintenance with the same configs seems not causing big issues; 2) on separate 
threads, the monitoring would be clearer as well.

> Consider adding a dedicated standby consumer
> --------------------------------------------
>
>                 Key: KAFKA-13500
>                 URL: https://issues.apache.org/jira/browse/KAFKA-13500
>             Project: Kafka
>          Issue Type: Improvement
>          Components: streams
>            Reporter: Matthias J. Sax
>            Priority: Minor
>
> We currently use the restore consumer to recover state for active tasks and 
> to maintain standby tasks during regular processing. This setup has a few 
> disadvantages
>  # During state recovery, we might want to apply different consumer configs 
> compared to standby maintenance during regular processing.
>  # It make monitoring confusing: because we never commit offsets for 
> changelog topics, users can only monitor the client's "lag metric" to 
> observer restore progress (without the need to register a restore listener). 
> However, if they are interesting in a restore metric, during regular 
> processing it would report the standby lag, which can be rather confusing.
> Because the restore consumer does not use consumer group management, it seems 
> to be low overhead to actually use a third consumer, because there won't be 
> any heartbeat thread.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Reply via email to