Matthias J. Sax created KAFKA-13500:
---------------------------------------
Summary: Consider adding a dedicated standby consumer
Key: KAFKA-13500
URL: https://issues.apache.org/jira/browse/KAFKA-13500
Project: Kafka
Issue Type: Improvement
Components: streams
Reporter: Matthias J. Sax
We currently use the restore consumer to recover state for active tasks and to
maintain standby tasks during regular processing. This setup has a few
disadvantages
# During state recovery, we might want to apply different consumer configs
compared to standby maintenance during regular processing.
# It make monitoring confusing: because we never commit offsets for changelog
topics, users can only monitor the client's "lag metric" to observer restore
progress (without the need to register a restore listener). However, if they
are interesting in a restore metric, during regular processing it would report
the standby lag, which can be rather confusing.
Because the restore consumer does not use consumer group management, it seems
to be low overhead to actually use a third consumer, because there won't be any
heartbeat thread.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
