[
https://issues.apache.org/jira/browse/KAFKA-19804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18035370#comment-18035370
]
Kuan Po Tseng commented on KAFKA-19804:
---------------------------------------
The [above
comment|https://issues.apache.org/jira/browse/KAFKA-19804?focusedCommentId=18034215&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-18034215]
is not correct anymore, after
[KAFKA-18376|https://issues.apache.org/jira/browse/KAFKA-18376] was resolved,
in AsyncKafkaConsumer#pollForFetches now the pollTimeout is reduced to
retryBackoffMs (default 100ms) when there aren't any assigned partitions. See
https://github.com/apache/kafka/blob/72532b6f738d2582354232d061d4d37ae5b52bee/clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java#L1873-L1894
That means by default the first pollForFetches will retry after 100ms until the
partition assignments are back, which should mitigate the problem mentioned in
the [above
comment|https://issues.apache.org/jira/browse/KAFKA-19804?focusedCommentId=18034215&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-18034215].
I think we can move on.
> Improve heartbeat request manager initial HB interval
> ------------------------------------------------------
>
> Key: KAFKA-19804
> URL: https://issues.apache.org/jira/browse/KAFKA-19804
> Project: Kafka
> Issue Type: Task
> Components: clients, consumer
> Reporter: Lianet Magrans
> Assignee: Kuan Po Tseng
> Priority: Major
> Fix For: 4.2.0
>
>
> With KIP-848, consumer HB interval config moved to the broker, so currently,
> the consumer HB request manager starts with a 0ms interval (mainly to send a
> first HB right away after the consumer subscribe + poll). Once a response is
> received, the consumer takes the interval from the response and starts using
> it.
> That 0ms initial interval makes that the HB mgr poll continuously executes
> logic on a tight loop that may not really be needed. It mostly has to wait
> for a response (or a failure).
> Probably worse than this, is the impact on the app thread, given that
> pollTimeout takes into account the maxTimeToWait from the network thread,
> that is directly impacted by the timeToNextHeartbeat
> *
> [https://github.com/apache/kafka/blob/388739f5d847d7a16e389d9891f806547f023476/clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java#L1764-L1766]
> *
> [https://github.com/apache/kafka/blob/781bc7a54b8c4f7c86f0d6bb9ef8399d86d0735e/clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractHeartbeatRequestManager.java#L255]
> We should review and consider setting a non-zero initial interval (while we
> wait for the actual interval from the broker). One option to consider would
> be using the request timeout maybe (just a first thought)
> High level goals here would be to:
> * maintain the behaviour of sending a first HB without delay
> * ensure no unneeded activity on the HB mgr poll in the background, in tight
> loop, while we're just waiting for the first HB response with an interval
> * ensure the app thread poll timeout is not affected
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
