[ https://issues.apache.org/jira/browse/KAFKA-15950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jun Rao updated KAFKA-15950: ---------------------------- Description: This is a followup issue from the discussion in [https://github.com/apache/kafka/pull/14836#discussion_r1409739363]. {{KafkaEventQueue}} does de-duping and only allows one outstanding {{CommunicationEvent}} in the queue. But it seems that duplicated {{{}HeartbeatRequest{}}}s could still be generated. {{CommunicationEvent}} calls {{sendBrokerHeartbeat}} that calls the following. {code:java} _channelManager.sendRequest(new BrokerHeartbeatRequest.Builder(data), handler){code} The problem is that we have another queue in {{NodeToControllerChannelManagerImpl}} that doesn't do the de-duping. Once a {{CommunicationEvent}} is dequeued from {{{}KafkaEventQueue{}}}, a {{HeartbeatRequest}} will be queued in {{{}NodeToControllerChannelManagerImpl{}}}. At this point, another {{CommunicationEvent}} could be enqueued in {{{}KafkaEventQueue{}}}. When it's processed, another {{HeartbeatRequest}} will be queued in {{{}NodeToControllerChannelManagerImpl{}}}. This probably won't introduce long lasting duplicated {{HeartbeatRequest}} in practice since {{CommunicationEvent}} is typically queued in {{KafkaEventQueue}} for heartbeat interval. By that time, other pending {{{}HeartbeatRequest{}}}s will be processed and de-duped when enqueuing to {{{}KafkaEventQueue{}}}. However, duplicated requests could make it hard to reason about tests. was: This is a follow up issue from the discussion in https://github.com/apache/kafka/pull/14836#discussion_r1409739363. {{KafkaEventQueue}} does de-duping and only allows one outstanding {{CommunicationEvent}} in the queue. But it seems that duplicated {{{}HeartbeatRequest{}}}s could still be generated. {{CommunicationEvent}} calls {{sendBrokerHeartbeat}} that calls the following. {code:java} _channelManager.sendRequest(new BrokerHeartbeatRequest.Builder(data), handler){code} The problem is that we have another queue in {{NodeToControllerChannelManagerImpl}} that doesn't do the de-duping. Once a {{CommunicationEvent}} is dequeued from {{{}KafkaEventQueue{}}}, a {{HeartbeatRequest}} will be queued in {{{}NodeToControllerChannelManagerImpl{}}}. At this point, another {{CommunicationEvent}} could be enqueued in {{{}KafkaEventQueue{}}}. When it's processed, another {{HeartbeatRequest}} will be queued in {{{}NodeToControllerChannelManagerImpl{}}}. This probably won't introduce long lasting duplicated {{HeartbeatRequest}} in practice since {{CommunicationEvent}} is typically queued in {{KafkaEventQueue}} for heartbeat interval. By that time, other pending {{{}HeartbeatRequest{}}}s will be processed and de-duped when enqueuing to {{{}KafkaEventQueue{}}}. However, duplicated requests could make it hard to reason about tests. > Serialize broker heartbeat requests > ----------------------------------- > > Key: KAFKA-15950 > URL: https://issues.apache.org/jira/browse/KAFKA-15950 > Project: Kafka > Issue Type: Bug > Components: core > Affects Versions: 3.7.0 > Reporter: Jun Rao > Assignee: Igor Soarez > Priority: Major > > This is a followup issue from the discussion in > [https://github.com/apache/kafka/pull/14836#discussion_r1409739363]. > {{KafkaEventQueue}} does de-duping and only allows one outstanding > {{CommunicationEvent}} in the queue. But it seems that duplicated > {{{}HeartbeatRequest{}}}s could still be generated. {{CommunicationEvent}} > calls {{sendBrokerHeartbeat}} that calls the following. > {code:java} > _channelManager.sendRequest(new BrokerHeartbeatRequest.Builder(data), > handler){code} > The problem is that we have another queue in > {{NodeToControllerChannelManagerImpl}} that doesn't do the de-duping. Once a > {{CommunicationEvent}} is dequeued from {{{}KafkaEventQueue{}}}, a > {{HeartbeatRequest}} will be queued in > {{{}NodeToControllerChannelManagerImpl{}}}. At this point, another > {{CommunicationEvent}} could be enqueued in {{{}KafkaEventQueue{}}}. When > it's processed, another {{HeartbeatRequest}} will be queued in > {{{}NodeToControllerChannelManagerImpl{}}}. > This probably won't introduce long lasting duplicated {{HeartbeatRequest}} in > practice since {{CommunicationEvent}} is typically queued in > {{KafkaEventQueue}} for heartbeat interval. By that time, other pending > {{{}HeartbeatRequest{}}}s will be processed and de-duped when enqueuing to > {{{}KafkaEventQueue{}}}. However, duplicated requests could make it hard to > reason about tests. -- This message was sent by Atlassian Jira (v8.20.10#820010)