[
https://issues.apache.org/jira/browse/KAFKA-5212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16199778#comment-16199778
]
Richard Yu edited comment on KAFKA-5212 at 10/11/17 4:10 AM:
-------------------------------------------------------------
[~hachikuji] During some study after the failure of my most recent commit,
there is this approach which one would note:
1. Due to hierarchy design, heartbeat is only accessible by ConsumerCoordinator.
2. In order to pass a variable to Fetcher, ConsumerCoordinator must pass the
necessary information through ConsumerNetworkClient.
One of the easiest ways is to define a variable with the time of the next
heartbeat (e.g. long nextHeartbeat = now + remainingMs). The variable is
defined as a private field of ConsumerNetworkClient but could be changed by
ConsumerCoordinator's methods. This way, Fetcher could have access to the next
heartbeat time.
However, this is when we encounter a problem: If fetcher's poll() is concluded
(it could not yield, thus we must wait), and if the time.milliseconds() exceeds
nextHeartbeat, we must resend a heartbeat. However, the pollHeartbeat() method
is currently only accessible in ConsumerCoordinator. Is there a way to get
around this?
was (Author: yohan123):
[~hachikuji] During some study after the failure of my most recent commit,
there is this approach which one would note:
1. Due to hierarchy design, heartbeat is only accessible by ConsumerCoordinator.
2. In order to pass a variable to Fetcher, ConsumerCoordinator must pass the
necessary information through ConsumerNetworkClient.
One of the easiest ways is to define a variable with the time of the next
heartbeat (e.g. long nextHeartbeat = now + remainingMs). The variable is
defined as a private field of ConsumerNetworkClient but is defined by
ConsumerCoordinator's methods. This way, Fetcher could have access to the next
heartbeat time.
However, this is when we encounter a problem: If fetcher's poll() is concluded
(it could not yield, thus we must wait), and if the time.milliseconds() exceeds
nextHeartbeat, we must resend a heartbeat. However, the poll() method is
currently only accessible in ConsumerCoordinator. Is there a way to get around
this?
> Consumer ListOffsets request can starve group heartbeats
> --------------------------------------------------------
>
> Key: KAFKA-5212
> URL: https://issues.apache.org/jira/browse/KAFKA-5212
> Project: Kafka
> Issue Type: Bug
> Components: consumer
> Reporter: Jason Gustafson
> Assignee: Richard Yu
> Fix For: 1.1.0, 1.0.1
>
>
> The consumer is not able to send heartbeats while it is awaiting a
> ListOffsets response. Typically this is not a problem because ListOffsets
> requests are handled quickly, but in the worst case if the request takes
> longer than the session timeout, the consumer will fall out of the group.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
