[
https://issues.apache.org/jira/browse/KAFKA-4405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15708050#comment-15708050
]
ASF GitHub Bot commented on KAFKA-4405:
---------------------------------------
GitHub user enothereska opened a pull request:
https://github.com/apache/kafka/pull/2193
KAFKA-4405: Check max.poll.records before prefetching
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/enothereska/kafka KAFKA-4405-prefetch
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/2193.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2193
----
commit 592482e0c41b14f66bd78f2f4f6247a686743280
Author: Eno Thereska <eno.there...@gmail.com>
Date: 2016-11-30T09:29:55Z
Check max.poll.records before prefetching
----
> Kafka consumer improperly send prefetch request
> -----------------------------------------------
>
> Key: KAFKA-4405
> URL: https://issues.apache.org/jira/browse/KAFKA-4405
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.10.0.1
> Reporter: ysysberserk
>
> Now kafka consumer has added max.poll.records to limit the count of messages
> return by poll().
> According to KIP-41, to implement max.poll.records, the prefetch request
> should only be sent when the total number of retained records is less than
> max.poll.records.
> But in the code of 0.10.0.1 , the consumer will send a prefetch request if it
> retained any records and never check if total number of retained records is
> less than max.poll.records..
> If max.poll.records is set to a count much less than the count of message
> fetched , the poll() loop will send a lot of requests than expected and will
> have more and more records fetched and stored in memory before they can be
> consumed.
> So before sending a prefetch request , the consumer must check if total
> number of retained records is less than max.poll.records.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
