[
https://issues.apache.org/jira/browse/KAFKA-761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13597377#comment-13597377
]
Lorenzo Alberton commented on KAFKA-761:
----------------------------------------
Some more details: this happens with a moderately high number of topics (a few
hundreds), when the consumers connect and disconnect quite often to fetch a few
batches at a time.
> Fetch request randomly returns 0 bytes when there's still data in the log
> -------------------------------------------------------------------------
>
> Key: KAFKA-761
> URL: https://issues.apache.org/jira/browse/KAFKA-761
> Project: Kafka
> Issue Type: Bug
> Components: consumer
> Affects Versions: 0.7.2
> Environment: centos 6
> Reporter: Lorenzo Alberton
> Assignee: Neha Narkhede
>
> I can't track the bug down in the code, but the behaviour has been hitting us
> for quite some time (on Kafka 0.7.2): sometimes, a new Simple Consumer
> object, on a new Fetch Request, returns an empty byte buffer (or a number of
> bytes considerably smaller than the buffer size), even if there's more data
> on disk.
> This happens randomly, with no errors, at different offsets in the same
> topic. Different consumer groups processing the same topic might get stuck at
> different offsets.
> There seems to be no correlation between empty responses and the offset, or
> the buffer size, or the log file size. At first we thought it only happened
> when reaching the end of a log file, before jumping to the next file, but it
> doesn't seem to be the case. It often happens in the middle of a file.
> Simply retrying the Fetch Request multiple times works quite reliably, after
> 2 or 3 empty fetches the consumer eventually gets unstuck (we check the size
> of the queue to see if there's more data before retrying).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
