[
https://issues.apache.org/jira/browse/KAFKA-3966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kanak Biscuitwala updated KAFKA-3966:
-------------------------------------
Component/s: consumer
> Consumer briefly ignores partitions on backlogs
> -----------------------------------------------
>
> Key: KAFKA-3966
> URL: https://issues.apache.org/jira/browse/KAFKA-3966
> Project: Kafka
> Issue Type: Bug
> Components: consumer
> Affects Versions: 0.10.0.0
> Reporter: Kanak Biscuitwala
> Attachments: screenshot-1.png
>
>
> Setup:
> Kafka 0.10.0.0
> 9 brokers
> 36 partitions
> 12 members in the consumer group
> 5 producers, partitioning data roughly evenly
> max.poll.records = 10
> ~1500 events/sec
> ~500 bytes/message
> KafkaConsumer API
> In the following setup, 3 partitions are assigned to each consumer (and 12
> are served by each broker). I've noticed that the third of the three
> partitions tends to be ignored in favor of the first two on each consumer,
> even though the third partition has data available. Only until the first two
> partitions are fully caught up does the consumer return back messages from
> the third. This causes a frustrating imbalance in which the same partitions
> always fall behind.
> As a side note, this is true for all of our topics, regardless of
> partitioning strategy. The problem goes away if there are exactly as many
> consumers as partitions.
> I can attach a screenshot showing the same partitions falling behind
> (verified that they're each assigned to different nodes), if that is helpful.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
