[
https://issues.apache.org/jira/browse/KAFKA-2565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14902366#comment-14902366
]
Sreenivasulu Nallapati commented on KAFKA-2565:
-----------------------------------------------
Earliest Offset after all consumers completed:
{0=1057585, 1=724375, 2=1057590}
724375 supposed to be committed as 1057715 but it did not.
> Offset Commit is not working if multiple consumers try to commit the offset
> ---------------------------------------------------------------------------
>
> Key: KAFKA-2565
> URL: https://issues.apache.org/jira/browse/KAFKA-2565
> Project: Kafka
> Issue Type: Bug
> Components: consumer
> Affects Versions: 0.8.1, 0.8.2.1, 0.8.2.2
> Reporter: Sreenivasulu Nallapati
> Assignee: Neha Narkhede
>
> We are seeing some strange behaviour with commitOffsets() method of
> kafka.javaapi.consumer.ConsumerConnector. We committing the offsets to
> zookeeper at the end of the consumer batch. We are running multiple consumers
> for the same topic.
> Test details:
> 1. Created a topic with three partitions
> 2. Started three consumers (cronjob) at the same time. The aim is that
> each consumer to process one partition.
> 3. Each consumer at the end of the batch, it will call the commitOffsets()
> method on kafka.javaapi.consumer.ConsumerConnector
> 4. The offsets are getting properly updated in zookeeper if we run the
> consumers for small set (say 1000 messages) of messages.
> 5. But for larger number of messages, commit offset is not working as
> expected…sometimes only two offsets are properly committing and other one
> remains as it was.
> 6. Please see the below example
> Partition: 0 Latest Offset: 1057585
> Partition: 1 Latest Offset: 1057715
> Partition: 2 Latest Offset: 1057590
> Earliest Offset after all consumers completed: {0=1057585, 1=724375,
> 2=1057590}
> Highlighted in red supposed to be committed as 1057715 but it did not.
> Please check if it is bug with multiple consumers. When multiple consumers
> are trying to update the same path in Zookeper, is there any synchronization
> issue?
> Kafka Cluster details
> 1 zookeeper
> 3 brokers
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
