[ https://issues.apache.org/jira/browse/KAFKA-3843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Manikumar resolved KAFKA-3843. ------------------------------ Resolution: Auto Closed The Scala consumers have been deprecated and no further work is planned, please upgrade to the Java consumer whenever possible. > Endless consuming messages in Kafka cluster > ------------------------------------------- > > Key: KAFKA-3843 > URL: https://issues.apache.org/jira/browse/KAFKA-3843 > Project: Kafka > Issue Type: Bug > Components: consumer > Affects Versions: 0.9.0.1 > Reporter: Tomas Benc > Assignee: Neha Narkhede > Priority: Major > > We are running Kafka in cluster (3 virtual machines). Kafka is configured > min.insync.replicas = 2 and topics are configured replication factor = 3. > This configuration means, there must be at least 2 brokers of 3 in cluster up > and running to receive any messages. This works as expected. > Our consumers are high level consumers and offsets are commited manually > (auto.commit disabled) and stored in Kafka. > Reproducing the issue: > 1. Kafka cluster up and running and receives messages > 2. Consumers are disabled (messages in Kafka are in lag) > 3. Disable 2 Kafka brokers in cluster > 4. Enable consumers > Consumers are consuming messages in batch, and commiting offsets after > processing. But commit offsets fails in Kafka, because of > NotEnoughReplicasException. That is correct. What is not correct, high level > consumer has no idea, that offset are not commited and consumes same messages > again and again. > It would be helpful, that method commitOffsets() in interface > kafka.javaapi.consumer.ConsumerConnector should return some information > (return boolean or throw exception) about this operation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)