Phil Hargett created KAFKA-919:
----------------------------------
Summary: Disabling of auto commit is ignored during consumer group
rebalancing
Key: KAFKA-919
URL: https://issues.apache.org/jira/browse/KAFKA-919
Project: Kafka
Issue Type: Bug
Components: consumer
Affects Versions: 0.8
Environment: java 7/linux
Reporter: Phil Hargett
Assignee: Neha Narkhede
>From the mailing list:
In one of our applications using Kafka, we are using the high-level consumer to
pull messages from our topic.
Because we pull messages from topics in discrete units (e.g., an hour's worth
of messages), we want to control explicitly when offsets are committed.
Even though "auto.commit.enable" is set to false, during consumer group
rebalancing, offsets are committed anyway, regardless of the setting of this
flag.
Is this a bug? Or just a known gap in offset management? I do see plenty of
notes on the wiki suggesting future releases may enable applications using the
high-level consumer to have more fine-grained control over offset management.
I also fully realize that different applications have different needs, and
meeting all of them with a clean API can be challenging.
In the case of this application, the high-level consumer solves the problem of
locating the correct in a cluster for a given topic, so there are advantages to
using it, even if we are not using it to balance fetch load across multiple
consumers. We ideally have only 1 consumer active per consumer group, and can
tolerate some duplicate messages. But, the consumer groups make it easy for 1
consumer to recover at the correct starting point, should the prior consumer in
the group have failed before doing a commit.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
